Course 6: Electrical Engineering and Computer Science| Acronym | Course 6 | | Homepage | | | Country | [nothing] | | ZIP, City | | | Address | | | Phone | | | Type | Academy | | Parent organization | Massachusetts Institute of Technology | | Current organization | Course 6: Electrical Engineering and Computer Science |
Open OpportunitiesThe advancement in humanoid robotics has reached a stage where mimicking complex human motions with high accuracy is crucial for tasks ranging from entertainment to human-robot interaction in dynamic environments. Traditional approaches in motion learning, particularly for humanoid robots, rely heavily on motion capture (MoCap) data. However, acquiring large amounts of high-quality MoCap data is both expensive and logistically challenging. In contrast, video footage of human activities, such as sports events or dance performances, is widely available and offers an abundant source of motion data.
Building on recent advancements in extracting and utilizing human motion from videos, such as the method proposed in WHAM (refer to the paper "Learning Physically Simulated Tennis Skills from Broadcast Videos"), this project aims to develop a system that extracts human motion from videos and applies it to teach a humanoid robot how to perform similar actions. The primary focus will be on extracting dynamic and expressive motions from videos, such as soccer player celebrations, and using these extracted motions as reference data for reinforcement learning (RL) and imitation learning on a humanoid robot. - Engineering and Technology
- Master Thesis
| In recent years, advancements in reinforcement learning have achieved remarkable success in quadruped locomotion tasks. Despite their similar structural designs, quadruped robots often require uniquely tailored reward functions for effective motion pattern development, limiting the transferability of learned behaviors across different models. This project proposes to bridge this gap by developing a unified, continuous latent representation of quadruped motions applicable across various robotic platforms. By mapping these motions onto a shared latent space, the project aims to create a versatile foundation that can be adapted to downstream tasks for specific robot configurations.
- Engineering and Technology, Information, Computing and Communication Sciences
- Master Thesis
| The remarkable agility of animals, characterized by their rapid, fluid movements and precise interaction with their environment, serves as an inspiration for advancements in legged robotics. Recent progress in the field has underscored the potential of learning-based methods for robot control. These methods streamline the development process by optimizing control mechanisms directly from sensory inputs to actuator outputs, often employing deep reinforcement learning (RL) algorithms. By training in simulated environments, these algorithms can develop locomotion skills that are subsequently transferred to physical robots. Although this approach has led to significant achievements in achieving robust locomotion, mimicking the wide range of agile capabilities observed in animals remains a significant challenge. Traditionally, manually crafted controllers have succeeded in replicating complex behaviors, but their development is labor-intensive and demands a high level of expertise in each specific skill. Reinforcement learning offers a promising alternative by potentially reducing the manual labor involved in controller development. However, crafting learning objectives that lead to the desired behaviors in robots also requires considerable expertise, specific to each skill.
- Information, Computing and Communication Sciences
- Master Thesis
| The project aims to explore curriculum learning techniques to push the limits of quadruped running speed using reinforcement learning. By systematically designing and implementing curricula that guide the learning process, the project seeks to develop a quadruped controller capable of achieving the fastest possible forward locomotion. This involves not only optimizing the learning process but also ensuring the robustness and adaptability of the learned policies across various running conditions. - Engineering and Technology
- Master Thesis
| Humanoid robots, designed to mimic the structure and behavior of humans, have seen significant advancements in kinematics, dynamics, and control systems. Teleoperation of humanoid robots involves complex control strategies to manage bipedal locomotion, balance, and interaction with environments. Research in this area has focused on developing robots that can perform tasks in environments designed for humans, from simple object manipulation to navigating complex terrains. Reinforcement learning has emerged as a powerful method for enabling robots to learn from interactions with their environment, improving their performance over time without explicit programming for every possible scenario. In the context of humanoid robotics and teleoperation, RL can be used to optimize control policies, adapt to new tasks, and improve the efficiency and safety of human-robot interactions. Key challenges include the high dimensionality of the action space, the need for safe exploration, and the transfer of learned skills across different tasks and environments. Integrating human motion tracking with reinforcement learning on humanoid robots represents a cutting-edge area of research. This approach involves using human motion data as input to train RL models, enabling the robot to learn more natural and human-like movements. The goal is to develop systems that can not only replicate human actions in real-time but also adapt and improve their responses over time through learning. Challenges in this area include ensuring real-time performance, dealing with the variability of human motion, and maintaining stability and safety of the humanoid robot.
- Information, Computing and Communication Sciences
- Master Thesis
| In recent years, advancements in reinforcement learning have achieved remarkable success in teaching robots discrete motor skills. However, this process often involves intricate reward structuring and extensive hyperparameter adjustments for each new skill, making it a time-consuming and complex endeavor. This project proposes the development of a skill generator operating within a continuous latent space. This innovative approach contrasts with the discrete skill learning methods currently prevalent in the field. By leveraging a continuous latent space, the skill generator aims to produce a diverse range of skills without the need for individualized reward designs and hyperparameter configurations for each skill. This method not only simplifies the skill generation process but also promises to enhance the adaptability and efficiency of skill learning in robotics. - Engineering and Technology, Information, Computing and Communication Sciences
- Master Thesis
| Recent advances in physically simulated humanoids have broadened their application spectrum, including animation, gaming, augmented and virtual reality (AR/VR), and robotics, showcasing significant enhancements in both performance and practicality. With the advent of motion capture (MoCap) technology and reinforcement learning (RL) techniques, these simulated humanoids are capable of replicating extensive human motion datasets, executing complex animations, and following intricate motion patterns using minimal sensor input. Nevertheless, generating such detailed and naturalistic motions requires meticulous motion data curation and the development of new physics-based policies from the ground up—a process that is not only labor-intensive but also fraught with challenges related to reward system design, dataset curation, and the learning algorithm, which can result in unnatural motions.
To circumvent these challenges, researchers have explored the use of latent spaces or skill embeddings derived from pre-trained motion controllers, facilitating their application in hierarchical RL frameworks. This method involves training a low-level policy to generate a representation space from tasks like motion imitation or adversarial learning, which a high-level policy can then navigate to produce latent codes that represent specific motor actions. This approach promotes the reuse of learned motor skills and efficient action space sampling. However, the effectiveness of this strategy is often limited by the scope of the latent space, which is traditionally based on specialized and relatively narrow motion datasets, thus limiting the range of achievable behaviors.
An alternative strategy involves employing a low-level controller as a motion imitator, using full-body kinematic motions as high-level control signals. This method is particularly prevalent in motion tracking applications, where supervised learning techniques are applied to paired input data, such as video and kinematic data. For generative tasks without paired data, RL becomes necessary, although kinematic motion presents challenges as a sampling space due to its high dimensionality and the absence of physical constraints. This necessitates the use of kinematic motion latent spaces for generative tasks and highlights the limitations of using purely kinematic signals for tasks requiring interaction with the environment or other agents, where understanding of interaction dynamics is crucial.
We would like to extend the idea of creating a low-level controller as a motion imitator to full-body motions from real-time expressive kinematic targets. - Engineering and Technology, Information, Computing and Communication Sciences
- Master Thesis
| In the burgeoning field of deep reinforcement learning (RL), agents autonomously develop complex behaviors through a process of trial and error. Yet, the application of RL across various domains faces notable hurdles, particularly in devising appropriate reward functions. Traditional approaches often resort to sparse rewards for simplicity, though these prove inadequate for training efficient agents. Consequently, real-world applications may necessitate elaborate setups, such as employing accelerometers for door interaction detection, thermal imaging for action recognition, or motion capture systems for precise object tracking. Despite these advanced solutions, crafting an ideal reward function remains challenging due to the propensity of RL algorithms to exploit the reward system in unforeseen ways. Agents might fulfill objectives in unexpected manners, highlighting the complexity of encoding desired behaviors, like adherence to social norms, into a reward function.
An alternative strategy, imitation learning, circumvents the intricacies of reward engineering by having the agent learn through the emulation of expert behavior. However, acquiring a sufficient number of high-quality demonstrations for this purpose is often impractically costly. Humans, in contrast, learn with remarkable autonomy, benefiting from intermittent guidance from educators who provide tailored feedback based on the learner's progress. This interactive learning model holds promise for artificial agents, offering a customized learning trajectory that mitigates reward exploitation without extensive reward function engineering. The challenge lies in ensuring the feedback process is both manageable for humans and rich enough to be effective. Despite its potential, the implementation of human-in-the-loop (HiL) RL remains limited in practice. Our research endeavors to significantly lessen the human labor involved in HiL learning, leveraging both unsupervised pre-training and preference-based learning to enhance agent development with minimal human intervention. - Engineering and Technology, Information, Computing and Communication Sciences
- Master Thesis
| Reinforcement learning (RL) can potentially solve complex problems in a purely data-driven manner. Still, the state-of-the-art in applying RL in robotics, relies heavily on high-fidelity simulators. While learning in simulation allows to circumvent sample complexity challenges that are common in model-free RL, even slight distribution shift ("sim-to-real gap") between simulation and the real system can cause these algorithms to easily fail. Recent advances in model-based reinforcement learning have led to superior sample efficiency, enabling online learning without a simulator. Nonetheless, learning online cannot cause any damage and should adhere to safety requirements (for obvious reasons). The proposed project aims to demonstrate how existing safe model-based RL methods can be used to solve the foregoing challenges. - Engineering and Technology
- Master Thesis
| While the history of machine learning so far largely encompasses a series of problems posed by researchers and algorithms that learn their solutions, an important question is whether the problems themselves can be generated by the algorithm at the same time as they are being solved. Such a process would in effect build its own diverse and expanding curricula, and the solutions to problems at various stages would become stepping stones towards solving even more challenging problems later in the process.
Consider the realm of legged locomotion: Training a robot via reinforcement learning to track a velocity command illustrates this concept. Initially, tracking a low velocity is simpler due to algorithm initialization and environmental setup. By manually crafting a curriculum, we can start with low-velocity targets and incrementally increase them as the robot demonstrates competence. This method works well when the difficulty correlates clearly with the target, as with higher velocities or more challenging terrains.
However, challenges arise when the relationship between task difficulty and control parameters is unclear. For instance, if a parameter dictates various human dance styles for the robot to mimic, it's not obvious whether jazz is easier than hip-hop. In such scenarios, the difficulty distribution does not align with the control parameter. How, then, can we devise an effective curriculum?
In the conventional RSL training setting for locomotion over challenging terrains, there is also a handcrafted learning schedule dictating increasingly hard terrain levels but unified with multiple different types. With a smart autonomous curriculum learning algorithm, are we able to overcome separate terrain types asynchronously and thus achieve overall better performance or higher data efficiency?
- Engineering and Technology
- Master Thesis
|
|
