Robot Learning Techniques for Diverse Systems

🦾 Great milestone for open-source robotics: pi0 & pi0.5 by Physical Intelligence are now on Hugging Face, fully ported to PyTorch in LeRobot and validated side-by-side with OpenPI for everyone to experiment with, fine-tune & deploy in their robots! π₀.₅ is a Vision-Language-Action model which represents a significant evolution from π₀ to address a big challenge in robotics: open-world generalization. While robots can perform impressive tasks in controlled environments, π₀.₅ is designed to generalize to entirely new environments and situations that were never seen during training. Generalization must occur at multiple levels: - Physical Level: Understanding how to pick up a spoon (by the handle) or plate (by the edge), even with unseen objects in cluttered environments - Semantic Level: Understanding task semantics, where to put clothes and shoes (laundry hamper, not on the bed), and what tools are appropriate for cleaning spills - Environmental Level: Adapting to "messy" real-world environments like homes, grocery stores, offices, and hospitals The breakthrough innovation in π₀.₅ is co-training on heterogeneous data sources. The model learns from: - Multimodal Web Data: Image captioning, visual question answering, object detection - Verbal Instructions: Humans coaching robots through complex tasks step-by-step - Subtask Commands: High-level semantic behavior labels (e.g., "pick up the pillow" for an unmade bed) - Cross-Embodiment Robot Data: Data from various robot platforms with different capabilities - Multi-Environment Data: Static robots deployed across many different homes - Mobile Manipulation Data: ~400 hours of mobile robot demonstrations This diverse training mixture creates a "curriculum" that enables generalization across physical, visual, and semantic levels simultaneously. Huge thanks to the Physical Intelligence team & contributors Model: https://lnkd.in/eAEr7Yk6 LeRobot: https://lnkd.in/ehzQ3Mqy

VLA models are systems that combine three capabilities into one framework: seeing the world through cameras, understanding natural language instructions like "pick up the red apple," and generating the actual motor commands to make a robot do it. Before these unified models existed, robots had separate modules for vision, language, and movement that were stitched together with manual engineering, which made them brittle and unable to handle new situations. This review paper covers over 80 VLA models published in the past three years, organizing them into a taxonomy based on their architectures—some use a single end-to-end network, others separate high-level planning from low-level control, some use diffusion models for smoother action sequences. The paper walks through how these models are trained using both internet data and robot demonstration datasets, then maps out where they're being applied. The later sections lay out the concrete technical problems that remain unsolved. Read online with an AI tutor: https://lnkd.in/eZdzYfdu PDF: https://lnkd.in/ezzncewE

Can my robot cook my food, tidy my messy table, rearrange my dresser and do much much more without ANY demos or real-world training data? Introducing ManipGen: A generalist agent for manipulation that can solve long-horizon robotics tasks entirely zero shot, from text input! Key idea: for many manipulation tasks of interest, they can be decomposed into two phases, contact-free reaching (aka motion planning!) and contact-rich local interaction. The latter is hard to learn, and we take a sim2real transfer approach! We define local policies, which operate in a local region around an object of interest. They are uniquely well-suited to generalization (see below!) and sim2real transfer. This is because they are invariant to: 1) Absolute pose 2) Skill orders 3) Environment configurations As an overview, our approach 1) acquires generalist behaviors for local skills at scale using RL 2) distills these behaviors into visuomotor policies using multitask DAgger and 3) deploys local policies in the real-world using VLMs and motion planning. Phase 1: Train state-based, single-object policies to acquire skills such as picking, placing, opening and closing. We train policies using PPO across thousands of objects, designing reward and observation spaces for efficient learning and effective sim2real transfer. Phase 2: We need visuomotor policies to deploy on robots! We distill single-object experts into multi-task policies using online imitation learning (aka DAgger) that observe local visual (wrist cam) input with edge and hole augmentation to match real-world depth noise. To deploy local policies in the real-world, we decompose the task into components (GPT-4o), estimate where to go using Grounded SAM, and motion plan using Neural MP. For control, we use Industreallib from NVIDIA, an excellent library for sim2real transfer! ManipGen can solve long-horizon tasks in the real-world entirely zero-shot generalizing across objects, poses, environments and scene configurations! We outperform SOTA approaches such as SayCan, OpenVLA, LLMTrajGen and VoxPoser across 50 tasks by 36%, 76%, 62% and 60%! ManipGen exhibits exciting capabilities such as performing manipulation in tight spaces and with clutter, entirely zero-shot! From putting items on the shelf, carefully extracting the red pepper from clutter and putting large items in drawers, ManipGen is quite capable. By training local policies at scale on thousands of objects, ManipGen generalizes to some pretty challenging out of distribution objects that don’t look anything like what was in training, such as pliers and the clamps as well as deformable objects such as the wire. This work was done at Carnegie Mellon University Robotics Institute, with co-lead Min Liu, as well as Deepak Pathak and Russ Salakhutdinov and in collaboration with Walter Talbott, Chen Chen, Ph.D., and Jian Zhang from Apple.  Paper, videos and code (coming soon!) at https://lnkd.in/ekjWPXHM

Humanoid robots need to adapt to different tasks, like moving around, handling objects while walking, and working on tables, each requiring a unique way to control the robot’s body. For instance, moving around focuses on tracking how fast the robot's base is moving, while working at a table relies more on controlling the robot's arm movements. Many current methods train robots with specific controls for each task, making it hard for them to switch between tasks smoothly. This new approach suggests using whole-body motion imitation to create a common base that can work for all tasks, helping robots learn general skills that apply to different types of control. With this idea, researchers developed HOVER (Humanoid Versatile Controller), a system that combines different control modes into one shared setup. HOVER allows robots to switch between tasks without losing the strengths needed for each one, making humanoid control easier and more flexible. This approach removes the need to retrain the robot for each task, making it more efficient and adaptable for future uses. The diverse team of researchers that developed HOVER come from: NVIDIA,  Carnegie Mellon University, University of California, Berkeley, The University of Texas at Austin, and UC San Diego. 📝 Research Paper: https://lnkd.in/eMatAxMu 📊 Project Page: https://lnkd.in/eY4gzmme #robotics #research

Articulated-Body Dynamics Network: Dynamics-Grounded Prior for Robot Learning Arxiv: https://lnkd.in/ePQe8ZuF Project: [Link not provided] 🔁 At a Glance 💡 Goal: Incorporate the dynamics structure of articulated robots into control policies to improve learning efficiency. ⚙️ Approach: - Inertia propagation: Adapted from the Articulated Body Algorithm, propagating inertial quantities. - Learnable parameters: Replace physical quantities with learnable ones. - Graph neural network: Embeds dynamic propagation physics into policy architecture. - Bottom-up message passing: Mimics forward dynamics accumulation. 📈 Impact (Key Results) 🧪 Sample efficiency & generalization: Outperforms baselines across diverse robots & tasks. - Validation on real robots shows robust sim-to-real transfer. 🔄 Robustness to dynamics shifts: Maintains performance with increased mass & different terrains. - Visualizations show learnt link representations capture meaningful physical relationships. 🤖 Model extensions & efficiency: - Compatible with model-based RL & dynamics prediction. - Computationally efficient inference suitable for real-time control. 🔬 Experiments 🧪 Benchmarks: Genesis, SAPIEN, MuJoCo, ManiSkill. 🎯 Tasks: Locomotion, velocity tracking, standing. 🦾 Setup: Sim-to-real on Unitree G1 & Go2, NVIDIA RTX 4090 hardware. 📐 Inputs: Proprioception, velocity commands, foot contacts, images (future work). 🛠 How to Implement 1️⃣ Extract robot kinematic tree. 2️⃣ Encode observations into link features. 3️⃣ Perform dynamics-inspired bottom-up message passing. 4️⃣ Decode actions from link representations. 5️⃣ Train with PPO & orthogonality regularization. 📦 Deployment Benefits ✅ Improved sample efficiency & robustness. ✅ Real-time inference on onboard hardware. ✅ Enhanced generalization to dynamics variations. ✅ Compatible with sim-to-real transfer pipelines. 📣 Takeaway This physics-grounded GNN architecture provides an effective inductive bias for articulated robot control. It captures inertial propagation, boosting learning speed, robustness, and transferability. Advances in physics-informed policies open new horizons for efficient, adaptable robot behaviors. Follow me to know more about AI, ML and Robotics!