Key Elements of Adaptive Robotics Frameworks

🤖 How can we teach robots to continually learn new skills, without forgetting the old ones? 👉 CLARE ensures that as your robot gets smarter, it doesn't lose the skills it (and you as teleoperator 😅) already worked hard to master. Fine-tuning pre-trained vision-language-action models (#VLAs) on a new task has become the standard for robotic manipulation. However, since this recipe updates existing representations, it is unsuitable for long-term operation in the real world, where robots must continually adapt to new tasks and environments without forgetting the knowledge they have already acquired. Existing continual learning methods for robotics require storing previous data (exemplars), struggle with long task sequences, or rely on oracle task identifiers for deployment. We present CLARE: Continual Learning for Vision-Language-Action Models via Autonomous Adapter Routing and Expansion. CLARE is a parameter-efficient, exemplar-free framework that allows robots to continuously adapt to new tasks and environments: 🚫 No Exemplars Needed: We don't need to store past data, which is often impossible due to privacy and storage constraints. 🧠 Autonomous Routing: Our autoencoder-based mechanism dynamically selects the right adapter for the current task—no task labels required during deployment. 📉 Efficient Dynamic Expansion: The model autonomously decides when to expand its capacity, increasing parameter counts by only ~2% per task. 🏆 SOTA Results: We achieve significantly higher continual learning performance on the LIBERO benchmark compared to baselines, including methods that replay past data. 📄 Paper: https://lnkd.in/dskhxphh 🌐 Project Website: https://lnkd.in/dRDk63dP 💻 Code: https://lnkd.in/d--udZja 🤗 Hugging Face: https://lnkd.in/dswqWWUr This work has been a great collaboration with Yi Zhang, who is currently on the job market :) Angela Schoellig Technical University of Munich Learning Systems and Robotics Lab Munich Institute of Robotics and Machine Intelligence (MIRMI) at the Technical University of Munich Robotics Institute Germany #Robotics #AI #MachineLearning #ContinualLearning

This recent study provides a rigorous framework showing that intelligent behavior in robots cannot be reduced to control algorithms alone. Cognitive capability arises from the joint dynamics of morphology, actuation, sensing, materials, and continuous coupling with the physical environment. The study formalizes how morphology, sensor placement, compliant materials, closed-loop control, and environmental feedback shape perception, planning, and action. It argues that scalable robotic intelligence requires co-design of software and physical structure, not post-hoc adaptation. https://lnkd.in/g8asGqt5 #EmbodiedIntelligence #EmbodiedAI #Robotics #RoboticSystems #MorphologicalComputation #ControlTheory #SensorimotorLoops #Mechatronics #CyberPhysicalSystems #AdaptiveRobotics #SoftRobotics #Perception #Actuation #MaterialsEngineering #DigitalIndustry #Siemens #AIEngineering #WorldModels #RobotLearning

🔴 NEW ARTICLE: "VERSES AI Leads Active Inference Breakthrough in Robotics." My latest article breaks down VERSES' newest research paper titled, “Mobile Manipulation with Active Inference for Long-Horizon Rearrangement Tasks,” that was oh-so-quietly released to the public a few weeks ago (Shhh 🤫 ) This new research, led by Dr. Karl Friston's team at VERSES is the blueprint for a new robotics control stack that achieves an inner-reasoning architecture comprised of a hierarchy of multiple active inference agents within a single robot body, all working together for whole-body control to adapt and learn from moment to moment in unfamiliar environments without any offline training. ◼️ Key Takeaways: Instead of a single, monolithic Reinforcement Learning (RL) policy, their architecture creates a hierarchy of intelligent agents inside the robot, each running on the principles of Active Inference and the Free Energy Principle, outperforming current robotic paradigms on efficiency, adaptability, and safety - without the data and maintenance burden of reinforcement learning. Here’s what’s different: 🔸 Agents at Every Scale - Every joint in the robot’s body has its own “local” agent, capable of reasoning and adapting in real time. These feed into limb-level agents (e.g., arm, gripper, mobile base), which in turn feed into a whole-body agent that coordinates movement. Above that sits a high-level planner that sequences multi-step tasks. 🔸 Real-Time Adaptation - If one joint experiences unexpected resistance, the local agent adjusts instantly, while the limb-level and whole-body agents adapt the rest of the motion seamlessly — without halting the task. 🔸 Skill Composition - The robot can combine previously learned skills in new ways, enabling it to improvise when faced with novel tasks or environments. 🔸 Built-In Uncertainty Tracking - Active Inference agents model what they don’t know, enabling safer, more cautious behavior in unfamiliar situations. The result: a robot that can walk into an environment it has never seen before, understand the task, and execute it — adapting continuously as conditions change. VERSES’ broader research stack ties this directly into scalable, networked intelligence with AXIOM, Variational Bayes Gaussian Splatting (VBGS), and the Spatial Web Protocol. Together, these form the technical bridge from a single robot as a teammate to globally networked, distributed intelligent systems, where every human, robot, and system can collaborate through a shared understanding of the world. The levels of interoperability, optimization, cooperation, and co-regulation are unprecedented and staggering. Every industry will be touched by this technology. Smart cities all over the globe will come to life through this technology. ➡️ Get the full story here: 🔗 https://lnkd.in/ghFizkhn #ActiveInferenceAI #AXIOM #VBGS #Robotics #VERSESAI

Why do powerful pretrained generalist robot models fail when you move an object a few inches, swap a target, or change the scene layout? It’s usually not a lack of motor skill — it’s an alignment problem at test time. In our new paper, we introduce Vision–Language Steering (VLS): a training-free, inference-time framework that adapts frozen diffusion and flow-matching robot policies to out-of-distribution (OOD) scenarios. Key idea: Treat adaptation as an inference-time control problem. Instead of retraining policies, we steer the denoising process using: -Vision–Language Models to interpret test-time constraints -Differentiable, programmatic rewards grounded in 3D geometry -Gradient-based guidance + particle resampling for stable long-horizon execution 📊 Results CALVIN: +31% absolute success over prior steering methods LIBERO-PRO: +13% improvement on strong VLAs (π0.5, OpenVLA) Real world (Franka): Robust execution under appearance shifts, position swaps, and novel object substitutions This work suggests a broader takeaway for robotics foundation models: Scaling policies alone isn’t enough — inference-time alignment matters. 📄Paper: https://lnkd.in/g67pf5Tm 🌐 Project page: https://lnkd.in/gkPxZjXw