Advances in Robotic Reflex Technology

What if robots could react, not just plan? A good read: https://lnkd.in/gEGSp_5U This paper proposes a Deep Reactive policy (DRP), a visuo-motor neural motion policy designed for generating reactive motions in diverse dynamic environments, operating directly on point cloud sensory input. Why does it matter? Most motion planners in robotics are either: Global optimizers: great at finding the perfect path, but they are way too slow and brittle in dynamic settings. Reactive controllers: quick on their feet, but they often get tunnel vision and crash in cluttered spaces. DRP claims to bridge the gap. And what makes it different? 1. IMPACT (transformer core): pretrained on 10 million generated expert trajectories across diverse simulation scenarios. 2. Student–teacher fine-tuning: fixes collision errors by distilling knowledge from a privileged controller (Geometric Fabrics) into a vision-based policy. 3. DCP-RMP (reactive layer): basically a reflex system that adjusts goals on the fly when obstacles move unexpectedly. Results are interesting for real-world evaluation: Static environments: Success Rate: DRP 90% | NeuralMP 30% | cuRobo-Voxels 60% Goal Blocking: Success Rate: DRP 100% | NeuralMP 6.67% | cuRobo-Voxels 3.33% Goal Blocking: Success Rate: DRP 92.86% | NeuralMP 0% | cuRobo-Voxels 0% Dynamic Goal Blocking: Success Rate: DRP 93.33% | NeuralMP 0% | cuRobo-Voxels 0% Floating Dynamic Obstacle: Success Rate: DRP 70% | NeuralMP 0% | cuRobo-Voxels 0% What stands out from the results is how well DRP handles dynamic uncertainty, the very scenarios where most planners collapse. NeuralMP, which relies on test-time optimization, simply can’t keep up with real-time changes, dropping to 0 in tasks like goal blocking and dynamic obstacles. Even cuRobo, despite being state-of-the-art in static planning, struggles once goals shift or obstacles move. DRP’s strength seems to come from its hybrid design: the transformer policy (IMPACT) gives it global context learned from millions of trajectories, while the reactive DCP-RMP layer gives it the kind of “reflexes” you normally don’t see in learned systems. The fact that it maintains 90% success even in cluttered or obstructed real-world environments suggests it isn’t just memorizing scenarios; it has genuinely learned a transferable strategy. That being said, the dependence on high-quality point clouds is a bottleneck. In noisy or occluded sensing conditions, performance may degrade. Also, results are currently limited to a single robot platform (Franka Panda). So this paper is less about replacing classical planning and more about rethinking the balance between experience and reflex. 

The dark matter of robotics is “physical commonsense.” It’s everywhere, yet hard to pin down. From gently nudging an object to make space for fingers to grasp, to placing down a slipping object to get a better grip—these tiny corrections, recoveries, and “obvious” actions are subtle and automatic. We rarely notice them, but together they account for much of our extraordinary human ability to manipulate the physical world. This is the intelligence behind dexterity. And it can be learned on robots with data—but only if it’s the right data. Much of robot data today comes from remote control teleoperation, which often breaks the human sensorimotor loop: latency, limited tactile feedback, and unnatural interfaces push operators away from fast, reactive control (System 1 thinking) and towards slow, deliberate planning (System 2 thinking) e.g. “put one finger here… then another finger there…” The resulting trajectories are stiff, stilted, and slow. The exception is data collection so seamless that it preserves natural human behavior — as though the mind of the operator can act directly through instincts refined over millions of years. At Generalist, our foundation models like GEN-0 are trained on data from lightweight handheld, ergonomic devices that let people manipulate objects almost as they would with their own hands. These devices feel balanced, and the force feedback is there — after a few minutes of doing a task, operators stop “thinking” and start reacting. The results look different. People knit, peel potatoes, paint miniatures. Not only does it expand what tasks are possible to get robot data on—the data itself captures reflexes, micro-corrections, and real-time recovery. Our models trained on this data produce robot behaviors that people consistently describe as “human-like.” This is no accident. And it is scaling. Robots that ship with physical commonsense will be better at just about everything. I wrote about why it’s been so hard for machines to acquire physical commonsense, and why large-scale, real-world physical interaction data may finally change that. Read the full article here 👉 https://lnkd.in/gCjHP-qQ

Humans don’t look at the ground every step. They rely on balance, reflexes, and a sense of their own body. This walking test from Foundation explores whether a humanoid robot can do something similar. Their robot, Phantom, is tested without cameras. Instead of vision, it relies on a reinforcement learning controller using internal sensors: IMUs across the body and torque sensors in the feet. The team then runs it through a series of intentionally messy obstacle courses. Legos. Marbles. Mouse traps. Fly paper. Even banana peels. The robot is guided forward with a PlayStation controller, but the controller only sets direction. The hard part, staying upright on unpredictable terrain, is handled entirely by the learned balance policy. What makes this interesting is the focus on proprioception. In robotics, vision often gets the spotlight. But before a robot can reason about the world, it needs a stable sense of its own body. Phantom estimates its center of mass and gravity vector in real time using its internal sensors, allowing it to react to slipping or shifting surfaces without seeing them first. There’s also a hardware constraint here. Humans have more than twenty muscles in each leg to maintain balance. Phantom achieves comparable stabilization with just six motors per leg. That puts much more pressure on the control algorithm. The broader challenge behind experiments like this is the sim-to-real gap. Policies are trained in simulation through millions of reinforcement learning trials. The real test is whether those policies hold up when the world becomes messy, noisy, and unpredictable. By deliberately pushing the robot into failure cases, the team is mapping where today’s humanoid control systems still break and where they’re starting to hold. For humanoid robotics, that boundary is exactly where the next breakthroughs usually happen. Great to see what you accomplished, Sankaet, Patrick and the entire team!!!

Robots don’t fall anymore. Kicked from the front, pushed from the back — still standing. But the most surprising moment? The floor mat slipped, the robot collapsed… and within a split second, it was standing tall again. This is more than just “balance technology.” Sensors and control systems now rival human reflexes. The design philosophy accepts that falls happen — but damage is avoided. What once took minutes to recover now takes milliseconds.