Robotics Innovation Demonstration Video

Happy to share our latest paper, "Enabling Novel Mission Operations and Interactions with ROSA: The Robot Operating System Agent". This work was led by Rob R. in collaboration with Marcel Kaufmann, Jonathan Becktor, Sangwoo Moon, Kalind Carpenter, Kai Pak, Amanda Towler, Rohan Thakker and myself. Please find the #OpenSource code, paper, and video demonstration linked below. Operating autonomous robots in the field is often challenging, especially at scale and without the proper support of Subject Matter Experts (SMEs). Traditionally, robotic operations require a team of specialists to monitor diagnostics and troubleshoot specific modules. This dependency can become a bottleneck when an SME is unavailable, making it difficult for operators to not only understand the system's functional state but to leverage its full capability set. The challenge grows when scaling to 1-to-N operator-to-robot interactions, particularly with a heterogeneous robot fleet (e.g., walking, roving, flying robots). To address this, we present the ROSA framework, which can leverage state-of-the-art Vision Language Models (VLMs), both on-device and online, to present the autonomy framework's capabilities to operators in an intuitive and accessible way. By enabling a natural language interface, ROSA helps bridge the gap for operators who are not roboticists, such as geologists or first responders, to effectively interact with robots in real-world missions. In our video, we demonstrate ROSA using the NeBula Autonomy framework developed at NASA Jet Propulsion Laboratory to operate in JPL's #MarsYard. Our paper also showcases ROSA's integration with JPL's EELS (Exobiology Extant Life Surveyor) robot and the NVIDIA Carter robot in the IsaacSim environment (stay tuned for ROSA IssacSim extension updates!). These examples highlight ROSA's ability to facilitate interactions across diverse robotic platforms and autonomy frameworks. Paper: https://lnkd.in/g4PRjF4V Github: https://lnkd.in/gwWXmmjR Video: https://lnkd.in/gxKcum27 #Robotics #Autonomy #AI #ROS #FieldRobotics #RobotOperations #NaturalLanguageProcessing #LLM #VLM

Ever wondered what it takes for a robot to master chaotic environments? This video of Bolero navigating an obstacle course at aggressive speeds has me absolutely hooked! I love this video because it shows the immense challenge and progress in autonomous driving, especially in complex scenarios like those found on Indian roads. Swaayatt Robots, demonstrates an advanced planner reacting to obstacles like traffic cones with incredible agility. Having personally experienced the unpredictable nature of Indian traffic—where lane discipline is a myth and obstacles appear out of nowhere—I can tell you this is no small feat. It highlights the critical need for robots to not just follow rules, but to adapt and react instantaneously in highly dynamic settings. This demonstrates how robotic systems are evolving to handle real-world unpredictability, pushing the boundaries of what's possible in autonomous navigation for all of us. Video credits: Swaayatt Robots

New video is out, teaching a Unitree G1 humanoid to walk using reinforcement learning (PPO). First time I've ever got sim2real to actually work with robotics, sharing what I've learned and testing out how good the policy actually is by walking around outside on some semi challenging terrain. Video: https://lnkd.in/eqwtCZB2

Caltech’s CAST and TII just showed something brilliant: a Unitree G1 humanoid carrying M4 — a backpack robot that flies, lands, drives, then transitions modes to overcome obstacles. In a campus demo the humanoid walked, deployed M4 from its back, M4 drove around a pond, then flew back over it to reach an “emergency” — all coordinated as one system. Why this matters: • Multimodal locomotion (walk + fly + drive) expands where robots can operate. • Tight hardware + control co-design (Saluki controller, lidar/cameras, model-based learning) makes autonomy safer and more adaptable. • Collaboration across CAST, TII, Northeastern, and Caltech labs shows the power of cross-discipline teams in solving real-world robotics problems. Biggest takeaway: combining locomotion modalities gives robots the complementary strengths of each — speed, endurance, and terrain dexterity — while shrinking their weaknesses. Exciting step toward more useful, resilient, real-world robot teams. Read Caltech’s writeup to see the demo and technical vision: https://lnkd.in/eCJd_5Mt

Robots Are Entering the Tennis Court. Humanoid robotics just served another milestone. UBTECH Robotics recently showcased its Walker S2 humanoid robot rallying with a human in a live tennis exchange. At first glance, it looks like a fun demo. But technically, it’s a serious robotics benchmark. To return a tennis ball, the robot must handle several complex tasks simultaneously: • Track a fast-moving object in real time • Predict the ball’s trajectory • Maintain balance while moving dynamically • Coordinate vision, motion planning, and actuation within milliseconds That’s sensorimotor intelligence — the same capability robots need to operate in factories, warehouses, and real-world environments. Sports environments are actually brutal testing grounds for robotics: • unpredictable motion • high-speed decision making • continuous physical adjustment If a robot can rally a tennis ball with a human, it’s a signal that real-world robotic autonomy is getting closer. The broader trend is clear. Humanoid robotics is shifting from lab demos → practical deployment. And companies like UBTECH are pushing that transition faster than many expected. The next wave of AI may not just live in software. It may be walking, balancing — and returning your tennis serve. #AI #Robotics #HumanoidRobots #ArtificialIntelligence #DeepTech #FutureOfWork