Real-Time Robotics Solutions with Google Gemini

Gemini VLAs can now run on-device! Google DeepMind just released 'Gemini Robotics On-Device', an optimized version of its VLA that can run locally on a robot. This builds on their March release of Gemini VLA, which demonstrated how a single VLA can control multiple embodiments (from ALOHA arms to a humanoid) using vision, language, and motion. Now, the lightweight version of the same model can run locally and in real-time. The demonstrations were performed with a Raspberry Pi 4 Coral Edge TPU (they still make those?! 😅). This is much needed. In the real world, network connections are flaky, and we need model inference on a device. It's natural to sacrifice performance when we optimize for local inference, but Gemini On-Device seems to be solid! On their 9-task generalization benchmark: - Gemini On-Device beats every prior on-device model - Performed nearly as well as cloud-inferred Gemini, despite drastically lower compute DeepMind also released the Safari SDK, which enables easy prompting, control, and evaluation of VLA models on both real and simulated hardware. Two things I found particularly useful were the unified API across embodiments and the out-of-the-box benchmarking/logging, which are usually annoying to build from scratch. Finally, they released a MuJoCo simulator designed for the Gemini family of VLAs. It's awesome to see a research lab consider the practical (and dare I say, production) use cases that robotics engineers have.

Imagine smarter robots for your business. New research from Google puts advanced Gemini AI directly into robots, which can now understand complex instructions, perform intricate physical tasks with dexterity (like assembly) and adapt to new objects or situations in real time. The paper introduces "Gemini Robotics," a family of AI models based on Google's Gemini 2.0, designed specifically for robotics. They present Vision-Language-Action (VLA) models capable of direct robot control, performing complex, dexterous manipulation tasks smoothly and reactively. The models demonstrate generalization to unseen objects and environments and can follow open-vocabulary instructions. It also introduces "Gemini Robotics-ER" for enhanced embodied reasoning (spatial/temporal understanding, detection, prediction), bridging the gap between large multimodal models and physical robot interaction. Here's why this matters: At scale, this will unlock more flexible, intelligent automation for the future of manufacturing, logistics, warehousing, and more, potentially boosting efficiency and enabling tasks previously too complex for robots as we've imagined in the past. Very, very promising! (Link in the comments.)

Google DeepMind has launched an on-device version of its Gemini Robotics AI, allowing robots to operate without an internet connection. This smaller, more efficient vision-language-action (VLA) model retains many of the dexterous capabilities of the original, enabling robots to generalize tasks and respond to commands with minimal training (50–100 demonstrations). Though not as powerful as the cloud-enabled flagship, the offline model is surprisingly capable and ideal for low-connectivity or high-security environments. It has been adapted to various robots, including Apptronik’s Apollo humanoid and Franka’s bi-arm robot. Google is also releasing an SDK to let developers evaluate and fine-tune the model — a first for its VLA tech. Initially, access is limited to trusted testers. Read more: https://lnkd.in/dFs5Sw5P