Energy Efficiency in Robotic Systems

Today, Science Robotics has published our work on the first drone performing fully #neuromorphic vision and control for autonomous flight! 🥳 Deep neural networks have led to amazing progress in Artificial Intelligence and promise to be a game-changer as well for autonomous robots 🤖. A major challenge is that the computing hardware for running deep neural networks can still be quite heavy and power consuming. This is particularly problematic for small robots like lightweight drones, for which most deep nets are currently out of reach. A new type of neuromorphic hardware draws inspiration from the efficiency of animal eyes 👁 and brains 🧠. Neuromorphic cameras do not record images at a fixed frame rate, but instead have the pixels track the brightness over time, sending a signal only when the brightness changes. These signals can now be sent to a neuromorphic processor, in which the neurons communicate with each other via binary spikes, simplifying calculations. The resulting asynchronous, sparse sensing and processing promises to be both quick and energy efficient! 🔋 In our article, we investigated how a spiking neural network (#SNN) can be trained and deployed on a neuromorphic processor for perceiving and controlling drone flight 🚁. Specifically, we split the network in two. First, we trained an SNN to transform the signals from a downward looking neuromorphic camera to estimates of the drone’s own motion. This network was trained on data coming from our drone itself, with self-supervised learning. Second, we used an artificial evolution 🦠🐒🚶♂️ to train another SNN for controlling a simulated drone. This network transformed the simulated drone’s motion into motor commands such as the drone’s orientation. We then merged the two SNNs 👩🏻🤝👩🏻 and deployed the resulting network on Intel Labs’ neuromorphic research chip "Loihi". The merged network immediately worked on the drone, successfully bridging the reality gap. Moreover, the results highlight the promises of neuromorphic sensing and processing: The network ran 10-64x faster 🏎💨 than a comparable network on a traditional embedded GPU and used 3x less energy. I want to first congratulate all co-authors at TU Delft | Aerospace Engineering: Federico Paredes Vallés, Jesse Hagenaars, Julien Dupeyroux, Stein Stroobants, and Yingfu Xu 🎉 Moreover, I would like to thank the Intel Labs' Neuromorphic Computing Lab and the Intel Neuromorphic Research Community (#INRC) for their support with Loihi (among others Mike Davies and Yulia Sandamirskaya). Finally, I would like to thank NWO (Dutch Research Council), the Air Force Office of Scientific Research (AFOSR) and Office of Naval Research Global (ONR Global) for funding this project. All relevant links can be found below. Delft University of Technology, Science Magazine #neuromorphic #spiking #SNN #spikingneuralnetworks #drones #AI #robotics #robot #opticalflow #control #realitygap

Researchers at EPFL have unveiled an innovative robot bird that blends terrestrial and aerial locomotion through advanced physics and engineering principles. Inspired by the biomechanics of avian species, it features lightweight, robust materials and multifunctional legs that store and release energy efficiently, enabling powerful jumps for rapid takeoffs. These legs are modeled to mimic the spring-like motion of tendons and muscles, leveraging principles of elastic potential energy to convert stored energy into kinetic energy during liftoff. This allows for faster, more energy-efficient flight initiation compared to traditional propeller-driven systems, which rely on continuous motor operation to achieve lift. The robot also integrates advanced aerodynamics for stable flight, utilizing biomimetic wing designs that optimize lift-to-drag ratios. Its ability to walk and hop over obstacles stems from precision actuators and sensors that calculate optimal force and trajectory, ensuring smooth transitions between ground and air mobility. These features make it highly adaptive to complex terrains, from rocky landscapes to dense forests, where conventional drones and robots would struggle. Future prospects for this #technology are promising. Its multi-modal capabilities could be applied in search-and-rescue missions, where navigating through collapsed structures or dense vegetation requires both ground movement and aerial maneuverability. In planetary exploration, it could traverse rugged terrains on Mars or the Moon, combining the efficiency of walking with the flexibility of flight. Further advancements may include incorporating solar-powered systems for extended autonomy, swarm robotics for collaborative tasks, and machine learning algorithms to enhance decision-making and obstacle avoidance. This groundbreaking #design not only bridges the gap between terrestrial and aerial robotics but also sets the stage for a new era of versatile, energy-efficient robotic systems capable of tackling a wide range of environmental and industrial challenges. 🎥@EPFL Video rights are reserved for the respective owner. #innovation #whatinspiresme

Drawing insights from biological signal processing, neuromorphic computing promises a substantially lower power solution to improve energy efficiency of visual odometry (VO) in robotics. Published in Nature Machine Intelligence, this novel approach develops a VO algorithm built from neuromorphic building blocks called resonator networks. Demonstrated on Intel’s Loihi neuromorphic chip, the network generates and stores a working memory of the visual environment, while at the same time estimating the changing location and orientation of the camera. The system outperforms deep learning approaches on standard VO benchmarks in both precision and efficiency – relying on less than 100,000 neurons without any training. This work is a key step in using neuromorphic computing hardware for fast and power-efficient VO and the related task of simultaneous localization and mapping (SLAM), enabling robots to navigate reliably.   A companion paper explores how the neuromorphic resonator network can be applied to visual scene understanding. By formulating the generative model based on vector symbolic architectures (VSA), a scene can be described as a sum of vector products, which can then be efficiently factorized by a resonator network to infer objects and their poses. The work demonstrates a new path for solving problems of perception and many other complex inference problems using energy efficient neuromorphic algorithms and Intel hardware. Congratulations to researchers from the Institute of Neuroinformatics, University of Zurich and ETH Zurich, Accenture Labs, Redwood Center for Theoretical Neuroscience at UC Berkeley, and Intel Labs.   Learn more about neuromorphic VO: https://lnkd.in/gJCVVMCz   Learn how the VSA framework was developed for neuromorphic visual scene understanding based on a generative model (companion paper): https://lnkd.in/gjAENfpp   #iamintel #Neuromorphic #Robotics

A new vision system called LENS enables a hexapod robot to recognize its surroundings using less energy and memory than a single photo on a smartphone, researchers report in Science Robotics. It uses just 10% of the power required by conventional location systems. Developed with the Speck sensor and chip from SynSense, LENS mimics the human eye, detecting only changes in brightness rather than capturing constant video like traditional cameras. This drastically reduces data and power consumption, which is crucial for drones, microrobots, and robots in space or underwater. The system combines a neuromorphic sensor, a low-power processor, and a tiny AI model that learns to recognize places by analyzing edges and key environmental features, not full images. Researchers say this could revolutionize how small, mobile robots navigate with minimal battery drain. Read more: https://lnkd.in/e3uujFg9