Robotics Simulation Programs

Build your first robot in simulation! 👾 📌 If you’re self-learning robotics, this is genuinely one of the better repos to save for later. NVIDIA Robotics released a "Getting Started with Isaac Sim" tutorial series covering everything from building your first robot to hardware-in-the-loop deployment. What's inside? → Building Your First Robot Explore the Isaac Sim interface, construct a simple robot model (chassis, wheels, joints), configure physics properties, implement control mechanisms using OmniGraph and ROS 2, integrate sensors (RGB cameras, 2D lidar), and stream sensor data to ROS 2 for real-time visualization in RViz. → Ingesting Robot Assets Import URDF files, prepare simulation environments, add sensors to existing robot models, and access pre-built robots to accelerate development. → Synthetic Data Generation Learn perception models for dynamic robotic tasks, understand synthetic data generation, apply domain randomization with Replicator, generate synthetic datasets, and fine-tune AI perception models with validation. → Software-in-the-Loop (SIL) Build intelligent robots, implement SIL workflows, use OmniGraph for robot control, master Isaac Sim Python scripting, deploy image segmentation with ROS 2 and Isaac ROS, and test with and without simulation. → Hardware-in-the-Loop (HIL) Understand HIL fundamentals, learn NVIDIA Jetson platform, set up the Jetson environment, and deploy Isaac ROS on Jetson hardware. The progression makes sense: start with basics (build a robot), add perception (sensors and data), generate training data (synthetic generation), develop software (SIL), then deploy to hardware (HIL). Each module builds on the previous one. For robotics teams, this is the path to faster iteration. Simulate first, validate in software-in-the-loop, generate synthetic training data at scale, then deploy to hardware with confidence. 🎓 If this helps at least one engineer to become more fluent in the world of robotics, means a lot to me! 🫶🏼 Here's the course (it's free): https://lnkd.in/dRYdkmdi ~~ ♻️ Join the weekly robotics newsletter, and never miss any news → ziegler.substack.com

Big shift in robotics: NVIDIA just open-sourced Isaac Sim and Isaac Lab. Isaac Sim has already been a cornerstone for high-fidelity robotics simulation—RTX-accelerated physics, realistic lidar/camera simulation, domain randomization, ROS/URDF support, and synthetic data pipelines. Now, it’s all on GitHub with full source access. But the real multiplier? The release of Isaac Lab—a modular, open reinforcement learning and robot control framework built directly on top of Isaac Sim. It comes with ready-to-use robots (Franka, UR5, ANYmal), training loops, and environments for manipulation, locomotion, and more. What’s different now: *You’re no longer limited to APIs—developers can modify physics, sensors, and control logic at the source level. *Isaac Lab provides a training-ready foundation for sim-to-real robotics, speeding up learning pipelines dramatically. *Debugging, benchmarking, and custom integrations are now transparent, flexible, and community-driven. *Collaboration across research and industry just got easier—with reproducible environments, tasks, and results. We’ve used Isaac Sim extensively, and this open-source release is going to accelerate innovation across the robotics community. GitHub: https://lnkd.in/gcyP9F4H

A humanoid robot costs $90K to break once. AI lets you break thousands... and learn from every fall. My background is mechanical engineering, robotics, and integration & test. But this field is moving so fast with AI that reading articles wasn't cutting it anymore. I felt out of the loop, so... I recently upgraded my personal setup to support AI training workloads and ran my first experiment: Teaching a bipedal (two-legged) humanoid robot to navigate a custom parkour course using reinforcement learning in NVIDIA Isaac Lab 5.1. But before I share what I learned, let me explain what's actually happening under the hood. A GPU-accelerated AI agent runs thousands of virtual robots in parallel. Each one learns from its own falls and successes simultaneously. The AI develops a "control policy," which is the brain that tells a robot how to move through the physical world. Why does this matter? Because what once required million-dollar labs and months of physical testing can now run on a single AI-capable GPU in hours. Robotics R&D is becoming software-first. Here's what that looked like for this experiment: 76 minutes of CUDA-accelerated training time. 393 million training steps. 4,096 robots learning in parallel on my RTX 5080. So what did I learn so far? Three things stood out to me: 》The setup before you can hit "Run" is a challenge. It took me seven hours to troubleshoot versioning, packages, and dependencies before I could run anything. I forced myself to do it manually because I wanted to understand what's under the hood. YouTube tutorials hit their limit quickly, but thankfully the NVIDIA developer forums saved me. 》The cost case is undeniable. A Unitree H1 costs around $90K. I *virtually* crashed thousands of them. My damage bill? $0. Simulation lets you fail-forward at scale. This gets you to a solid starting point for physical testing, but... 》The Sim-to-Real gap is real. This policy works well in simulation, but I couldn't get a feel for stress points, sensor behavior, or true stability. Failure is not predictable and happens at the edges. The next step would be to transfer this policy to a physical robot, gather real-world data, and continuously aligning the simulation to close that gap. The key thing here is: Testing real hardware is expensive. Simulation in software is cheap. How can you leverage both, intelligently? The benefit isn't limited to cost savings. This workflow also compresses developmental cycles and allows you to field systems faster. Do you think virtual simulation is a game-changer that is here to stay, or a fad? How would you build confidence in a robotic control policy that is trained in a virtual world? #robotics #ai #nvidia #omniverse #isaaclab ~~~~~~~~ Citations: NVIDIA IsaacLab -> https://lnkd.in/ekVMDnDc RSL-RL -> https://lnkd.in/eJye3XTW Unitree H1-> unitree.com/h1/ Note: this is an educational personal project. Opinions are my own, no affiliation or endorsement.

Triple Inverted Pendulum Control with LQR & UKF in MATLAB ➡ Nonlinear dynamic modeling of the triple inverted pendulum on a cart ➡ State-space representation via numerical linearization ➡ LQR optimal controller for stabilizing all three pendulums ➡ Unscented Kalman Filter (UKF) for nonlinear state estimation ➡ Robust handling of process and measurement noise ➡ Real-time simulation with animation and visualization ✨ Why this matters: The triple inverted pendulum is an advanced benchmark problem in control engineering due to its highly nonlinear and unstable behavior. Unlike a single pendulum, multiple links introduce strong dynamic coupling, making stabilization significantly more challenging. To maintain balance, the controller must continuously compute optimal control inputs while accurately estimating system states under noisy conditions. This project demonstrates how modern control (LQR) and estimation (UKF) techniques work together to stabilize complex nonlinear systems. Such approaches are widely used in robotics, aerospace systems, autonomous vehicles, and intelligent control applications. 📊 Key Highlights: ✔ Nonlinear dynamic modeling using Lagrangian formulation ✔ Numerical linearization for state-space control design ✔ LQR controller for multi-link stabilization ✔ UKF implementation for accurate state estimation ✔ Real-time MATLAB simulation and animation ✔ Performance comparison between true and estimated states 💡 Future Potential: This framework can be extended toward: ➡ Model Predictive Control (MPC) implementation ➡ Reinforcement learning-based control strategies ➡ Sensor-based real-time state estimation ➡ Hardware implementation on embedded systems ➡ Advanced robotic balancing and stabilization systems 🔗 For students, engineers & robotics enthusiasts: This MATLAB simulation provides a complete framework for understanding nonlinear dynamics, optimal control, and probabilistic state estimation in complex engineering systems. 🔁 Repost to support robotics research & engineering education! #Robotics #MATLAB #ControlSystems #LQR #UKF #KalmanFilter #Automation #Mechatronics #EngineeringProjects #Simulation #NonlinearSystems #StateEstimation #STEM #EngineeringEducation #DynamicSystems

This is the moment simulation becomes more important than prototyping. In our last posts, Pascalis and I showed two things: First, how you can generate a full production and warehouse environment in NVIDIA Omniverse using Claude Code and the USDA data format. Second, how NVIDIA’s new Kimodo model can generate robot motions from simple text prompts. Now we are taking the next step: Transferring robot motion into Omniverse and merging both use cases. Omniverse is not just for static visualizations. It allows dynamic simulation of movements, interactions and behavior with CAD components inside a virtual environment. And this is where it gets interesting for future product development. The vision is clear: If we can model production environments, warehouses, and real operating environments of products, we can simulate mechatronic products in realistic conditions before they physically exist. Environment → Sensor & actuator interaction → Model-in-the-loop simulation. Very similar to how autonomous vehicles are developed today, but applied to all kinds of mechatronic products. The effects are huge: • Less physical prototyping • Earlier insights without building hardware • Faster iteration cycles • Better product decisions earlier in development • Simulation becomes the main development environment Omniverse already shows how granular these simulations can be created today. Not through months of manual modeling, but increasingly through prompts that generate environments, movements and soon maybe even control logic. We are moving from designing products to designing behavior in simulated worlds first. And that will fundamentally change how we develop products. Curious to hear your thoughts! When will simulation become the primary development environment in your industry? Vlad Larichev | Rüdiger Stern | Rick Bouter | Ruben Hetfleisch | Dr.-Ing. Tobias Guggenberger

Stop building custom eval scripts. Start scaling your robot policies. Generalist robot policies are the future—but evaluating them is a notorious headache. If you’re tired of building high-overhead custom infrastructure every time you want to test a new task or robot embodiment, you need to check out NVIDIA Isaac Lab-Arena. Co-developed with Lightwheel, this new open-source framework is a game-changer for the #Robotics community. It’s designed to bridge the gap between training and real-world deployment by making simulation-based evaluation scalable, repeatable, and actually modular. Why developers should care: 🔹 0 to 1 (Simplified Curation): Use streamlined APIs to create and manage complex tasks without rewriting the system from scratch. 🔹 1 to Many (Automated Diversification): Effortlessly mix and match robots, objects, and environments. Swap a soda can for an industrial pipe in seconds. 🔹 Massive Parallelization: Built on NVIDIA Isaac Lab, it leverages GPU-acceleration to run large-scale parallel benchmarks. 🔹 Standardized Benchmarking: Direct connection to industry benchmarks like Libero and Robocasa, and full integration with the Hugging Face LeRobot ecosystem. Whether you're working on humanoids, AMRs, or manipulators, Isaac Lab-Arena lets you focus on the AI, not the infrastructure. Let's stop guessing if our policies are robust and start proving it in the Arena. 🏟️ 👉 Read the full breakdown and get the GitHub link here: https://bit.ly/4jsmZAZ #NVIDIA #IsaacLab #Robotics #PhysicalAI #HuggingFace

University of Michigan Researchers Introduce OceanSim: A High-Performance GPU-Accelerated Underwater Simulator for Advanced Marine Robotics Researchers from the University of Michigan have proposed OceanSim, a high-performance underwater simulator accelerated by NVIDIA parallel computing technology. Built upon NVIDIA Isaac Sim, OceanSim leverages high-fidelity, physics-based rendering, and GPU-accelerated real-time ray tracing to create realistic underwater environments. It bridges underwater simulation with the rapidly expanding NVIDIA Omniverse ecosystem, enabling the application of multiple existing sim-ready assets and robot learning approaches within underwater robotics research. Moreover, OceanSim allows the user to operate the robot, visualize sensor data, and record data simultaneously during GPU-accelerated simulated data generation. OceanSim utilizes NVIDIA’s powerful ecosystem, providing real-time GPU-accelerated ray tracing while allowing users to customize underwater environments and robotic sensor configurations. OceanSim implements specialized underwater sensor models to complement Isaac Sim’s built-in capabilities. These include an image formation model capturing water column effects across various water types, a GPU-based sonar model with realistic noise simulation for faster rendering, and a Doppler Velocity Log (DVL) model that simulates range-dependent adaptive frequency and dropout behaviors. For imaging sonar, OceanSim utilizes Omniverse Replicator for rapid synthetic data generation, establishing a virtual rendering viewport that retrieves scene geometry information through GPU-accelerated ray tracing..... Read full article: https://lnkd.in/gjTAkB2b Paper: https://lnkd.in/gEhq-SNQ