Making AI accessible for all | Solving AI problems for Industries | Co-founder at Vizuara, First Principle Labs and Videsh | IIT Madras and Purdue
"Rethinking Teleoperation for Low-Cost Robot Arms" At Vizuara, we have been exploring alternatives to traditional teleoperation for collecting robot manipulation data. The standard approach: A leader-follower arm setup — the operator physically moves one arm and the other mirrors it. It works, but comes with friction: cost of the second arm, calibration drift, mechanical wear, and physically tethering the operator. So we tried something different: webcam-based hand tracking as a teleoperation interface for our SO-101 arm. Using MediaPipe's hand and pose landmarkers, we map hand gestures and arm pose to all 6 DOF — pinch for gripper, wrist orientation for roll/pitch, elbow angle, and shoulder position from pose estimation. The core challenge: decoupling degrees of freedom . When you tilt your wrist, your fingers move too. When you lift your arm, your elbow angle changes. A single webcam compresses 3D motion into 2D, making it hard to isolate independent joint commands. We addressed this by fusing two separate models — hand landmarks for wrist/gripper, pose landmarks for shoulder/elbow — essentially treating different body parts as independent control channels, with smoothing and dead zones to reduce cross-talk. It's still early. The mapping is imperfect, and we're nowhere near the fidelity of purpose-built devices. But at zero additional hardware cost (just a laptop webcam), it's a compelling direction for rapid prototyping. The teleoperation space is moving fast: (1) GELLO — open-source 3D-printed leader arms that match your robot's kinematics. Elegant, but requires a physical device per morphology. (2) Bunny-VisionPro / OPEN TELEVISION (UCB) — using Apple Vision Pro and VR headsets for immersive teleoperation with high-fidelity hand tracking. Incredible demos, expensive hardware. (3) 1X Technologies — building neural teleoperation at scale for humanoid data collection, treating the teleop interface as a first-class product. The broader trend is clear: the bottleneck in robot learning has shifted from algorithms to data, and teleoperation is the bridge. Whether it's a $10K haptic suit or a webcam running MediaPipe, the question is the same — how do you translate human intent into robot action, faithfully and at scale? We’ll keep experimenting. If you're working on teleoperation alternatives or low-cost data collection for manipulation, we’d love to connect.