The video uses English captions over the same recorded simulation as the Japanese edition. The robot motion and camera frames are unchanged.
Reaching a gauge is not the same as reading it. Distance and viewing angle can leave a robot with an image that is unsuitable for inspection.
TeamZ built a Go2 quadruped demonstration that checks an image and changes its viewpoint when reading conditions are not met. We created a fictional factory with tanks, pipes and pumps in Isaac Sim, a simulator for robot motion, contact and camera imagery.
The next action matters when an image is unreadable
At the first viewpoint, none of five images supplied enough reliable correspondences to the registered gauge face. The controller withheld the reading and moved to the next preconfigured viewpoint, closer to the gauge.
The five new images produced values from 0.914 to 0.917 MPa. We recorded their median, 0.916 MPa. MPa is a unit of pressure. The simulated gauge was set to 0.920 MPa, a difference of 0.004 MPa in this trial.
Images support both motion and inspection
We added an ideal virtual stereo rig—two cameras with a known separation—to Go2. This does not reproduce Go2's standard camera specification. Differences between the two images and features tracked over time support position estimation.
cuVSLAM performs Visual SLAM: simultaneously estimating camera position and surrounding visual features. We use its pose estimate to adjust velocity toward predefined waypoints in a known clear aisle. The line in the video is the estimated trajectory, not a displayed navigation map.
For walking, we use Isaac Lab's published Go2 policy—a learned model that turns velocity commands and robot state into joint actions. We did not train or fine-tune it. The robot moves through physics rather than by teleporting its base along an animated path.
Simulator ground-truth pose is used for audit records and the filming camera, not for aisle position control. The locomotion policy still uses joint and orientation information. This is not a claim that the entire robot is controlled using cameras alone.
Rectify the gauge face, then read the needle
A gauge viewed at an angle distorts the needle angle. We match the observed scale to a needle-free reference image, rectify sufficiently reliable matches to a frontal view, and then estimate the needle direction.
SIFT finds image features that can be matched despite changes in scale and orientation. RANSAC estimates a perspective transformation while rejecting inconsistent matches. In addition to geometric checks, at least three of five frames must produce readings, with a maximum-to-minimum spread no greater than 0.015 MPa. We then record the median.
This is image processing calibrated for a specific gauge, without a vision-language-action model or a large vision-language model. The pressure fixture value is used for evaluation only; it is not supplied to the reader.
Compare with a fixed viewpoint
We compared withholding a reading without moving against repositioning after rejection. We also changed the needle position to check that the reader was not returning a fixed answer. All trials use the same simulated factory, gauge face and lighting.
| Trial | First view | Next view | Recorded / fixture |
|---|---|---|---|
| No repositioning | 0/5 accepted | No movement | Unresolved / 0.920 |
| Repositioning comparison | 0/5 accepted | 3/5 accepted | 0.916 / 0.920 MPa |
| Lower needle setting | 0/5 accepted | 5/5 accepted | 0.421 / 0.420 MPa |
| Higher needle setting | 0/5 accepted | 5/5 accepted | 1.319 / 1.320 MPa |
| Final published recording | 0/5 accepted | 5/5 accepted | 0.916 / 0.920 MPa |
Each row is one run. For the final recording, we corrected the method for logging ground-truth camera pose for auditing; image-based control and reading criteria were unchanged. Failed frames remain in the records. These small tests are not a success-rate estimate or a physical-hardware accuracy benchmark.
What works, and what remains
We connected visual localization in a known aisle, learned walking, rejection of an insufficient image, movement to another viewpoint, and multi-frame reading. The video and the decisions come from the same simulation run.
| Not yet demonstrated | Why, and the next verification needed |
|---|---|
| Autonomous patrol in an unfamiliar factory | Aisle waypoints and viewpoint candidates are configured in advance. Persistent mapping, path planning and obstacle handling remain to be verified. |
| Automatic optimal-viewpoint search | A rejected reading triggers the next configured candidate. Generating and ranking viewpoints by visibility and movement cost is not implemented. |
| Glare, dirt, darkness or other gauges | Only one gauge face and fixed lighting were tested. The initial failure was insufficient matching, not detected glare avoidance. |
| Inspection on a physical Go2 | The rig uses ideal simulated cameras. Real optics, exposure, vibration, communication latency and locomotion error need hardware tests. |
What can be tested before going on site?
A simulated environment lets us compare gauge heights, camera mounts, viewpoint candidates and unresolved-reading behavior before a hardware trial. Next we will vary lighting, occlusion and gauge placement to find the conditions where reading breaks down.
TeamZ develops simulation environments, connects robot control with perception, and evaluates individual trials. Contact us to discuss what to verify first for factory patrol or equipment inspection.
Inspiration and implementation sources
The idea of revisiting the observation viewpoint was informed by Tuomisto et al., Automating on-site object inspection with a quadruped robot and BIM. That study uses Spot, BIM and 3D LiDAR, among other components. Our Go2, Visual SLAM and gauge-reading prototype is a separate implementation, not a reproduction of that paper.
The runtime used Isaac Sim 5.1, a pinned Isaac Lab revision, RSL-RL 3.0.1 and the cuVSLAM Python API. The fictional factory is our own work. We publish our recorded footage and results, not third-party robot assets or pretrained weights.
LET’S BUILD TOGETHER
Turn your robotics ideas into questions we can test.
Tell us about your task, equipment and current challenges. Through simulation and tests on real robots, we explore feasibility and technical challenges together.
Discuss joint development