Around the rocks to the observation site: a lunar rover demo in Isaac Sim

Move the rocks, and the route changes. We built a simulated lunar environment in which a rover observes its surroundings, avoids rocks and drives to a designated observation site. It stops, lowers and retracts an instrument, then continues to the next waypoint.

We authored the visual model and terrain with reference to publicly available LUPEX information, and connected perception, path planning and motion control in Isaac Sim. This is an independent TeamZ demonstration, not commissioned or supervised by JAXA. It does not validate performance on a real rover or on the Moon.

Different rock layouts, the same code

We changed only the lateral positions of the rocks between the two cases. Perception, planning and control used the same code and configuration. Instead of supplying the rocks' ground-truth positions to perception, we identified points above the ground from onboard depth images.

The planner accounts for detected obstacles and the rover's size, then uses A* path search to find a route to the target. Changing the rock layout changed the direction of the detour.

The final evaluation included one run per case. Both completed driving, stopping, instrument lowering and retraction, and travel to the next waypoint, with zero recorded rock contacts. Distance to the observation target at arrival was approximately 17 cm in each run. These two runs do not establish a general success rate or real-world positioning accuracy.

Instrument preparation, without excavation

After stopping, the drill-shaped instrument descends, holds roughly 2.5 cm above the ground and retracts. The body and instrument move as simulated rigid bodies and joints. Soil excavation, sample collection and water detection are not part of the demonstration.

Conditions and limitations

Depth images are ideal, noise-free distance measurements generated by the simulator. Localization also uses ideal simulator state. We did not implement stereo depth estimation or SLAM. Obstacle detection and path planning use geometric methods, without a learned AI policy or VLA.

The terrain is an authored rigid surface. Soil deformation, sinkage, excavation, vacuum, thermal effects and radiation are not evaluated. Safety on arbitrary unknown terrain, including cliffs and holes, remains untested. The rover is a visual reference model inspired by LUPEX, not JAXA-supplied CAD or a validated digital twin of the real vehicle.

What would you like to test in simulation?

This prototype connects environment construction, sensor output, obstacle perception, path planning, vehicle motion and result recording. We can help develop similar simulation environments around your robot and task.

We can start by identifying the conditions you need to compare, such as obstacle layouts or transitions from driving to instrument operation. For comparisons against hardware, we first agree on reference data, measurement methods and who will collect the data, then define the evaluation scope. Discuss a development or evaluation project.

Reference

JAXA: LUPEX — searching for water at the lunar south pole, used as a reference for appearance and equipment layout. TeamZ authored the video, terrain and model; the published material does not reproduce JAXA photographs, footage or logos.