The video re-renders recorded simulation poses with updated orchard visuals; it is not live camera footage. On-screen captions are in Japanese, with English explanations below.
When we pick up a mandarin, we adjust our grip by feel. Too much force can damage it; too little can let it slip. We are exploring how a robot can make those adjustments.
Team Z is developing a mandarin-harvesting simulation with the G1 humanoid robot. We have tested contact-responsive finger control and a mechanism that stops motion progression when excessive force is detected. Stable holding and harvesting remain in development.
Damage-free harvesting has not been achieved. Hardware harvesting and VLA training for this task are also not complete.
Seeing contact does not tell us the grip force
We use forces from simulated contact as virtual tactile feedback to adjust finger movement. These are provisional sensor settings, not a calibrated reproduction of measurements from a physical hand.
Show / hide concept animation
Stopping matters as much as moving
In the tested sequence, excessive force was detected as the motion moved toward twisting, and progression stopped. This demonstrates a stopping mechanism, not a successful harvest. The settings do not establish a damage threshold for real mandarins.
Developing the environment and the contact motion
The orchard layout combines tree rows, paths, ground and a distant forest. It is a visual prototype, not a surveyed reconstruction of a real farm. Repeated tree shapes and artificial-looking ground still need work.
We plan to photograph a real fruit-bearing branch from multiple angles to improve close-up fruit, leaves and branches. Visual quality and physical validity need separate checks.
Task decisions and local force adjustment
A vision-language-action model, or VLA, selects robot actions from images and instructions. Our intended approach combines task decisions with contact-responsive adjustment at the hand.
The finger control tested here was not learned by a VLA. We first need reliable holding, detachment and transport before collecting successful task demonstrations.
Next: supporting the fruit reliably
Our goal extends beyond making contact: support the fruit, detach it and carry it without dropping it. We are recording failures as we develop both the environment and the control.
LET’S BUILD TOGETHER
What are you working on?
Tell us about the software you need to implement or test. We can discuss the work, the development setup and how we could contribute.
Discuss your development needs