G1 mandarin harvesting remains unachieved: tactile control and open challenges

Update: harvesting has not been achieved. The video shows contact-responsive finger adjustment and a force-triggered stop, not a successful harvest. In a subsequent isolated holding test, the fruit slipped out about 2.5 seconds after its stem support was deliberately disabled. Detaching the fruit and retaining it in the hand remain unachieved.

When we pick up a mandarin, we adjust our grip by feel. Too much force can damage it; too little can let it slip. We are exploring how a robot can make those adjustments.

Team Z is developing a mandarin-harvesting simulation with the G1 humanoid robot. We have tested contact-responsive finger control and a mechanism that stops motion progression when excessive force is detected. Stable holding and harvesting remain in development.

Damage-free harvesting has not been achieved. Hardware harvesting and VLA training for this task are also not complete.

Seeing contact does not tell us the grip force

We use forces from simulated contact as virtual tactile feedback to adjust finger movement. These are provisional sensor settings, not a calibrated reproduction of measurements from a physical hand.

Show / hide concept animation

The hand model: Unitree Dex3-1

We use a 3D model of the Unitree Robotics Dex3-1 three-finger hand on G1's right arm. The manufacturer specifies seven degrees of freedom and 33 tactile sensing points per hand. See the official product page.

This is not a physical hand test. Contact reaction forces from the physics simulation are assigned to 33 provisionally placed virtual sensing points. Their positions and calibration are not a validated reproduction of the physical sensors.

How much force is the controller targeting?

This trial targets 0.6 N per finger, summing the contact forces assigned to the thumb, index finger and middle finger separately. For scale, 0.6 N is approximately the weight of a 61 g object on Earth; it is not a payload rating.

A target is not a measured force maintained throughout the video. Adding the three targets does not establish a successful 1.8 N grip: contact direction, slip and arm loading also matter.

These are provisional development settings, not validated damage limits for real mandarins.

Why choose 0.6 N?

It is a provisional initial target for exploring low-force contact control. It was not derived from measurements of mandarin weight, friction or damage onset, and this trial does not validate it as an appropriate grip force.

An appropriate force must be high enough to prevent slipping, low enough to avoid damage, and sufficient to resist detachment loads. Contact area and placement also matter: the same force concentrated on a smaller area creates more pressure.

How does the hand respond to stronger resistance?

Here, resistance means the contact reaction force opposing a finger pressing against the fruit. It is not a measurement of bounce or fruit stiffness.

The controller adjusts target joint angles using proportional feedback: close slightly when force is too low and open slightly when it is too high. Larger errors produce stronger corrections, with a speed limit. It updates 120 times per second.

For example, a finger reading of 0.9 N calls for opening relative to the 0.6 N target; 0.3 N calls for closing. These are explanatory examples, not measurements taken from the video.

A separate guard triggers above 1.5 N at one sensing point or 2 N summed over one finger. It commands finger relaxation and stops progression of twisting or carrying. Missing feedback and excessive unmapped contact force also trigger protection.

Stopping the arm does not necessarily remove contact force. The current stop holds joint positions, so arm pressure against the fruit can remain. Reducing that pressure is an outstanding control task.

Why does the video not show a harvested mandarin?

The recorded trial stopped on a sensing-point overload shortly after twisting began. The stem did not detach, and its constraint continued to support the fruit. Grasping, twisting off and retaining the fruit in the hand have not been demonstrated as a complete sequence.

Next we will refine finger contact placement and holding posture, reduce arm-induced pressure, and test a stem model that permits rotation and resistance, followed by unsupported holding after detachment. Fruit deformation and damage remain unvalidated; a visually plausible motion is not proof of damage-free harvesting.

Stopping matters as much as moving

In the tested sequence, excessive force was detected as the motion moved toward twisting, and progression stopped. This demonstrates a stopping mechanism, not a successful harvest. The settings do not establish a damage threshold for real mandarins.

Developing the environment and the contact motion

The orchard layout combines tree rows, paths, ground and a distant forest. It is a visual prototype, not a surveyed reconstruction of a real farm. Repeated tree shapes and artificial-looking ground still need work.

We plan to photograph a real fruit-bearing branch from multiple angles to improve close-up fruit, leaves and branches. Visual quality and physical validity need separate checks.

Task decisions and local force adjustment

A vision-language-action model, or VLA, selects robot actions from images and instructions. Our intended approach combines task decisions with contact-responsive adjustment at the hand.

The finger control tested here was not learned by a VLA. We first need reliable holding, detachment and transport before collecting successful task demonstrations.

Next: supporting the fruit reliably

Our goal extends beyond making contact: support the fruit, detach it and carry it without dropping it. We are recording failures as we develop both the environment and the control.