DexSafe

Pushing Dexterity and Safety for In-Hand Object Pose Reaching That Works Remarkably Well

Ian Chuang1 Hiya Shah1 Vamsi Chilamkur1 Iman Soltani2 Claire Tomlin1

1UC Berkeley2UC Davis

Try the policy in your browser Watch all 4,800 goals Paper (arXiv) Code: coming soon
Our robot hand reaches diverse in-hand object poses at any wrist orientation. One policy per object; the outline is the goal and turns green when reached. Sped up only once every object holds its goal.
4,800unseen goals on the real robot
13.3 hof real-robot evaluation
95.2%goals reached (state policy)
−75%drops, state policy with the safety filter (16 → 4)

Try the policy in your browser

The deployed policy and safety filter, running live. Move the goal, turn the wrist, or drag the object to push it.

Run the policy live

MuJoCo in your browser · about 16 MB

Abstract

Dexterous manipulation with multi-fingered hands is the next frontier of robotics, yet learned hand skills remain limited to simple grasps or basic repeated motions, such as rotating an object. As robots attempt more demanding in-hand maneuvers, safety, such as not dropping the object, becomes critical, yet it is rarely measured. We study in-hand object pose reaching, a task that demands diverse and difficult grasps: moving an object to any position and orientation relative to the hand, at any wrist orientation, through long sequences of consecutive goals. We present a recipe for training such policies for a LEAP hand in simulation without human data, built on a range of insights, from diverse resets and goals via generated grasp datasets to an adaptive friction curriculum. With careful system identification, the policies transfer zero-shot to the real hand, using either motion capture or a single wrist-mounted depth camera, and prove remarkably capable and robust. On top of the task policy, we add a learned residual safety filter with adjustable conservativeness, letting us tune the balance between performance and safety and greatly reduce object drops. In 13.3 hours of real-robot evaluation over 4,800 unseen goals across 4 objects and diverse wrist orientations, the state-based policy reaches 95% of goals within 2 cm and 0.4 rad, and the filter cuts its drops by 75% at a cost of 3.7 points of success. The policies can also spin a lightbulb held in an arbitrary pose, reach out-of-distribution goals, withstand fast arm motion and perturbations, and, needing only a wrist camera, serve as portable skills such as screwing in a lightbulb. We release the full recipe as a platform for studying performance, reliability, and safety in high-dimensional dexterous systems.

Any wrist orientation

Evaluated at these six wrist orientations.

Rubber duck, state policy, three consecutive goals at each of the six evaluation orientations.

Ten consecutive goals per orientation

4,800 goals, all on video

96 runs of 50 consecutive unseen goals, 10 s each: 4 objects × 6 wrist orientations × state or depth policy × with or without the filter.

All 96 runs at once, 50× timelapse.
Results table
PolicyObjectFilterPrecise /300Coarse /300Drops /300Time to 1st drop (s)Reached before 1st drop /50Avg. action change
StateLightbulbnone277297242742.3–
ResSafe268292050048.78.5%
Rubber ducknone255291729829.3–
ResSafe194262244139.215.4%
Knifenone234288425724.7–
ResSafe216281147044.59.3%
Wineglassnone141266335230.8–
ResSafe128263144939.28.9%
DepthLightbulbnone191284337135.2–
ResSafe183278050046.37.4%
Rubber ducknone1522641226823.5–
ResSafe95233239230.313.1%
Knifenone60163221639.0–
ResSafe53146440622.06.6%
Wineglassnone56151511113.7–
ResSafe50143401175.56.6%

Each row pools 6 runs (one per wrist orientation). Precise: 1 cm / 0.2 rad; coarse: 2 cm / 0.4 rad. Filter at w = 0.5.

Explore every run

Pick a run: watch all of its goals at once, one tile per goal, or the whole run as a timelapse.

Set
View
Speed

Trained in simulation, transferred zero-shot

No human data: grasps and goals come from generated datasets. A privileged teacher is distilled into a motion-capture (state) student and a wrist-camera (depth) student, and a residual safety filter is learned on top.

Training pipeline: stages A1 to A4 for the task policy, B1 and B2 for the safety filter, and deployment on the real robot
1,024 parallel environments, each at its own wrist orientation.
System identification: the real and simulated hand play the same actions.
Simulated wrist depth with random distractors: the depth student reaching consecutive goals.
On the real robot: the state policy reads the object pose from motion capture; the depth policy reads only the wrist camera's depth.

A safety filter that prevents drops

ResSafe adds a small learned correction to the policy's action, trained only to keep the object in hand. On the real robot it cuts the state policy's drops by 75%.

Filter saves
Filter drops

Beyond the benchmark

Spinning a lightbulb

Wrist
Highlight
About its own axis, one whole 20 s goal (depth policy, no filter). The ring turns with the bulb.

Out-of-distribution goals

Object
Highlight
A hard goal from a 1.5× larger box (palm down, state policy), then the next goal of the run.

Goals moved by hand

An operator steers the goal live. Inset: wrist camera with the commanded goal.

Physical disturbances

Depth policy, no filter: fingers pulled away, then the object pushed into the palm.

Unseen arm motion

The arm swings the wrist for 30 s while the object turns at its goal (depth policy, wrist-camera inset). Pick any trial.

Lightbulb

Rubber duck

Multi-step tasks with only a wrist camera

The depth policy as a skill, with scripted arm motions. No motion capture, no external cameras.

Rubber duck: grasp, reorient, place

Wineglass: pick up, flip, flip back

Lightbulb: regrasp, screw in

Every trial. The filter runs at w = 1.5 in these tasks.

BibTeX

@misc{chuang2026dexsafe,
      title={Pushing Dexterity and Safety for In-Hand Object Pose Reaching That Works Remarkably Well},
      author={Ian Chuang and Hiya Shah and Vamsi Chilamkur and Iman Soltani and Claire Tomlin},
      year={2026},
      eprint={2610.37652},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2610.37652},
}