Methodology
The framework pretrains foundational dexterity on a generic task objective and post-trains for contact-rich downstream tasks. It consists of three stages:
- Pretraining a dexterous policy in simulation on a generic object-manipulation task.
- Transferring the pretrained behavior through distillation and adapting it through massively parallel on-policy reinforcement-learning post-training, while preventing catastrophic forgetting through stable policy updates.
- Distilling a task-specialist teacher into a stereo RGB vision-based student that can be deployed zero-shot in the real world.
Limitations
Although the real-world policies exhibit behaviors closely matching those observed in simulation, perception remains the primary bottleneck for distillation. Failures frequently coincide with incorrect estimates of the asymmetric peg’s orientation under occlusion, while grasp instability can arise from the small contact patch between the rounded Allegro fingertips and the rounded side of the peg under fast arm motion.
Fingertip tactile sensing on Flexiv-Sharpa mitigates contact ambiguity, but robust object-centric perception under occlusion remains an open problem. A wrist-mounted camera and tactile sensing on additional platforms may help address these limitations.
Domain Randomization (ADR)
The following parameters are annealed from easy to hard over 50 ADR increments, triggered when the running success rate exceeds 0.4:
- Gravity: Gravity is annealed across a specified range in meters per second squared.
- Goal sharpness: Goal precision requirements are tightened during training.
B.1 Pull-Back from TaskMaps to the Root
The second term is the curvature force required to keep the leaf acceleration equal to the pulled-back leaf force under the chain rule.
B.2 C-Space Attractors
The metric is isotropic, with a smooth switch between a minimum and maximum mass around the error norm.
B.8 ADR-Annealed Fabric Parameters
Several fabric parameters are part of the ADR curriculum rather than fixed at YAML defaults:
- C-space damping gain: Annealed from a high value, which heavily smooths control and makes it easier, toward a lower value, which produces more reactive behavior as the success rate improves.
- Speed-control energy target: Annealed to broaden the range of motion speeds to which the policy is exposed.
- Velocity attenuation: A scalar applied between the fabric velocity output and the joint controller, also annealed through ADR.
The fabric is therefore not treated as a fixed black-box controller but as part of the curriculum. As policy competence grows, the regularization provided by the fabric is gradually relaxed.
G.1 Behavior-Cloning Loss
The behavior-cloning term matches both moments of the student’s action distribution to the teacher’s through a Mahalanobis-weighted L2 distance.
For the KUKA-iiwa7 and Allegro system, the teacher’s fixed standard deviation serves as a per-dimension precision weighting. Joints about which the teacher is more confident, indicated by a smaller standard deviation, contribute more strongly to the loss.
G.2 Auxiliary Eight-Keypoint Pose Loss
The auxiliary head predicts the eight bounding-box corners of the peg in the world frame. The loss is the L2 norm of the per-corner residuals.
G.3 Total Distillation Objective
The behavior-cloning and auxiliary pose-loss terms are combined using their respective weighting coefficients.
Frequently Asked Questions
What are the three stages of the ADEPT framework? The stages are simulation pretraining, reinforcement-learning post-training with distillation, and teacher-student distillation for a stereo RGB vision-based student.
What is the primary bottleneck during real-world distillation? Perception is the primary bottleneck, especially estimating the asymmetric peg’s orientation under occlusion.
How does ADR change the training process? ADR anneals parameters from easy to hard over 50 increments when the running success rate exceeds 0.4.
What does the auxiliary pose head predict? It predicts the eight bounding-box corners of the peg in the world frame.