Coke in Refrigerator
One humanoid holds the refrigerator open while the other places the coke inside.
MATE
MATE: Multi-Agent Virtual Teleoperation Platform for Humanoid Collaboration Data Collection
1 ShanghaiTech University2 Nanyang Technological University* Corresponding authors
ABSTRACT
Learning from interaction-rich experience.
Humanoid robots require diverse embodied experiences to acquire complex loco-manipulation and collaborative skills. MATE is a multi-agent virtual teleoperation platform that enables multiple geographically distributed operators to simultaneously control whole-body humanoids in a shared physics-based environment.
We construct a multi-humanoid collaboration dataset comprising 24.1 hours of coordinated behavior across 2,500 joint episodes and five long-horizon tasks. We further introduce Execution-Aligned Interaction Sampling (EAIS), which prioritizes task-progressing and interaction-critical behaviors. Experiments with imitation learning and vision-language-action policies demonstrate effective policy learning and zero-shot transfer from virtual demonstrations to a physical humanoid without real-world fine-tuning.
FULL DEMO
From operators to real hardware.
DATA COLLECTION
Coupled behavior, captured at scale.
Each operator controls one humanoid through an independent interface. All agents, objects, and contacts evolve together in simulation, producing synchronized joint trajectories rather than isolated single-agent recordings.
DATASET TASKS
Five tasks, one shared world.
Bottle Handoff
Two humanoids coordinate a direct transfer of the bottle.
Bottle Relay
Intermediate placement, acquisition, and final delivery are shared across agents.
Bottle into Cart
One humanoid positions the cart while the other transports and places the bottle.
Push Hospital Bed
One humanoid holds the door while the other pushes the bed through.
COLLECTION EFFICIENCY
Faster resets, synchronized episodes.
On the same Bottle Relay procedure, MATE reduces amortized collection time per successful episode from about 89 seconds to about 41 seconds under the reported protocol.
POLICY ROLLOUTS
Learning long-horizon coordination.
GR00T N1.7 · Coke in Refrigerator
Policy-controlled coordination over an extended horizon.
Psi-zero · Bottle Handoff
The learned receiver responds to the partner’s handoff.
Psi-zero · Push Hospital Bed
Long-horizon navigation and physical coordination.
GR00T N1.7 · Bottle Relay
Coordination through intermediate placement and final delivery.
Psi-zero · Bottle into Cart
Long-horizon transport and cooperative placement.
EXECUTION-ALIGNED INTERACTION SAMPLING
See what the learner should remember.
ROLLOUT COMPARISON
SIM-TO-REAL
Virtual demonstrations, physical execution.
Demonstrations collected in the shared virtual environment transfer to a physical humanoid without additional real-world training data.
MATE
Scalable humanoid collaboration data.
More resources will be released with the paper.
Back to top ↑