Summary (Overview)

• Xiaomi-Robotics-1 is a foundational vision-language-action (VLA) model for mobile manipulation, trained on over 100,000 hours of real-world UMI trajectories and then aligned to robot embodiments via post-training.
• A two-stage training recipe is proposed: pre-training on massive, diverse UMI data with auto-labeled state-transition language prompts, and post-training on cross-embodiment data with imperative instructions, enabling out-of-the-box performance in unseen environments.
• The model adopts a Mixture-of-Transformers (MoT) architecture coupling a pre-trained VLM (Qwen3-VL) with a diffusion transformer (DiT), using flow matching for action generation and an auxiliary Choice Policy loss to accelerate convergence.
• Scaling behavior is demonstrated: validation action error decreases monotonically with both data scale and model size during pre-training; this scaling directly transfers to post-training, with stronger pre-trained models yielding higher real-robot success rates in novel environments.
• Xiaomi-Robotics-1 achieves state-of-the-art results on multiple simulation benchmarks: 57.4% success rate on RoboCasa365 (vs. 46.6% previous best), 20.07 average score on RoboDojo (vs. 13.07), and strong performance on RoboCasa and VLABench. In downstream fine-tuning, it achieves 75% success rate with less than 10 hours of data per task, outperforming π0.5 (40%).

Introduction and Theoretical Foundation

The research is motivated by the scaling paradigm that has driven breakthroughs in large language models (LLMs) and vision-language models (VLMs). Applying similar scaling to robotics is appealing but faces a unique bottleneck: real-robot teleoperation is slow, costly, and hardware-bound, resulting in data that is often redundant and limited in diversity. The paper aims to overcome this by leveraging a scalable data source—Universal Manipulation Interface (UMI) handheld grippers—which allow massive collection of real-world manipulation trajectories across diverse environments without requiring physical robot embodiments.

The theoretical foundation builds on vision-language-action (VLA) models and world-action models (WAMs). VLA models harness pre-trained VLM backbones to leverage rich semantic knowledge for action prediction, while WAMs model future observations or environment dynamics. The authors propose a two-stage training paradigm inspired by LLM pre-training and fine-tuning: pre-training on a large-scale, diverse dataset to learn generalizable action representations, and post-training to align these representations with robot embodiments and imperative instructions.

Key insights:

• Traditional trajectory labeling (manual segmentation and language annotation) is prohibitive at scale; an auto-labeling pipeline using a pre-trained VLM (Qwen3.5-27B) to caption state transitions in fixed-length trajectory segments is developed.
• The model learns to generate actions that transform the scene from its current state to the target state described by the language annotation.
• Post-training bridges two gaps: adapting from UMI gripper actions to robot embodiments, and transitioning from state-transition descriptions to imperative instructions.

Methodology

Model Architecture

Xiaomi-Robotics-1 adopts a Mixture-of-Transformers (MoT) architecture (Fig. 2) consisting of:

• Pre-trained VLM (Qwen3-VL): encodes the observation $o_t$ and language instruction $l$.
• Diffusion Transformer (DiT): generates action chunks via flow matching, conditioned on the robot proprioceptive state $s_t$ and the VLM’s KV cache of the observation and language tokens.

Three model variants are specified (Table 1):

Action generation uses flow matching:

$$\mathcal{L}_{\text{Flow}}(\theta) = \left\| v_\theta(o_t, l, s_t, \tilde{a}^\tau_{t:t+H}, \tau) - u(\tilde{a}^\tau_{t:t+H}, a_{t:t+H}, \tau) \right\|_2^2$$

where $\tau$ is the flow-matching timestep sampled from a Beta distribution: $u \sim \text{Beta}(1.5, 1), \tau = (1-u) \cdot 0.999 \in [0, 0.999]$. Inference uses 5-step Euler integration with $\Delta\tau = 0.2$.

To accelerate convergence, an auxiliary Choice Policy loss is added on the VLM:

$$\mathcal{L}_{\text{Regression}}(\theta) = \left\| \hat{a}^*_{t:t+H} - a_{t:t+H} \right\|_1 + \sum_{k=1}^K \left\| \hat{s}_k - s_k \right\|_2^2$$

where $\hat{a}^*$ is the candidate with smallest L1 distance to ground truth, and $s_k = \| \hat{a}^k_{t:t+H} - a_{t:t+H} \|_1$. Action-related tokens are excluded from the DiT’s attention to avoid shortcut copying.

Training & Data

Pre-training (Fig. 3):

• Dataset: Over 100,000 hours of real-world UMI trajectories collected in households, commercial premises, industrial sites, offices, and outdoor spaces.
• Auto-labeling pipeline: Trajectories are divided into equal-length segments; Qwen3.5-27B captions state transitions of grippers and interacting objects. A producer–consumer pipeline with CPU workers and hundreds of concurrent captioning requests labels the entire corpus in ~2 weeks.
• Training objective: Joint minimization of $\mathcal{L}_{\text{Flow}} + \mathcal{L}_{\text{Regression}} + \lambda \mathcal{L}_{\text{NTP}}$ with $\lambda = 0.1$. Co-training on high-quality VL data (ratio 1:9) preserves VLM capabilities.

Post-training (Fig. 4):

• Dataset: ~10,000 hours of cross-embodiment data, including 7,200+ hours of in-house robot data (mobile manipulators and dual-arm robots), 1,000+ hours of instruction-labeled UMI data, and open-source datasets (Bridge V2, RT-1, DROID).
• Action alignment: Arm actions are relative delta end-effector poses: $a_{t+i} = (^{EE}_{\text{Base}} T_t)^{-1} \, ^{EE}_{\text{Base}} \hat{T}_{t+i}$. Orientation unified across embodiments. Missing dimensions masked during loss computation.
• Training: Same objective as pre-training; data sampling ratio 0.5:0.5:0.5:8.5 (VL: open-source: instruction-labeled UMI: in-house robot).

Empirical Validation / Results

Pre-training: Data and Model Scaling

Data scaling (Fig. 5, left): Using Xiaomi-Robotics-1-5B on 12.5%, 25%, 50%, and 100% of ~20k hours of UMI data. Validation MSE decreases with more data; 12.5% and 25% show overfitting, while 50% and 100% yield monotonic improvement.

Model scaling (Fig. 5, right): 2B, 5B, and 10B variants on the same 20k hours. Validation MSE improves with model size, but gains are less pronounced than data scaling, suggesting data volume is the primary bottleneck.

Post-training: Out-of-the-Box Evaluation in Novel Environments

Evaluated on 4 tasks (shoe storage, bag packing, table organization, sofa tidying) in unseen environments (Fig. 7).

Scaling pre-training data (Fig. 8, left):

• Overall success rate increases monotonically with pre-training data scale: 26% (no pre-training) → 53% (12.5%) → 75% (100%).
• Largest gains on contact-rich tasks (shoe tidying: 0% → 75%).

Scaling model size (Fig. 8, right):

• Success rate: 61% (2B) → 75% (5B) → 79% (10B).
• Shoe tidying improves from 58% (2B) to 92% (10B).

Downstream Fine-tuning: Efficient Adaptation to New Tasks

Four novel tasks (phone packing, laundry loading, printer refilling, box packing) with limited data (Fig. 9, 10).

Xiaomi-Robotics-1 substantially outperforms baselines, especially on dexterous and mobile manipulation tasks.

Simulation Benchmarks

RoboCasa (Table 2):

RoboCasa365 (Table 3):

VLABench (Table 4, selected metrics):

RoboDojo (Table 5):

Theoretical and Practical Implications

• Scaling laws for robotics: The paper provides strong empirical evidence that VLA models exhibit predictable scaling behavior with data and model size during pre-training, and that this scaling transfers to real-world performance after post-training. This suggests that continued investment in data collection and model scaling will yield further improvements.
• Data efficiency: The auto-labeling pipeline using pre-trained VLMs to annotate state transitions at scale is a key practical contribution, enabling the use of massive UMI datasets that would otherwise be infeasible to label manually.
• Cross-embodiment alignment: The post-training phase demonstrates that generalizable action representations learned from UMI data can be effectively transferred to different robot embodiments, bridging the gap between non-robot and robot data.
• Foundation model for robotics: Xiaomi-Robotics-1 serves as a strong base policy for efficient fine-tuning on novel tasks, achieving high success rates with minimal data (e.g., 75% average with <10 hours per task). This is crucial for practical deployment where task-specific data collection is expensive.
• Benchmark leadership: The model sets new state-of-the-art results on multiple challenging simulation benchmarks, particularly on RoboCasa365 (Composite-Unseen split: 32.1% vs. 14.9% previous best) and RoboDojo, highlighting its generalization to unseen task compositions and diverse manipulation capabilities.

Conclusion

Xiaomi-Robotics-1 is a foundational VLA model that enables out-of-the-box mobile manipulation in unseen environments and efficient adaptation to novel tasks with minimal data. Its two-stage training recipe—pre-training on over 100k hours of UMI trajectories with auto-labeled state-transition prompts, followed by cross-embodiment post-training with imperative instructions—proves effective. The model exhibits clear scaling behavior in both data and model size, translating directly to real-world performance. It achieves state-of-the-art results on four challenging simulation benchmarks (RoboCasa, RoboCasa365, VLABench, RoboDojo) and demonstrates strong data efficiency in downstream fine-tuning. Future work may explore further scaling of data and model size, integration of memory mechanisms, and deployment in more complex long-horizon tasks. Code and model checkpoints will be released.

Key Quote: “The performance of Xiaomi-Robotics-1 consistently improves with increasing data scale and model size during pre-training. More importantly, the scaling property directly translates to out-of-the-box performance in unseen environments after post-training.”