Summary (Overview)

• RynnBrain 1.1 is a family of embodied foundation models spanning three scales (2B, 9B, and 122B-A10B), trained with a unified spatio-temporal and physically grounded framework that supports embodied perception, spatial reasoning, localization, and planning.
• Two new capabilities are introduced: (1) contact point prediction—a compact contact-centered representation $(p, \theta)$ for grasp grounding—and (2) native 3D grounding for the 2B and 9B models, enabling language-conditioned 3D bounding box prediction in camera coordinates.
• RynnBrain-VLA is developed with a unified cross-embodiment action space (81 dimensions) and embodiment-specific masking, deployed on three heterogeneous robots: Unitree G1 humanoid, Astribot-S1 bimanual robot, and Tianji-Wuji dexterous-hand system.
• The 122B-A10B model outperforms all evaluated proprietary and open-source models on VSI-Bench, MMSI, and RefSpatial-Bench, demonstrating a clear scaling trend in embodied capabilities.
• Real-robot experiments show RynnBrain-initialized policies outperform Qwen-based and representative generalist VLAs (GR00T N1.7, π0.5), with joint multi-task and multi-embodiment training improving average process scores from 91.28% to 94.14% and success rates from 86.67% to 91.67%.

---

Introduction and Theoretical Foundation

Embodied intelligence requires models to go beyond visual recognition and language understanding. A robot operating in the physical world must perceive objects, reason about spatial relations, identify actionable regions, and translate understanding into executable behaviors. The central challenge for embodied foundation models is whether visual understanding is grounded in the physical world and can be transferred to real-robot action.

RynnBrain 1.0 [20] introduced a unified embodied multimodal model for egocentric understanding, spatial grounding, physically grounded reasoning, and planning. However, two key aspects were not fully explored:

1. How can representations and outputs be brought closer to robot manipulation?
2. How effectively can RynnBrain serve as initialization for downstream VLA post-training?

RynnBrain 1.1 addresses these questions through:

• Contact point prediction: locating the center point and rotation angle of the gripper for grasping, which is more precise and meaningful than the previous grasp rectangle detection.
• 3D grounding: introducing explicit 3D modeling during training, since robots operate in three-dimensional environments.
• Unified action space: organizing embodiment-specific actions into semantically aligned body-part groups with embodiment-specific masks, enabling joint training across heterogeneous robots without forcing alignment between incompatible action spaces.

Built upon Qwen3.5 [53], all models share a unified capability definition and training framework, enabling systematic study of how embodied capabilities evolve with model scale.

---

Methodology

Model Architecture

RynnBrain 1.1 adopts a decoder-only vision–language architecture following Qwen3.5 design principles, consisting of:

• A vision encoder
• A vision–language projector
• An LLM backbone initialized from Qwen3.5 variants

The architecture employs DeepStack [45] and Interleaved MRoPE [30] for better multimodal integration and long-context spatio-temporal modeling. Models use either a dense or mixture-of-experts (MoE) decoder.

Unified Spatio-temporal Representation

Visual observations are represented as ordered frame sequences $V = \{I_t\}_{t=1}^T$, where static images correspond to $T=1$ and videos have $T>1$. Temporal position information is attached to tokens so the model can distinguish frame order.

Physically Grounded Output Space

Physical grounding is incorporated directly into the model's generation vocabulary. In addition to natural-language tokens, outputs may contain discrete representations of:

• Bounding boxes $B$
• Points $P$
• Trajectory waypoints $T$

Continuous image coordinates are mapped to integers in $[0, 1000]$ and predicted autoregressively like text tokens.

For 3D grounding, models predict a 3D bounding box parameterized in camera coordinates:

• Center position $(c_x, c_y, c_z)$
• Box dimensions $(w, l, h)$
• Orientation $(\text{pitch}, \text{yaw}, \text{roll})$

All in physical units (meters and radians), discretized into integer tokens within a fixed range.

Optimization Objective

All variants are optimized end-to-end with a causal autoregressive objective:

$$\mathcal{L} = -\sum_{i=1}^{L} \log P(y_i | y_{<i}, V; \Theta)$$

where $L$ is the target-sequence length and $\Theta$ denotes learnable model parameters.

Pretraining Data Mixture

The pretraining mixture includes:

• General MLLM data: LLaVA-OV-SI, LLaVA-Video, ShareGPT-4o-video, VideoGPT-plus, FineVideo, CinePile, ActivityNet, YouCook2, LLaVA-SFT, VL3, RynnBrain-Thinking
• Cognition data: object understanding, spatial understanding, counting, OCR, egocentric task understanding
• Spatio-temporal localization data: object localization, area localization, affordance localization, trajectory prediction, grasp/contact prediction
• 3D-grounded data: WildDet3D (Essential: 102,979 images, 374K annotations; Synthetic: 896,004 images, 888K annotations) and FoundationPose (1,849K images)
• Contact point data: 2.6M training samples from Grasp-Anything, Jacquard V2, GraspFactory, GraspNet-1B, GraspClutter6D
• Planning data: AgibotWorld, Open X-Embodiment, RynnBrain-Planning, Galaxea-G0

Contact Point Representation

RynnBrain 1.1 adopts a compact contact-centered representation $a = (p, \theta)$, where $p = (x, y)$ denotes the center of the grasp configuration and $\theta$ is the in-plane angle of the line connecting gripper fingers. Coordinates are normalized to $[0, 1000]$, serialized as <grasp pose> (x, y), θ </grasp pose>. This representation factors out annotation-dependent rectangle extents while preserving an action-relevant spatial anchor and orientation cue.

RynnBrain-VLA Architecture

The VLM backbone serves as a single-stream diffusion transformer processing dense interaction among vision, language, and action in each layer. A flow matching framework predicts action chunks [5]. The input format follows:

<|im_start|>user
INSTRUCTION: Pour the wine from the bottle into the wine glass...
OBSERVATION: <camera_1><camera_2><camera_3>
STATE: <state>
What action should the robot take?<|im_end|>
<|im_start|>assistant
<action>

Unified Cross-Embodiment Action Space

Actions from heterogeneous robots are represented within a shared 81-dimensional action space partitioned into semantically aligned groups:

Each embodiment activates only physically available dimensions via an embodiment-specific mask:

• Unitree G1: activates Hand (14D, 7D per hand) + separately predicted 64D SONIC latent token
• Tianji-Wuji: activates Arm-Joint (14D) + Hand (40D) = 54 active dimensions
• Astribot S1: activates Arm-Joint (14D), Gripper (2D), Head (3D), Torso (4D)

Real-Time Chunking (RTC)

The model predicts a 32-step action chunk but triggers new inference every 5 steps. Action-guidance follows Black et al. [7] with strength $\beta = 10.0$, assigning guidance weights based on whether steps will be consumed before the new chunk arrives.

---

Empirical Validation / Results

Embodied Cognition Capability

2B Scale (Table 3): RynnBrain 1.1 improves over RynnBrain 1.0 on reasoning-intensive tasks:

• MMSI: 34.1 → 40.5
• MindCube: 50.1 → 61.7
• RynnBrain-Spatial: 57.2 → 63.8

9B Scale (Table 4): Systematic gains over RynnBrain-8B on 8 of 10 shared benchmarks:

• MindCube: 56.6 → 86.9
• MMSI: 39.6 → 47.0
• RynnBrain-Spatial: 59.9 → 67.9

122B-A10B Scale (Table 5): Extends advantage to the large-model regime:

• MMSI: 40.5 → 47.0 → 52.0 (2B → 9B → 122B-A10B)
• MindCube: 61.7 → 86.9 → 89.6

Embodied Localization Capability

Clear scaling trend on RefSpatial-Bench: 58.5 (2B) → 67.2 (9B) → 79.1 (122B-A10B), showing relational spatial grounding benefits strongly from model capacity.

Scaling Analysis: Three Distinct Regimes

Key finding: Reasoning-intensive capabilities are not emergent from general VLM scaling—they require explicit embodied supervision. Qwen3.5 exhibits negative scaling (−39.2%) on reasoning-intensive tasks, while RynnBrain 1.1 improves steadily (+38.6%).

3D Grounding Results

RynnBrain 1.1-2B already outperforms Seed1.5-VL (33.5) and Gemini 2.0 Pro (32.5) on SUN RGB-D, while the 9B model surpasses the specialized WildDet3D detector trained with additional in-domain data.

Real-Robot Evaluation

Comparison with Baselines (Table 6):

The process score is defined as:

$$\text{ProcessScore} = \frac{1}{20K} \sum_{i=1}^{20} \sum_{j=1}^{K} s_{i,j}$$

where $s_{i,j} \in \{0, 1\}$ indicates whether the $j$-th sub-task is successfully completed in the $i$-th trial.

Key findings:

• RynnBrain-VLA outperforms Qwen-Based-VLA on all tasks (average success rate: 86.67% vs. 60.00%)
• RynnBrain-VLA outperforms GR00T N1.7 and π0.5 generalist baselines
• On Unitree G1, RynnBrain-VLA achieves 90% success on Pull the Chair vs. 75% for GR00T N1.7 with SONIC
• Joint multi-task/multi-embodiment training (Generalist) further improves performance over per-task training

---

Theoretical and Practical Implications

Theoretical Implications

1. Non-uniform scaling laws in embodied AI: Unlike standard vision-language benchmarks where scaling consistently improves performance, embodied capabilities exhibit three distinct scaling regimes. Reasoning-intensive cognition requires explicit embodied supervision—larger language priors in general VLMs may override weak visual-spatial signals, producing more confident but less accurate responses.

2. Data scaling vs. model scaling: For embodied localization, training on explicit coordinate supervision is more effective than model scaling alone. RynnBrain's embodied pretraining equips even compact models with structured spatial output capabilities that pure language modeling must recover implicitly.

3. Embodied pretraining as a scaling enabler: On reasoning-intensive tasks, embodied pretraining determines whether scaling helps or hurts—transforming a capability that degrades under pure VLM scaling into one that benefits consistently.

Practical Implications

1. Cross-embodiment knowledge sharing: The unified action space with embodiment-specific masking allows different embodiments to teach each other shared interaction structures (object affordances, task progress, reach-grasp-manipulate patterns) while preserving embodiment-specific control.

2. Better VLA initialization: A stronger embodied foundation model serves as a better starting point for VLA post-training, with capabilities acquired during embodied pretraining transferring effectively to downstream manipulation tasks.

3. Deployment framework: The cross-embodiment, cross-driver framework decouples VLA policy from embodiment-specific execution, allowing new embodiments to be added by only implementing the embodiment layer.

---

Conclusion

RynnBrain 1.1 presents an embodied foundation model family spanning 2B, 9B, and 122B-A10B scales, demonstrating:

• Clear improvements in multi-view reasoning, spatial grounding, and robot-oriented understanding under a unified training framework
• Native language-conditioned 3D grounding through explicit 3D-grounded supervision
• A compact contact-centered representation for interaction grounding
• RynnBrain-VLA that substantially outperforms Qwen-based and generalist VLA baselines on long-horizon manipulation tasks

Future directions include:

• Extending RynnBrain toward a unified multimodal model combining perception, understanding, generation, spatial grounding, and action
• Developing RynnBrain into the cognitive core of a general embodied agent with long-term memory, world modeling, task planning, active perception, and closed-loop interaction
• Enabling continuous reasoning about physical world changes, plan generation/revision from feedback, and skill acquisition through interaction across diverse tasks and embodiments