Summary (Overview)

• GroundingPI is a 4B-parameter grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary, designed to serve as a perceptual backbone for physical intelligence (robotics and autonomous driving).
• State-of-the-art grounding performance: Across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI outperforms 44 baselines with an average score of 73.68%, surpassing the larger GPT-6 Astra (71.54%).
• Downstream gains: As a visual backbone, GroundingPI improves robotic manipulation (RoboTwin 2.0, RoboCasa-GR1) and autonomous driving (nuScenes), with up to 24.8% relative improvement over the strongest backbone in out-of-distribution settings.
• Training pipeline: Combines multimodal/spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using public datasets and dedicated data engines.
• Key insight: Dense grounding (points and boxes) provides a better perceptual foundation for physical intelligence than general-purpose vision-language or video-generation backbones.

---

Introduction and Theoretical Foundation

The paper addresses a critical gap in the emerging field of physical intelligence—the ability of AI systems to perceive and act in the physical world. While vision-language-action (VLA) models and world-action models (WAMs) have made significant progress, they inherit perception capabilities from general-purpose vision-language and video-generation backbones that fail in precise grounding tasks.

The authors argue that precise grounding is fundamental:

"Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control."

Key limitations of existing backbones:

• Poor performance in cluttered scenes
• Failure on tiny objects
• Insufficient speed for closed-loop control

The theoretical premise is that perception quality is the bottleneck for downstream action generation. The paper proposes that a dedicated grounding foundation model—trained explicitly for spatial precision—can serve as a stronger perceptual foundation than general-purpose models, even if those general models are larger.

---

Methodology

Model Architecture and Grounding Formulation

GroundingPI uses a shared vocabulary of quantized coordinates to represent both points and bounding boxes:

• Points: $(x, y)$ coordinates
• Boxes: $(x_1, y_1, x_2, y_2)$ coordinates

Both are quantized to discrete tokens in a shared vocabulary, enabling unified training across tasks.

Training Pipeline

The training consists of three stages:

1. Base VLM Training: Multimodal and spatial pretraining on public datasets to establish foundational visual-linguistic understanding.

2. Supervised Fine-Tuning (SFT): Fine-tuning on grounding tasks with supervision from public datasets and dedicated data engines.

3. Reinforcement Post-Training: Using GRPO (Group Relative Policy Optimization) to further refine grounding precision and robustness.

Data

Training data comes from:

• Public datasets for multimodal and spatial understanding
• Dedicated data engines designed to generate high-quality grounding supervision at scale

---

Empirical Validation / Results

Grounding Benchmark Results

GroundingPI was evaluated across 34 benchmarks spanning 11 perceptual capabilities, against 44 baselines:

GroundingPI achieves state-of-the-art results, surpassing even much larger models.

Physical Intelligence Performance

Autonomous Driving (nuScenes)

Using GroundingPI as the visual backbone for open-loop driving:

$$L2_{\text{error}} = 0.296 \, \text{m}$$

This is the average open-loop L2 error, demonstrating strong spatial perception for driving.

Robot Manipulation (RoboTwin 2.0)

GroundingPI outperforms every mainstream backbone evaluated in all four out-of-distribution settings, with improvements of up to 24.8% relative to the strongest competing backbone.

Data Efficiency (RoboCasa-GR1)

A notable finding: GroundingPI trained with 50% of demonstrations outperforms baselines trained with 75%, demonstrating superior data efficiency.

Ablations and Analysis

• Scaling behavior: Downstream autonomous driving and robotic manipulation improve as pretraining is scaled.
• Data composition: Dense grounding provides substantial benefits for both driving and manipulation.
• OCR as catalyst: Optical character recognition (OCR) shows potential as a catalyst for perceptual learning.
• Grounding training effects: Dedicated grounding pretraining transfers positively to action tasks.

---

Theoretical and Practical Implications

Theoretical Implications

The results support the paper's central thesis: grounding can serve as a perceptual foundation for physical intelligence. The findings suggest that:

1. Dedicated perceptual pretraining is a promising direction for foundation models of physical intelligence, potentially more valuable than general-purpose pretraining at scale.
2. Dense grounding (point- and box-level supervision) transfers more effectively to action tasks than coarse or semantic-only perception.
3. Perception quality, not model size alone, is the key bottleneck for downstream physical intelligence tasks.

Practical Implications

• Architecture choice: Practitioners building VLA/WAM systems should consider grounding-specific backbones rather than defaulting to general vision-language models.
• Data efficiency: GroundingPI's superior data efficiency (50% vs. 75% demonstrations) has significant cost implications for robotics data collection.
• Real-time capability: The model is designed to be fast enough for closed-loop control, addressing a practical constraint for deployment.

---

Conclusion

GroundingPI demonstrates that precise grounding is a powerful foundation for physical intelligence. Key takeaways:

1. A 4B grounding model can outperform much larger general-purpose models on perception benchmarks.
2. Grounding-based perception transfers effectively to both robotic manipulation and autonomous driving.
3. Scaling dedicated perceptual pretraining improves downstream performance.
4. Dense grounding supervision and diverse perceptual capabilities (including OCR) enhance action transfer.

Future Directions

The paper suggests that:

• Further scaling of grounding-focused pretraining could yield even stronger physical intelligence backbones.
• Exploring additional perceptual capabilities beyond the 11 studied could further enhance transfer.
• The grounding foundation model paradigm could be extended to more complex physical interaction tasks.

The authors conclude that dedicated perceptual pretraining is a promising direction for building foundation models of physical intelligence, challenging the prevailing assumption that general-purpose pretraining alone is sufficient.