Summary (Overview)

• ABot-N1 is a general Visual Language Navigation (VLN) foundation model that decouples cognition from control via a slow–fast dual-system architecture, using a 4B-parameter vision-language reasoner for deliberative planning and a 2B-parameter fast action expert for reactive waypoint generation.
• The slow system produces Chain-of-Thought (CoT) reasoning traces and pixel goals (affordance and target pixels) in the egocentric view, serving as a universal interface for five navigation tasks: point-goal, object-goal, POI-goal, instruction-following, and person-following.
• Post-training with GRPO (Group Relative Policy Optimization) aligns the slow system’s pixel-goal generation with downstream navigation rewards, improving safety and robustness beyond supervised imitation.
• Two new benchmarks are released: ABotN-PointBench (indoor/outdoor point-goal with social compliance) and ABotN-POIBench (urban POI-goal with entrance-level success criteria).
• ABot-N1 achieves state-of-the-art results across all five tasks, with massive gains in urban-scale navigation: POI arrival +35.0% (to 77.3%), and 95.4%/92.9% SR in complex indoor/outdoor scenes.

Introduction and Theoretical Foundation

Embodied navigation requires an agent to operate in open-world environments, handling precise metric targets, free-form language instructions, open-vocabulary objects, points of interest (POIs), and moving people. Historically, these tasks have been addressed by specialized architectures with bespoke goal interfaces, hindering cross-task transfer and real-world deployment.

The paper identifies three critical challenges in existing generalist navigation models:

1. To-Point Navigation: Coordinate inaccuracies from SD maps or localization can shift targets into non-traversable regions (e.g., vehicle lanes), requiring explicit traversability reasoning.
2. To-Target Navigation: End-to-end fine-tuning on object-search trajectories risks eroding pretrained semantic priors and conflates “search” with “approach,” leading to brittle training.
3. Interpretability & Safety: Monolithic policies lack intermediate reasoning traces, obscuring the causal link between perception and action.

These challenges stem from a fundamental architectural flaw: monolithic policies that bypass intermediate reasoning. This creates a spectral mismatch between the slow dynamics of semantic reasoning and the fast dynamics of motor control, and causes optimization interference across heterogeneous tasks.

The authors propose a modular, factorized approach that decouples cognition from control. The slow system (4B VLM) performs low-frequency deliberative reasoning, producing CoT traces and pixel goals. The fast system (2B VLM) operates at high frequency, consuming these outputs to generate continuous waypoints. The pixel goal — a set of 2D anchors in the egocentric view — serves as a transparent, structured bottleneck, replacing black-box latent representations.

Methodology

Model Architecture (Sec. 4.1):

• Slow System (Qwen-3.5-4B): Given reference memory, tri-view observation, task specification, and prior decision, it produces:
  • CoT trace ( C_n ) (natural-language rationales)
  • Pixel goals ( p_n ) (affordance pixel – next safe waypoint; target pixel – final goal when visible)
• Fast System (Qwen-3.5-2B): Encodes current observations, CoT trace, pixel goals, reference observation, and task specification to produce a fused hidden state ( h_t ). Learnable action queries ( q_{\text{act}} ) interact via QFormer cross-attention, and an MLP decodes into continuous waypoints:

$$a_{t:t+H} = \text{MLP}(\text{QFormer}(q_{\text{act}}, h_t))$$

where ( H = 5 ), each action ( a_i = (x_i, y_i, \sin\theta_i, \cos\theta_i, c_i) \in \mathbb{R}^4 \times {0,1} ).

• Asynchronous inference: Slow system runs at low frequency (e.g., 1–2 Hz), fast system at 10 Hz, using cached pixel goals between slow updates.

Training (Sec. 4.2–4.3):

• Pretraining (30M samples): Supervised imitation on five tasks, with CoT and pixel-goal annotations. Slow system trained with cross-entropy on tokens and pixel coordinates; fast system with smooth-L1 loss on waypoints and binary cross-entropy on arrival flag.
• Post-training (GRPO): Applied to the slow system. For each state, ( G ) candidate outputs are sampled and scored with a composite reward:

$$R = w_f R_{\text{format}} + w_t R_{\text{target}} + w_o R_{\text{safety}}$$

• ( R_{\text{format}} ): binary indicator for valid JSON schema
• ( R_{\text{target}} ): exponential decay of L2 distance to ground-truth target pixel: $$R_{\text{target}} = \sum_{\text{frame}} \alpha_t \exp\left(-\frac{\|\hat{p} - p^*\|_2}{\text{scale}}\right)$$
• ( R_{\text{safety}} ): penalty based on distance ( d ) to nearest non-traversable region: $$R_{\text{safety}}(d) = \begin{cases} -\alpha_0, & d \geq d_{\text{safe}} \\ -\alpha_o \exp(\beta(d_{\text{safe}} - d)), & d < d_{\text{safe}} \end{cases}$$
  • GRPO objective (Eq. 8):
  $$\mathcal{L}_{\text{GRPO}} = \mathbb{E}\left[ \sum_{i,t} \min(\rho_t^{(i)} A^{(i)}, \text{clip}(\rho_t^{(i)}, 1\pm\epsilon) A^{(i)}) \right] - \beta \text{KL}(\pi_\theta \| \pi_{\text{ref}})$$
• Balanced sampling: Episodes stratified by proximity to non-traversable regions into Safe, Critical, Danger zones (ratio 5:3:2) to balance gradient fidelity and safety enforcement.

New Benchmarks (Sec. 5):

• ABotN-PointBench: 31 scenes (16 indoor, 15 outdoor), 465 reference trajectories stratified by distance (Low/Medium/High). Outdoor uses relaxed collision budget (SR<3col), indoor strict (SR<1col). Metrics: SR, SPL.
• ABotN-POIBench: 11 commercial regions, 163 POIs, entrance-level arrival (2m threshold). Metrics: SR<2m, SPL.

Empirical Validation / Results

Instruction-Following (VLN-CE R2R/RxR, Table 1):

ABot-N1 achieves best NE (3.32m), SR, and SPL on R2R-CE, and best NE (3.13m) on RxR-CE, using only tri-view RGB without depth or odometry.

Object-Goal (Short-Horizon OVON, Table 2):

ABot-N1 raises SR by +11.7 points over ABot-N0 and compresses DTG from 1.44m to 0.82m.

Point-Goal (ABotN-PointBench, Tables 3 & 4):

• Outdoor (SR<3col): ABot-N1 achieves 92.9% overall SR, +16.0 over ABot-N0 (76.9%). On the High tier: 88.0% vs 65.3%.
• Indoor (SR<1col): ABot-N1 achieves 95.4% SR, +5.8 over ABot-N0 (89.6%). On High tier: 95.8% vs 87.5%.

POI-Goal (ABotN-POIBench, Table 5):

ABot-N1 boosts SR<2m by +35.0 points over POINav.

Person-Following (EVT-Bench, Table 6):

ABot-N1 leads on all SR metrics and most TR metrics, with notable gains on Distracted and Ambiguity tracking.

Real-World Deployment: Deployed on AMap TuTu quadruped with Jetson AGX Orin, achieving 10Hz closed-loop control. Qualitative results demonstrate safe outdoor navigation (construction detour, traffic-light compliance), object recognition under occlusion, POI entrance localization, instruction decomposition, and person-following under distraction.

Theoretical and Practical Implications

• Theoretical: The slow-fast factorization with pixel-goal interface provides a principled solution to the spectral mismatch between semantic reasoning and motor control. The GRPO post-training stage demonstrates that outcome-driven reinforcement learning can be effectively applied to the reasoning component, aligning pixel-goal generation with navigation success.
• Practical: The unified pixel-goal interface enables a single checkpoint to handle five diverse tasks with positive cross-task transfer, eliminating the need for task-specific models. The architecture is inherently interpretable through CoT traces and pixel overlays, facilitating debugging and safety audits. The new benchmarks fill a critical gap in evaluating urban-scale navigation with social compliance and entrance-level accuracy.
• Limitations: The slow system’s inference cost (4B model) limits its frequency on edge devices; asynchronous execution mitigates this but introduces staleness. The person-following collision rate on DT/AT is slightly higher than baselines, suggesting room for collision-avoidance reward tuning.

Conclusion

ABot-N1 presents a general VLN foundation model whose slow–fast architecture factorizes high-level cognition from low-level control, using CoT traces and pixel-level grounding as a universal action interface. The dual visual-language guidance supports five navigation tasks from a single multi-task checkpoint, achieving state-of-the-art results on all benchmarks (70.9% SR on R2R-CE, 92.9% outdoor point-goal, 77.3% POI-goal, 84.9% object-goal, 90.1% person-following). Real-world deployment on a quadrupedal robot confirms generalization to diverse physical environments, demonstrating a viable path toward embodied navigation agents that are simultaneously robust, versatile, and interpretable. Future work (ABot-N1.1) will further verify data-centric scaling by independently expanding training corpora for perception and control.