# LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding > LocateAnything introduces Parallel Box Decoding, a method that predicts entire bounding boxes in one parallel step, achieving state-of-the-art accuracy and up to 2.5x faster inference across diverse visual grounding tasks. - **Source:** [arXiv](https://arxiv.org/abs/2605.27365) - **Published:** 2026-05-28 - **Permalink:** https://picx.dev/p/9Gh56k - **Whiteboard:** https://picx.dev/p/9Gh56k/image ## Summary # LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding ## Summary (Overview) * **Parallel Box Decoding (PBD):** Introduces a novel framework that decodes bounding boxes as atomic units in a single parallel step, breaking the token-by-token generation bottleneck of traditional vision-language models (VLMs). This preserves intra-box geometric coherence and enables substantial inference speedup. * **Unified High-Performance Grounding:** LocateAnything achieves state-of-the-art (SOTA) accuracy across diverse visual grounding and detection tasks, including object detection (COCO, LVIS), dense detection (Dense200, VisDrone), GUI grounding (ScreenSpot-Pro), document layout analysis (DocLayNet), and referring expression comprehension (RefCOCOg, HumanRef). * **Massive-Scale Training Data:** Develops a scalable data engine to curate **LocateAnything-Data**, a large-scale dataset with over 138 million training samples (12M unique images, 785M bounding boxes), significantly increasing data diversity for high-precision localization. * **Hybrid Inference for Speed-Robustness Trade-off:** Proposes three on-demand inference modes: **Fast Mode** (full parallel decoding for max throughput), **Slow Mode** (autoregressive for max stability), and a **Hybrid Mode** that defaults to parallel decoding and falls back to autoregressive generation only when unreliable blocks are detected, balancing speed and accuracy. * **Significant Speed-Up:** Achieves up to **2.5× higher decoding throughput** (12.7 Boxes Per Second in Hybrid Mode) compared to competitive methods like Rex-Omni (5.0 BPS), while simultaneously improving localization quality, especially at high IoU thresholds. ## Introduction and Theoretical Foundation Vision-language models (VLMs) are increasingly used as general-purpose backbones for interactive systems, requiring high-quality, low-latency localization of entities from natural language intents. Current VLM-based detection and grounding methods commonly formulate the problem as a **generative next-token prediction (NTP)** task, serializing 2D bounding box coordinates into a 1D token stream (e.g., as textual digits "1024" or quantized tokens $x_1 \rightarrow y_1 \rightarrow x_2 \rightarrow y_2$). This **token-by-token decoding creates a practical inference bottleneck** and fails to leverage the strong structured correlation among coordinates $(x_1, y_1, x_2, y_2)$. While **Multi-Token Prediction (MTP)** techniques in language modeling offer a path to parallel decoding, they are largely **structure-agnostic**, grouping tokens into arbitrary chunks. This can lead to learning spurious correlations across bounding-box boundaries, harming accuracy and reliability (see Figure 2 in the paper). **LocateAnything** addresses these issues by introducing **Parallel Box Decoding (PBD)**, which aligns MTP blocks with structured geometric units. The core idea is to treat each bounding box (or point) as an **atomic unit** and learn to predict its full coordinate set in one parallel step during training. This box-aligned training target avoids arbitrary token chunking, improving both localization performance and unlocking the speed benefits of parallel decoding. ## Methodology ### 3.1 Model Architecture and Formulation LocateAnything builds upon a native-resolution VLM with a Moon-ViT vision encoder and Qwen2.5 language decoder. It abandons standard NTP coordinate generation in favor of a **block-based output formulation**. * Continuous coordinates are normalized to $[0, 1000]$, discretized into tokens, and reorganized into a sequence of blocks $\mathbf{B} = (b_1, b_2, ..., b_N)$. * The joint probability is formulated as $P(\mathbf{B} | \mathcal{Z}, \mathcal{E}) = \prod_{i=1}^{N} P(b_i | b_{` and ``). Unoccupied positions are padded with a `` token. Four functional block types are defined: 1. **Semantic Block:** Encodes the linguistic identity. 2. **Box Block:** Contains four quantized coordinates for a bounding box. 3. **Negative Block:** Indicates the absence of a queried object. 4. **End Block:** Signals generation termination. ### 3.2 Training Design A dual-formulation training strategy jointly optimizes **NTP** (to preserve causal reasoning) and **block-wise MTP** (for box-aligned predictions). A single concatenated input sequence is constructed: $x_{\text{all}} = x_{\text{vis}} \oplus x_{\text{q}} \oplus x_{\text{ntp}} \oplus x_{\text{blk}}$, where $x_{\text{blk}}$ is created by traversing $x_{\text{ntp}}$, splitting it into blocks, retaining the first token per block as context, and masking subsequent tokens. A specialized **attention mask** (Figure 4) governs information flow: * **Causal Attention for NTP:** The shared context and NTP sequence use a causal mask, isolated from $x_{\text{blk}}$ to prevent leakage. * **Causal Flow Across Blocks:** Attention across different blocks in $x_{\text{blk}}$ is strictly causal (block $i$ can attend to blocks $