# LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory > LoGeR introduces a hybrid memory module that enables long-context 3D reconstruction by combining lossless local alignment with compressed global context, achieving state-of-the-art accuracy with linear complexity. - **Source:** [arXiv](https://arxiv.org/abs/2603.03269) - **Published:** 2026-03-11 - **Permalink:** https://picx.dev/p/jJmbNY - **Whiteboard:** https://picx.dev/p/jJmbNY/image ## Summary # LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory - Summary ## Summary (Overview) * **Architectural Innovation:** Introduces LoGeR, a novel feedforward architecture for scaling dense 3D reconstruction to thousands of frames. It combines **causal chunk-wise processing** with a **hybrid memory module**. * **Hybrid Memory Design:** The memory module synergizes two components: a **non-parametric Sliding Window Attention (SWA)** for lossless, high-precision alignment between adjacent chunks, and a **parametric Test-Time Training (TTT)** memory for compressed, long-range global context to prevent scale drift. * **Breaking Data and Context Walls:** The model is trained on short sequences (128 frames) but generalizes to extremely long videos (up to 19k frames) during inference by leveraging chunk-based processing and a curriculum training strategy on diverse, large-scale datasets. * **State-of-the-Art Performance:** Demonstrates substantial improvements over prior feedforward and optimization-based methods. Reduces Absolute Trajectory Error (ATE) on KITTI by **over 74%** and achieves a **30.8%** relative improvement on the expansive VBR benchmark. * **Comprehensive Evaluation:** Validated on standard benchmarks (KITTI, ScanNet, TUM) and a new long-context benchmark derived from VBR, spanning trajectories up to 11.5 km. ## Introduction and Theoretical Foundation Large-scale, dense 3D reconstruction is a core challenge in computer vision. While classical optimization-based methods (e.g., SLAM) scale to large scenes, they are computationally intensive and offline. Recent **geometric foundation models** (e.g., DUSt3R, π³) enable robust, feedforward inference but are confined to short sequences due to the quadratic complexity of global attention and a lack of long-horizon training data—termed the **"context wall"** and **"data wall."** The paper argues that **end-to-end chunk-wise processing** is a practical strategy to overcome these walls, as it keeps local inferences within the distribution of existing short-context training data. However, this introduces the critical challenge of maintaining coherence across chunk boundaries. Existing sequential models (recurrent or causal) fail to balance the need for **high-fidelity local detail**, **uncompressed context for adjacent alignment**, and **global structural integrity**. **Theoretical Insight:** A single memory strategy is insufficient. The paper proposes a **learning-based hybrid memory module** that decouples these tasks using complementary mechanisms, as summarized in the architectural trade-offs table (Table 1). > **Table 1. Architectural trade-offs in sequence modeling.** Our hybrid memory module achieves a balance, preserving lossless local geometric details while maintaining global structural consistency at a linear computational cost with respect to sequence length. > > | Mechanism | Compute Cost | Local Context | Global Context | > | :--- | :--- | :--- | :--- | > | Full Attention | $O(N^2)$ | Lossless | Lossless | > | Sliding Window Attn. | $O(N)$ | Lossless | Limited | > | TTT / Linear Attn. | $O(N)$ | Compressed | Compressed | > | **Ours (Hybrid Memory)** | **$O(N)$** | **Lossless** | **Compressed** | ## Methodology LoGeR processes an input video stream $X = \\{ I_t \\}_{t=1}^T$ sequentially in $M$ chunks $\\{C_m\\}_{m=1}^M$ with minimal overlap (e.g., one frame). Each chunk is processed by a strong bidirectional geometry backbone (e.g., π³). The novel **hybrid memory module** is integrated into the network blocks to propagate information across chunks. **1. Block Structure (Fig. 2):** For chunk $C_m$ with token sequence $H^{C_m}$ entering a block: * **Per-frame attention:** Self-attention applied independently to each frame's tokens. $$ H^{C_m} \leftarrow H^{C_m} + [\text{Attn}_{\text{frame}}(\text{LN}(H^{C_m}_i); \theta), | i \in \\{1,...,n\\}] $$ * **Sparse Sliding-Window Attention (SWA):** Applied at only 4 network depths to align adjacent chunks $C_{m-1}$ and $C_m$. $$ H^{C_m} \leftarrow H^{C_m} + \text{Attn}_{\text{swa}}([\text{LN}(H^{C_{m-1}}), \text{LN}(H^{C_m})]; \theta) $$ * **Chunk-wise TTT Layer (Fast Weights):** Maintains fast weights $W_m$ summarizing history up to chunk $m$. * **Apply:** Injects memory into current tokens. $$ \tilde{H}^{C_m} = H^{C_m} + f_{W_m}(\text{LN}(H^{C_m})) $$ * **Update:** Writes chunk summary into $W$ for the next chunk. $$ W_{m+1} = \mathcal{U}(W_m; H^{C_m}) $$ * **Chunk-wise Bidirectional Attention:** Final powerful geometric reasoning within the chunk. $$ H^{C_m} \leftarrow \tilde{H}^{C_m} + \text{BiAttn}_{\text{chunk}}(\text{LN}(\tilde{H}^{C_m}); \theta) $$ **2. Learning Objectives:** The model is trained with a combination of losses from π³: * **Scale-invariant local pointmap loss** $L_{\text{local}}$ (Eq. 8). * **Affine-invariant relative pose loss** $L_{\text{pose}}$ (Eq. 9). * **Additional global pointmap loss** $L_{\text{global}}$ to over-constrain long-sequence training (Eq. 10). The total loss is: $L = L_{\text{local}} + L_{\text{pose}} + \lambda_{\text{global}} L_{\text{global}}$. **3. LoGeR* Variant (Feedforward Alignment):** To mitigate error accumulation in extremely long streams, an optional SE(3) alignment step is applied using overlapping frames between consecutive chunks. If $\hat{T}_k^{(m)}$ is the raw predicted pose of the overlapping frame $k$ in chunk $C_m$, and $\tilde{T}^{(m-1)}$ are the aligned poses of the previous chunk, the alignment $A_m$ and final poses are computed as: $$ A_m = \tilde{T}_k^{(m-1)} (\hat{T}_k^{(m)})^{-1}, \quad \tilde{T}_t^{(m)} = A_m \hat{T}_t^{(m)}, \forall t \in C_m $$ **4. Data and Curriculum Training:** * **Data Mixture:** To overcome the "data wall," training uses a mixture heavily weighted towards **large-scale navigation datasets** (e.g., TartanAirV2, Waymo). * **Curriculum Strategy:** Training progresses in stages: starting with few chunks (4), increasing chunk density (to 12), and finally scaling context length to 128 frames over 20 chunks. This stabilizes TTT optimization and shifts reliance from local SWA to the global TTT state. ## Empirical Validation / Results **1. Long-Sequence Benchmarks (KITTI & VBR):** * **KITTI:** LoGeR substantially outperforms all prior feedforward methods (Table 2). The variant **LoGeR\*** achieves an average ATE of **18.65 m**, a **74.4% reduction** from the previous best feedforward method (TTT3R: 72.86 m). * **VBR (Repurposed):** On sequences up to ~19k frames, LoGeR shows clear quantitative (Fig. 4) and qualitative (Fig. 5) superiority, maintaining global scale consistency where baselines like Pi3-Chunk suffer from severe drift. > **Table 2. Comparison of Absolute Trajectory Error (ATE ↓, m) on KITTI.** Bold and underline indicate the best and second-best performance among feedforward methods. > | Methods | 00 | 01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 | 10 | **Avg.** | > | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | > | **Opt.-based** | | | | | | | | | | | | | > | DROID-SLAM | 92.10 | 344.60 | 107.61 | 2.38 | 1.00 | 118.50 | 62.47 | 21.78 | 161.60 | 72.32 | 118.70 | 100.28 | > | VGGT-Long | 8.67 | 121.17 | 32.08 | 6.12 | 4.23 | 8.31 | 5.34 | 4.63 | 53.10 | 41.99 | 18.37 | **27.64** | > | **Feedforward** | | | | | | | | | | | | | > | TTT3R | 119.94 | 99.59 | 238.07 | 16.83 | 3.98 | 36.38 | 47.20 | 11.62 | 107.33 | 86.96 | 33.58 | 72.86 | > | Pi3-Chunk (Baseline) | 26.65 | 196.04 | 157.92 | 5.13 | 1.09 | 12.79 | 27.66 | 5.94 | 61.26 | 56.31 | 21.96 | 52.07 | > | **LoGeR (Ours)** | 62.34 | 41.64 | 39.64 | 4.89 | 1.82 | 41.27 | 13.99 | 16.24 | 26.46 | 22.71 | 8.84 | 25.44 | > | **LoGeR* (Ours)** | **30.47** | **47.91** | **36.32** | **5.38** | **1.95** | **26.34** | **6.60** | **5.55** | **24.41** | **10.12** | **10.11** | **18.65** | **2. Short-Sequence Benchmarks (7-Scenes, ScanNet, TUM):** * **3D Reconstruction (7-Scenes):** LoGeR achieves a **69.2%** lower Chamfer Distance than prior work (Fig. 6) and produces qualitatively superior, undistorted geometry (Fig. 7). * **Pose Estimation (ScanNet & TUM):** LoGeR and the Pi3-Chunk baseline significantly outperform prior methods, with relative gains of **80.0%** on ScanNet and **66.1%** on TUM-Dynamics (Fig. 9). **3. Ablation Study (Table 3):** * **Architecture:** Removing either TTT or SWA components leads to performance drops, confirming the necessity of the hybrid design. Qualitative analysis (Fig. 10) shows TTT prevents long-range drift, while SWA ensures local smoothness. * **Data Mixture:** Excluding large-scale datasets from training degrades performance, validating the need to overcome the "data wall." * **Curriculum Training:** The progressive curriculum strategy consistently improves performance for both LoGeR and LoGeR*. > **Table 3. Ablation results on ScanNet and TUM datasets (ATE ↓).** > | Method | ScanNet (500f) | ScanNet (1000f) | TUM (500f) | TUM (1000f) | > | :--- | :---: | :---: | :---: | :---: | > | LoGeR | **0.087** | **0.107** | **0.033** | **0.050** | > | w/o TTT | 0.108 | 0.162 | 0.043 | 0.079 | > | w/o SWA | 0.115 | 0.143 | 0.039 | 0.053 | > | All datasets | **0.087** | **0.107** | **0.033** | **0.050** | > | w/o 5 large datasets | 0.102 | 0.156 | 0.050 | 0.072 | ## Theoretical and Practical Implications * **Theoretical:** Demonstrates that a **dual-component memory system**—pairing lossless local memory (SWA) with lossy global memory (TTT)—is an effective architectural principle for tasks requiring both high local fidelity and long-range consistency under linear compute constraints. * **Practical:** Provides a **fully feedforward alternative** to SLAM systems for long-context 3D reconstruction, enabling applications in robotics and autonomous driving without backend optimization. The method's ability to generalize from short training sequences to extremely long inference sequences reduces data dependency. * **Benchmarking:** Introduces a repurposed **VBR dataset** as a challenging benchmark for evaluating long-context geometric reconstruction at an unprecedented scale (up to ~19k frames, 11.5 km). ## Conclusion LoGeR successfully scales feedforward dense 3D reconstruction to minute-long videos by introducing a **causal chunk-wise architecture with a hybrid memory module**. The synergy between **Sliding Window Attention** (for local consistency) and **Test-Time Training memory** (for global consistency) allows the model to be trained on short sequences and generalize to thousands of frames. LoGeR sets a new state-of-the-art on long-context benchmarks and opens avenues for long-context spatio-temporal reasoning. **Future Directions:** Addressing the **length-generalization bottleneck** of TTT weights beyond training context, curating more **diverse long-horizon datasets**, and extending the hybrid memory architecture to other domains requiring long-term global and local coherence. --- _Markdown view of https://picx.dev/p/jJmbNY, served by PicX — AI-generated visual whiteboard summaries of research papers._