GrepSeek: Training Search Agents for Direct Corpus Interaction

Summary (Overview)

• Introduces Direct Corpus Interaction (DCI): A novel paradigm where search agents interact directly with a raw text corpus via executable shell commands (e.g., grep, rg), bypassing traditional pre-computed retrieval indices.
• Proposes GrepSeek: A two-stage training pipeline for a compact LLM agent (Qwen3.5-9B) that learns effective, interpretable, and lexically precise retrieval behavior. The pipeline consists of:
  1. Cold-start SFT: Generates a synthetic dataset using an answer-aware Tutor and answer-blind Planner to create verified, causally grounded search trajectories.
  2. Policy Refinement: Uses Group Relative Policy Optimization (GRPO) to improve task-oriented search through direct RL on the corpus.
• Achieves Strong Performance: Outperforms index-based RAG and agentic search baselines on 4 out of 7 open-domain QA benchmarks, with significant gains on multi-hop reasoning tasks (NQ, HotpotQA, 2WikiMultihopQA, MuSiQue).
• Enables Efficient Execution: Develops a semantics-preserving sharded-parallel execution engine that accelerates shell-based retrieval by up to 7.6× (0.71s vs. 5.39s) while maintaining byte-exact equivalence with sequential execution, reducing end-to-end latency to ~8.6 seconds per query.

Introduction and Theoretical Foundation

Large Language Model (LLM) search agents typically access information through a retriever that queries a pre-computed index of document representations. This paper explores a complementary perspective: Direct Corpus Interaction (DCI). Here, the agent treats the corpus as a search environment and finds evidence by issuing executable shell commands (e.g., rg -F "keyword"). This enables:

• Surgical retrieval: Access to text at any granularity, not just pre-chunked documents.
• Explicit, controllable operations: Shifts from a black-box ranking procedure to a sequence of inspectable corpus operations.
• Effective for exact matching and compositional reasoning: Particularly useful for tasks requiring precise entity matching, lexical filtering, and following bridge entities across documents.

Contemporary work (Li et al., 2026; Sen et al., 2026) uses DCI as an inference-time prompting strategy with large, proprietary models, leading to computational expense and inefficiency. This paper focuses on training compact models to learn DCI as a capability, making it practical for real-world use.

Methodology

GrepSeek Agent Framework

The DCI search agent $π_θ$ operates within a ReAct framework. Given a question $q$ and corpus $C$ (a file with one document per line), it produces a trajectory $τ = \{(t_i, a_i, o_i)\}_{i=1}^T$, where:

• $t_i$: Reasoning trace.
• $a_i$: Action (shell command or termination).
• $o_i$: Observation (command output).

The agent uses a specific interaction format with think, <tool_call>, <tool_response>, and <answer> tags.

Two-Stage Training Pipeline

1. Cold-Start Data Generation (Algorithm 1) A pipeline using two LLMs (Qwen3.5-27B as Tutor $M_T$ and Planner $M_P$) generates verified training trajectories.

• Phase A (Backward Verification): The answer-aware Tutor, given the gold answer $y$, decomposes the question into sub-queries $(q_1,..., q_N)$ and constructs a retrieval chain backwards ($N → 1$). For each step, it proposes a target-masked shell command $c_i$ (forbidding the use of the target answer or its aliases) to retrieve a document $d_i$ that supports the current target answer $a$. A bridge extraction step identifies the entity in $d_i$ that answers the preceding sub-query, which becomes the target for the next hop.
• Phase B (Forward Assembly): The verified chain is reversed into chronological order. The answer-blind Planner drafts a reasoning trace and action based only on the observable history $H$. The Tutor then aligns this draft to logically motivate the verified command $c_i$ while remaining causally grounded, producing the final trajectory $T_{train}$.
• Phase C (Quality Filtering): Trajectories are filtered for answer quality ($F_1(\hat{y}, y) > 0$) and judged for causal/logical consistency to prevent information leakage.

2. Policy Optimization

• Supervised Fine-Tuning (SFT): The policy model (Qwen3.5-9B) is first fine-tuned on the 10k-sample cold-start dataset to learn stable retrieval behavior.
• Reinforcement Learning with GRPO: The policy is further optimized using Group Relative Policy Optimization (GRPO). For a query $q$, the policy samples a group of $n=5$ trajectories $τ^{(1)}, ..., τ^{(n)} \sim π_θ(· | q)$. Each trajectory receives a reward: $$R(τ^{(i)}) = ϕ(τ^{(i)}) \cdot R_{ans}(τ^{(i)})$$ where $ϕ(τ^{(i)}) \in \{0,1\}$ is a binary format indicator, and $R_{ans}(τ^{(i)})$ is the token-level $F_1$ score between the predicted answer $\hat{y}^{(i)}$ and the gold set $Y$. The advantage is computed as a relative score within the group: $$A^{(i)} = \frac{R(τ^{(i)}) - \text{mean}(\{R(τ^{(j)})\}_{j=1}^n)}{\text{std}(\{R(τ^{(j)})\}_{j=1}^n) + \epsilon}$$

Efficient Corpus Interaction Engine

To make DCI practical over large corpora (e.g., 21M documents, ~14GB), a semantics-preserving sharded-parallel execution engine is developed (Algorithm 2).

• Sharded-Parallel Search: The corpus is split into $S$ line-aligned shards. Compatible shell pipelines are executed in parallel across shards.
• Semantics Preservation: The engine classifies pipelines and applies deterministic merge strategies (CONCAT, HEAD, COUNT, SORTHEAD) to reconstruct output byte-exact to sequential execution. Incompatible or stateful pipelines fall back to sequential execution.
• Performance: This optimization reduces average retrieval latency from 5.39s (sequential) to 0.71s (32 shards), a 7.6× speedup.

Empirical Validation / Results

Experimental Setup

• Datasets: Seven open-domain QA benchmarks: Single-hop: Natural Questions (NQ), TriviaQA, PopQA. Multi-hop: HotpotQA, 2WikiMultihopQA (2Wiki), MuSiQue, Bamboogle.
• Corpus: 2018 Wikipedia dump (21M documents).
• Baselines: Include direct LLM, RAG, IRCoT, Search-O1, Rejection Sampling, and Search-R1 (GRPO-optimized) with three retrievers: BM25 (sparse), E5-110M (dense), and Qwen3-4B (dense).
• Primary Metric: Token-level $F_1$ score.

Main Findings

Table 1: Model performance ($F_1$ scores) across QA datasets.

↑/↓: statistically significant improvement/degradation vs. best baseline (p<0.05). Bold: best per column.

• Overall Performance: GrepSeek achieves the best overall micro-average $F_1$ score (0.5691), significantly outperforming the best dense retrieval baseline (Search-R1 with Qwen3-4B, 0.5441).
• Multi-hop Strength: Gains are most pronounced on multi-hop benchmarks (HotpotQA, 2Wiki), where DCI's lexical precision helps avoid semantic conflation and entity ambiguity common with dense retrievers.
• Limitations: Shows minor degradation on datasets with substantial surface-form variation (PopQA) or semantically broad phrasing, highlighting the brittleness of purely lexical search compared to semantic embedding generalization.

Efficiency Analysis (Figure 3):

• Inference Latency: GrepSeek has higher end-to-end latency (8.67s) than dense baselines (E5: 4.77s, Qwen3-4B: 6.07s), primarily due to longer reasoning and decoding.
• Memory & Preprocessing Cost: GrepSeek requires only 14 GB RAM (raw corpus size), eliminating the massive memory footprint of embedding indices (E5: 70 GB, Qwen3-4B: 221 GB) and expensive offline indexing (Qwen3-4B: 62.4 A100-hours).

Ablation Study (Table 2):

Training Dynamics (Figure 5): GrepSeek achieves higher rewards during training but generates longer sequences. Interestingly, it learns to reduce the number of commands over time by composing more expressive multi-stage shell pipelines.

Retrieval Behavior Analysis (Table 3):

• The agent consistently uses | head -n to limit output and .-F for exact-string matching.
• ~70% of commands use cascaded filtering (e.g., rg ... | rg ...).
• SFT establishes low-level syntactic "primitives," while RL refines higher-level search efficiency and reasoning depth.

Theoretical and Practical Implications

• DCI as a Competitive Paradigm: Establishes direct corpus interaction via learned shell commands as a practical and highly effective alternative to index-based retrieval, especially for tasks requiring precision and multi-hop reasoning.
• Interpretability & Control: Shifts retrieval from an opaque ranking to an interpretable sequence of operations, offering greater transparency and user control.
• Efficiency Trade-offs: Demonstrates a favorable trade-off: while inference latency increases, DCI eliminates costly offline indexing and drastically reduces memory requirements, simplifying deployment.
• Limitations Highlight Research Directions: The sensitivity to surface-form variation underscores the need for hybrid approaches combining lexical precision with semantic robustness.

Conclusion

GrepSeek demonstrates that compact LLMs can be effectively trained to perform direct, surgical retrieval over large text corpora using shell commands. The two-stage training pipeline (cold-start SFT + GRPO) stabilizes learning and yields an agent that excels at multi-hop reasoning through lexical precision. The optimized execution engine makes this approach practical at scale. While purely lexical interaction has limitations on semantically broad queries, GrepSeek establishes DCI as a highly competitive and practical paradigm for agentic search, complementary to existing retrieval methods. Future work will explore hybrid retrieval architectures, richer matching primitives, and improved inference efficiency.