# A New Role for Relevance: Guiding Corpus Interaction in Agentic Search

> Relevance should guide corpus traversal and match visibility, not just top-k selection, achieving state-of-the-art accuracy with far fewer tool calls.

- **Source:** [arXiv](https://arxiv.org/abs/2607.24223)
- **Published:** 2026-07-30
- **Permalink:** https://picx.dev/p/ZIoiZE
- **Whiteboard:** https://picx.dev/p/ZIoiZE/image

## Summary

# Summary of "A New Role for Relevance: Guiding Corpus Interaction in Agentic Search"

## Summary (Overview)

- **New role for relevance**: The paper argues that relevance should directly guide corpus interaction (traversal order and match visibility) rather than merely selecting top-*k* content or constructing a workspace.
- **RARG agent**: The Relevance-Aware RipGrep Search Agent injects document-level relevance into grep traversal order, entry-point initialization, and match-level reranking to accelerate search convergence.
- **State-of-the-art results**: On BrowseComp-Plus (100K docs), RARG++ achieves 84% accuracy with GPT-5.4-mini (vs. 78% for DCI and RISE) while using only 23.9 tool calls (vs. 99.1 for DCI). On GPT-5.4, RARG++ reaches 91% accuracy, exceeding RISE by 9 points.
- **Robust to scaling**: When expanding the corpus to 1M documents, RARG++ retains 79% accuracy (vs. 69% for RISE-BM25) with minimal cost increase.
- **Reasoning retrieval**: On BRIGHT, RARG+ achieves 53.36 average nDCG@10, outperforming the retrieval-specialized NeMo agent (52.89).

## Introduction and Theoretical Foundation

The paper re-examines the role of **relevance** in agentic search. Relevance is defined as a *query-dependent estimate of how likely a document, passage, or matched excerpt is to contain useful evidence*. Traditional retrieval-augmented generation (RAG) and retrieval agents use relevance to rank documents and expose a top-*k* set, but this conflates document relevance with evidence utility:

- A relevant document may hide the decisive fact in a small span omitted by truncation.
- In multi-hop search, a document's usefulness may emerge only after an intermediate entity is discovered.
- Independent query-document ranking cannot itself localize, connect, compare, or verify evidence.

**Direct Corpus Interaction (DCI)** addresses this by allowing agents to search raw documents with general-purpose tools like grep, but DCI is relevance-agnostic – it scans all locations equally, causing useful clues to appear late or be lost under output truncation. This leads to poor **search convergence** (how quickly iterative interactions narrow an open-ended search into verified evidence).

Recent advances (Pi-Serini, RISE, DR-DCI) use retrieval to construct a bounded candidate space, but relevance remains a mechanism for *selecting the space* rather than a fine-grained execution signal for interactions *within* it.

The paper argues that relevance should guide corpus interaction at two complementary resolutions:

- **Global relevance**: determines where a pattern-matching operation searches first (traversal order).
- **Local relevance**: determines which matched excerpts become visible under the agent's observation budget.

This preserves DCI's high-resolution interface while using retrieval scores as an execution prior.

## Methodology

### 3.1 DCI-based Agent Structure

RARG is built on **DCI-Agent-Lite**, exposing two tools:
- **Bash(command)**: issues grep commands to find local matches across the corpus.
- **Read(doc_path, offset, limit)**: returns lines of a document when a match calls for more context.

Context management uses Level-3 compaction: truncate each tool result to *C* characters, and once the context length exceeds a threshold, retain only the most recent *T* turns.

### 3.2 Document-Level Relevance Insertion (RARG)

The key innovation is a new tool **embed_recall(scope_query)** that takes an LLM-generated query, uses an embedding retriever to rank documents, and writes the top-10,000 document paths to a temporary file `/tmp/scope_{N}.txt`. The LLM then uses:

```
cat /tmp/scope_{N}.txt | xargs -d '\n' rg [OPTIONS] "PATTERN"
```

A rule-based pass injects `-j1` to force sequential, single-threaded scanning, preserving the ranked order so that matches from more relevant documents are exposed earlier. This turns document-level relevance into search order rather than top-*k* content selection.

### 3.3 Entry Point Initialization (RARG+)

To avoid the agent wasting steps before finding a good entry point, the top-*X* documents of a scope are split into paragraphs (400–1000 characters), encoded with the embedding model, scored against the scope query, and the top-10 paragraphs are appended to the `embed_recall` output wrapped in `<qr_paragraph></qr_paragraph>`. This serves as a starting point without functioning as the answer itself.

### 3.4 Match-Level Relevance Insertion (RARG++)

To further speed convergence, the raw grep output is truncated, and informative excerpts from lower-ranked documents may be lost. RARG++ reranks up to *M* matches using the embedding model and presents only the top *m* to the LLM. The reranking query is **constructed** by concatenating the scope query (global goal) with the grep keywords (local intent):

```
Instruct: ...
Query: [scope query]
RG focus: [keyword1] [keyword2] ...
```

This gives more candidate clues a chance to reach the LLM under the observation budget. A generative variant (where the LLM supplies `rerank_query`) is also explored but degrades performance.

## Empirical Validation / Results

### 4.2 Results on Challenging QA (BrowseComp-Plus)

**Table 1: Main results on 100-query BrowseComp-Plus (100K doc corpus)**

| Model | Method | Acc ↑ | Turns | Tools | Search | Bash | Read |
|-------|--------|-------|-------|-------|--------|------|------|
| GPT-5.4 mini (med) | RISE | 78 | 24.3 | 28.7 | 13.1 | 9.2 | 6.4 |
| | DCI | 78 | 48.8 | 99.1 | – | 90.3 | 8.8 |
| | **RARG++** | **84** | **17.6** | **23.9** | 1.5 | 21.1 | 1.3 |
| GPT-5.4-nano (high) | RISE | 68 | 23.4 | 28.7 | 10.4 | 8.4 | 4.6 |
| | **RARG++** | **79** | 36.0 | 36.1 | 9.6 | 23.4 | 3.1 |
| GPT-5.4 (med) | RISE | 82 | 32.20 | 34.30 | 11.00 | 16.20 | 7.10 |
| | **RARG++** | **91** | **13.59** | **25.43** | 1.58 | 19.46 | 4.39 |

Key findings:
- RARG++ achieves the highest accuracy on every backbone while sharply reducing interaction cost.
- Relevance acts as an execution prior over rg traversal, not as the evidence channel itself (low Search counts).
- The generative variant of RARG++ converges fastest (17.8 tools) but drops to 75% accuracy, indicating a train–evaluation gap.

### 4.3 Scaling the Corpus (100K → 1M documents)

**Table 2: Results after scaling to 1M documents (GPT-5.4-mini)**

| Method | Acc ↑ | Turns ↓ | Tools ↓ | Search | Bash | Read |
|--------|-------|---------|---------|--------|------|------|
| RISE-BM25 | 69 | 25.4 | 32.1 | 20.3 | 8.3 | 3.5 |
| RARG | 78 | 19.3 | 31.8 | 1.5 | 28.7 | 1.7 |
| RARG+ | 78 | 21.9 | 30.4 | 1.5 | 27.5 | 1.4 |
| **RARG++** | **79** | **17.8** | **24.7** | 1.4 | 22.0 | 1.4 |

RARG++ maintains a 10-point margin over RISE-BM25. The cost increase is minimal, but performance degrades due to incidental lexical matches from long noisy documents.

### 4.4 Results on Reason-Intensive Retrieval (BRIGHT)

**Table 3: Retrieval effectiveness on BRIGHT (NDCG@10)**

| Method | Avg. ↑ | Bio. | Earth | Eco. | Rob. | Tools | Search | Bash | Read |
|--------|--------|------|-------|------|------|-------|--------|------|------|
| DCI | 48.43 | 62.05 | 54.94 | 37.13 | 39.59 | 40.04 | – | 14.73 | 25.31 |
| RISE-BM25 | 41.60 | 50.27 | 47.80 | 33.65 | 34.67 | 31.82 | 7.55 | 2.15 | 22.12 |
| NeMo Agent | 52.89 | 65.15 | 61.85 | 39.05 | 45.49 | 7.68 | 6.36 | – | – |
| **RARG+** | **53.36** | **66.70** | **62.16** | 37.23 | **47.34** | 27.55 | 1.23 | 9.87 | 16.45 |
| RARG++ | 50.55 | 61.65 | 61.32 | 36.14 | 43.10 | 27.28 | 1.14 | 8.40 | 17.74 |

RARG+ achieves the best average nDCG@10, outperforming the retrieval-specialized NeMo agent. The ordering flips relative to QA: RARG+ > RARG > RARG++, indicating that match-level reranking concentrates the observation budget (good for depth-first QA) but hurts when the task rewards wide recall.

### 4.5 Analysis of RARG Behavior

**Figure 3**: Heatmap of relevant-document hits over scope ranks. RARG stays front-loaded (relevant documents scanned first), while RARG++ recovers hits from lower-ranked documents via match-level reranking, confirming the two-level design.

**Figure 4**: Scope recall and RG coverage. Mini achieves 95–97% scope recall; nano is lower due to weaker instruction following. The `-j1` sequential scan dominates Bash usage, confirming the relevance-aware traversal.

## Theoretical and Practical Implications

**Theoretical implications**:
- Relevance should be viewed as an **execution signal** for corpus interaction, not a fixed content bottleneck. This shifts the paradigm from "retrieve then reason" to "retrieve to guide reasoning."
- The coarse-to-fine design (document-level order → entry-point initialization → match-level visibility) provides a principled framework for integrating retrieval with fine-grained search.
- The paper identifies a missing role of relevance in agentic search, filling the gap between corpus-level prioritization and local evidence exploitation.

**Practical implications**:
- RARG substantially improves the accuracy–efficiency trade-off, making expensive deep reasoning more affordable (e.g., 9-point accuracy gain with 46% cost reduction on GPT-5.4).
- The method is robust to corpus scaling, retaining effectiveness even when the corpus grows tenfold.
- For reasoning-intensive retrieval, relevance-aware interaction (RARG+) outperforms both pure retrieval agents and relevance-agnostic DCI.
- The approach is practical: it can be built on top of existing DCI harnesses and embedding models.

## Conclusion

The paper revisits the role of relevance in agentic search, arguing that it should guide corpus interaction rather than merely select inputs. The proposed **RARG** agent turns retrieval scores into an execution prior for grep exploration at two resolutions:

- **Document-level relevance** orders rg traversal so promising documents are scanned first.
- **Match-level relevance** controls which local excerpts remain visible under a limited observation budget.
- **Entry-point initialization** gives the agent a precise place to begin.

Across challenging QA, corpus scaling, and reasoning-intensive retrieval, RARG advances the accuracy–efficiency frontier over retrieval-only agents, unrestricted DCI, and retrieval-constructed interaction spaces. The analysis confirms that RARG keeps hits front-loaded toward relevant documents while recovering useful evidence from lower-ranked ones.

**Limitations**:
- Depends on embedding model quality; different granularities (document vs. match) may not be served equally well by the same model.
- Sequential `-j1` scanning and reranking incur extra latency.
- Sensitive to backbone instruction-following ability (GPT-5.4-nano is less reliable).
- Not immune to corpus-induced interference from long noisy documents.
- Generative reranking variant shows a train–evaluation gap that needs bridging.

Future work includes closing the generative reranking gap, broader validation across backbones and domains, and reducing the weakness of local matching under noisy distractors.

---

_Markdown view of https://picx.dev/p/ZIoiZE, served by PicX — AI-generated visual whiteboard summaries of research papers._