# RAGU: A Multi-Step GraphRAG Engine with a Compact Domain-Adapted LLM

> RAGU's multi-step GraphRAG engine with

- **Source:** [arXiv](https://arxiv.org/abs/2607.11683)
- **Published:** 2026-07-21
- **Permalink:** https://picx.dev/p/PBY7DM
- **Whiteboard:** https://picx.dev/p/PBY7DM/image

## Summary

## Summary (Overview)

- **RAGU** is a modular, multi-step GraphRAG engine that separates entity extraction from consolidation, using two-stage typed extraction, DBSCAN-backed deduplication, LLM summarization, and Leiden community detection to produce cleaner, more connected knowledge graphs.
- **Meno‑Lite‑0.1**, a 7 B model fine‑tuned for language skills (not world knowledge), outperforms Qwen2.5‑32B on knowledge‑graph construction (+12.5% relative harmonic mean) and rivals models up to ∼4× larger on downstream GraphRAG tasks.
- On GraphRAG‑Bench (Medical), RAGU retrieves the most complete context at every factoid level (evidence recall up to 0.84 vs. ≤ 0.76 for competitors) and overtakes HippoRAG 2 on synthesis tasks (Creative Generation AC and Coverage).
- The apparent HippoRAG 2 advantage on multi‑hop factoid QA is largely an answer‑format artifact; when format is controlled, RAGU ties or exceeds HippoRAG 2 on BioASQ and closes the gap on 2WikiMultiHopQA.
- RAGU is open‑source (MIT license), installable via `pip install graph_ragu`, runs on a single GPU, and includes engineering features such as Pydantic‑validated structured outputs, an async‑first API, and ∼374 automated tests with a deterministic mock LLM server.

## Introduction and Theoretical Foundation

Retrieval‑augmented generation (RAG) grounds LLMs in external knowledge, but traditional RAG retrieves flat chunks without capturing cross‑document relationships. GraphRAG addresses this by building a knowledge graph and using graph traversal during retrieval, yet practical adoption faces three obstacles:

1. **Single‑pass extraction** – existing systems construct knowledge graphs in a single LLM extraction pass, producing noisy, duplicated entities.
2. **Dependence on expensive LLMs** – practitioners default to large API models (GPT‑4‑class) under a false premise: the capabilities an LLM needs inside a RAG pipeline (comprehension, extraction, reasoning over context) are **language skills**, not factual recall.
3. **Engineering immaturity** – many open‑source frameworks suffer from installation failures, unsafe code paths (e.g., `eval()` on raw LLM output), and lack of testability.

**Language/World Knowledge Hypothesis.** The authors hypothesize that world knowledge scales near‑linearly with model size, whereas language skills scale markedly more slowly. They test this on the Qwen2.5‑Instruct family: on CheGeKa (world‑knowledge quiz), F1 increases $21.1\times$ from 0.5 B to 72 B, while on MultiQ (all facts in‑context) it increases only $4\times$. Log‑linear slopes are 0.65 vs. 0.26 (see Figure 1 in the paper). This prediction motivates a compact extractor.

## Methodology

### Multi‑Step Graph Construction (RAGU Pipeline)

RAGU processes documents through six configurable stages:

1. **Chunking** – three strategies: SimpleChunker (fixed‑size overlapping), SemanticTextChunker (embedding‑based split points), SmartSemanticChunker (cross‑encoder reranking).
2. **Two‑stage extraction** – entity extraction (Stage 1) and relation extraction (Stage 2) are separated. Entities are validated against the NEREL schema (29 entity types, 49 relation types); relations are constrained to validated entities, eliminating spurious mismatches. Optional ICL examples are injected.
3. **Consolidation** – EntitySummarizer groups entities by `(name, type)`; for entities with many duplicates, DBSCAN clustering and LLM summarization are applied. RelationSummarizer follows the same pattern. This step is absent from single‑pass systems like LightRAG.
4. **Community detection** – hierarchical Leiden clustering partitions the deduplicated graph.
5. **Community summarization** – an LLM generates structured reports (title, summary, findings).
6. **Refinement** – pluggable modules (e.g., `RemoveIsolatedNodes`) optionally clean the graph.

### Search Engines

RAGU provides five engines: `LocalSearch` (vector‑similarity entity retrieval expanded to relations and chunks), `GlobalSearch` (LLM‑rated community summarization), `NaiveSearch` (standard vector RAG), `MixSearch` (parallel multi‑engine), and `QueryPlanEngine` (DAG decomposition). All support cross‑encoder reranking and hybrid dense+sparse retrieval via Qdrant.

### Engineering and Deployment

- Three‑tier storage abstraction (graph/KV/vector) with lifecycle callbacks enables backend swapping (NetworkX → Neo4j, NanoVDB → Qdrant).
- Async‑first API with bounded concurrency.
- Structured LLM outputs validated through Pydantic v2, removing manual JSON post‑processing and preventing code injection.
- Incremental upsert/update/delete with deterministic hash‑based IDs and a consistency auditor.
- ∼374 tests and a deterministic mock LLM server enable CI without API keys.

### Compact Model: Meno‑Lite‑0.1

Meno‑Lite‑0.1 (7 B parameters) is derived from RuadaptQwen2.5‑7B through continued pre‑training (1.3 B tokens, Russian+English educational/scientific texts) and supervised fine‑tuning (50 M tokens) covering NEREL‑based extraction, multi‑hop QA, and query logs. The critical distinction: instructions teach the model to use **context** rather than recall facts, investing compute in language skills, not world knowledge. Key properties: 128 K context window (passkey retrieval 0.98 at 128 K), 47% better tokenizer efficiency than vanilla Qwen2.5 on Russian text, and single‑consumer‑GPU deployment via vLLM.

## Empirical Validation / Results

### GraphRAG‑Bench (Medical Domain)

Table 1 and Figure 3 show a cross‑over by task complexity. On factoid levels, HippoRAG 2 leads (e.g., Fact Retrieval AC 72.4 vs. 54.2 for RAGU). As tasks demand broad synthesis, the gap closes and reverses: on Creative Generation, RAGU wins AC (59.0 vs. 56.9) and Faithfulness (34.2 vs. 26.6). RAGU leads on Coverage throughout (57.4 vs. 34.7 on Creative Generation). LightRAG is weakest at every level.

**Retrieval results** confirm the mechanism: RAGU attains the highest Evidence Recall at every factoid level (84 vs. ≤ 76% for competitors), directly supporting the consolidation hypothesis.

**Table 1: Generation quality on GraphRAG‑Bench (Medical domain).** AC = Answer Correctness, Cov = Coverage, Faith = Faithfulness (all ×100). All systems use `bge-large-en-v1.5` for embeddings and `gpt-4o-mini` for answer generation.

| System | Index LLM | Fact Retr. AC | Complex Reas. AC | Contextual Summ. AC | Contextual Summ. Cov | Creative Gen. AC | Creative Gen. Cov | Creative Gen. Faith |
|--------|-----------|--------------|-----------------|-------------------|---------------------|-----------------|------------------|---------------------|
| LightRAG | Qwen2.5-7B | 25.9 | 20.3 | 22.1 | 51.0 | 14.2 | 3.1 | 23.1 |
| LightRAG | Meno-Lite-0.1 | 26.2 | 20.2 | 22.6 | 51.2 | 14.4 | 3.9 | 27.6 |
| HippoRAG 2 | Qwen2.5-7B | 72.7 | 67.9 | 64.6 | 51.6 | 57.2 | 33.2 | 31.5 |
| HippoRAG 2 | Meno-Lite-0.1 | 72.4 | 68.4 | 65.0 | 51.7 | 56.9 | 34.7 | 26.6 |
| RAGU | Qwen2.5-7B | 54.1 | 54.6 | 64.9 | 73.2 | 58.1 | 56.4 | 35.1 |
| RAGU | Meno-Lite-0.1 | 54.2 | 53.7 | 64.1 | 71.1 | 59.0 | 57.4 | 34.2 |

### Multi‑Hop QA Results

Table 2 reports two answer‑generation protocols. Under the verbose protocol (a), HippoRAG 2 dominates (AC up to 74.1 vs. 56.0 for RAGU on BioASQ). This gap is largely a format artifact: verbose answers mismatch terse gold references. Once format is controlled (b), RAGU ties HippoRAG 2 on BioASQ AC (72.9 vs. 72.4) and closes the 2WikiMultiHopQA gap from −19.3 pp to −5.5 pp. HippoRAG 2 retains a genuine lead only on MuSiQue (54.4 vs. 40.1), the hardest multi‑hop benchmark.

**Table 2: Multi‑hop QA under two answer‑generation protocols.** AC = Answer Correctness, RL = ROUGE‑L (×100). All systems use `gte-multilingual-base` for embeddings and `gpt-4o-mini` for answer generation.

| System | BioASQ AC | BioASQ RL | MuSiQue AC | MuSiQue RL | 2WikiMultiHop AC | 2WikiMultiHop RL |
|--------|-----------|-----------|------------|------------|-----------------|-----------------|
| **(a) Verbose generation prompt** | | | | | | |
| NaiveRAG | 55.3 | 12.4 | 41.7 | 6.9 | 43.5 | 12.0 |
| LightRAG (GPT) | 43.9 | 6.5 | 34.5 | 3.8 | 36.2 | 8.2 |
| HippoRAG 2 (GPT) | 74.1 | 48.8 | 56.3 | 42.7 | 65.9 | 54.7 |
| RAGU (GPT) | 56.0 | 12.2 | 43.5 | 7.6 | 46.6 | 13.2 |
| RAGU (Ours) | 54.5 | 12.1 | 42.0 | 7.4 | 45.2 | 12.9 |
| **(b) Terse generation prompt** | | | | | | |
| NaiveRAG | 71.7 | 49.2 | 36.6 | 24.7 | 53.7 | 45.2 |
| LightRAG (GPT) | 63.9 | 42.9 | 26.0 | 12.1 | 44.3 | 35.7 |
| HippoRAG 2 (GPT) | 72.4 | 48.8 | 54.4 | 42.6 | 63.5 | 54.7 |
| RAGU (GPT) | 72.9 | 48.7 | 40.1 | 26.5 | 58.0 | 49.6 |
| RAGU (Ours) | 72.8 | 48.2 | 40.7 | 27.5 | 55.1 | 46.3 |

### Model Evaluation (IE Benchmark)

Table 3 confirms the prediction: Meno‑Lite‑0.1 achieves the highest harmonic mean on the IE benchmark, outperforming Qwen2.5‑32B by 12.5% relative—driven by relation extraction (F1 0.347 vs. 0.239), the sub‑task most dependent on language comprehension.

**Table 3: IE benchmark (knowledge‑graph construction).** NER = entity recognition (F1), RE = relation extraction (F1), Def = entity definition (chrF++), RDef = relation definition (chrF++), HM = harmonic mean of all four tasks.

| Model | Size | NER | Def | RE | RDef | HM |
|-------|------|-----|-----|----|------|----|
| Meno‑Lite‑0.1 | 7B | 0.504 | 0.527 | 0.347 | 0.558 | **0.468** |
| Qwen2.5‑32B | 32B | 0.536 | 0.528 | 0.239 | 0.599 | 0.416 |
| gemma‑3‑27b | 27B | 0.544 | 0.482 | 0.224 | 0.583 | 0.396 |
| Qwen2.5‑14B | 14B | 0.510 | 0.518 | 0.222 | 0.583 | 0.396 |
| Qwen2.5‑7B | 7B | 0.477 | 0.479 | 0.192 | 0.541 | 0.356 |
| T‑lite‑1.0 | 7B | 0.466 | 0.464 | 0.174 | 0.533 | 0.336 |

**Fine‑tuning payoff vs. pipeline robustness.** Meno‑Lite‑0.1’s large standalone extraction edge compresses to ≤ 1 pp on end‑to‑end GraphRAG‑Bench QA—evidence that graph‑RAG QA quality is largely robust to extractor choice once consolidation is present.

## Theoretical and Practical Implications

- **Language skills scale weakly with model size.** The scaling analysis (21.1× vs. 4×, slopes 0.65 vs. 0.26) supports the hypothesis that a compact, skill‑or

---

_Markdown view of https://picx.dev/p/PBY7DM, served by PicX — AI-generated visual whiteboard summaries of research papers._