Summary (Overview)

• RAGU is a modular, multi-step GraphRAG engine that separates entity extraction from consolidation, using two-stage typed extraction, DBSCAN-backed deduplication, LLM summarization, and Leiden community detection to produce cleaner, more connected knowledge graphs.
• Meno‑Lite‑0.1, a 7 B model fine‑tuned for language skills (not world knowledge), outperforms Qwen2.5‑32B on knowledge‑graph construction (+12.5% relative harmonic mean) and rivals models up to ∼4× larger on downstream GraphRAG tasks.
• On GraphRAG‑Bench (Medical), RAGU retrieves the most complete context at every factoid level (evidence recall up to 0.84 vs. ≤ 0.76 for competitors) and overtakes HippoRAG 2 on synthesis tasks (Creative Generation AC and Coverage).
• The apparent HippoRAG 2 advantage on multi‑hop factoid QA is largely an answer‑format artifact; when format is controlled, RAGU ties or exceeds HippoRAG 2 on BioASQ and closes the gap on 2WikiMultiHopQA.
• RAGU is open‑source (MIT license), installable via pip install graph_ragu, runs on a single GPU, and includes engineering features such as Pydantic‑validated structured outputs, an async‑first API, and ∼374 automated tests with a deterministic mock LLM server.

Introduction and Theoretical Foundation

Retrieval‑augmented generation (RAG) grounds LLMs in external knowledge, but traditional RAG retrieves flat chunks without capturing cross‑document relationships. GraphRAG addresses this by building a knowledge graph and using graph traversal during retrieval, yet practical adoption faces three obstacles:

1. Single‑pass extraction – existing systems construct knowledge graphs in a single LLM extraction pass, producing noisy, duplicated entities.
2. Dependence on expensive LLMs – practitioners default to large API models (GPT‑4‑class) under a false premise: the capabilities an LLM needs inside a RAG pipeline (comprehension, extraction, reasoning over context) are language skills, not factual recall.
3. Engineering immaturity – many open‑source frameworks suffer from installation failures, unsafe code paths (e.g., eval() on raw LLM output), and lack of testability.

Language/World Knowledge Hypothesis. The authors hypothesize that world knowledge scales near‑linearly with model size, whereas language skills scale markedly more slowly. They test this on the Qwen2.5‑Instruct family: on CheGeKa (world‑knowledge quiz), F1 increases $21.1\times$ from 0.5 B to 72 B, while on MultiQ (all facts in‑context) it increases only $4\times$. Log‑linear slopes are 0.65 vs. 0.26 (see Figure 1 in the paper). This prediction motivates a compact extractor.

Methodology

Multi‑Step Graph Construction (RAGU Pipeline)

RAGU processes documents through six configurable stages:

1. Chunking – three strategies: SimpleChunker (fixed‑size overlapping), SemanticTextChunker (embedding‑based split points), SmartSemanticChunker (cross‑encoder reranking).
2. Two‑stage extraction – entity extraction (Stage 1) and relation extraction (Stage 2) are separated. Entities are validated against the NEREL schema (29 entity types, 49 relation types); relations are constrained to validated entities, eliminating spurious mismatches. Optional ICL examples are injected.
3. Consolidation – EntitySummarizer groups entities by (name, type); for entities with many duplicates, DBSCAN clustering and LLM summarization are applied. RelationSummarizer follows the same pattern. This step is absent from single‑pass systems like LightRAG.
4. Community detection – hierarchical Leiden clustering partitions the deduplicated graph.
5. Community summarization – an LLM generates structured reports (title, summary, findings).
6. Refinement – pluggable modules (e.g., RemoveIsolatedNodes) optionally clean the graph.

Search Engines

RAGU provides five engines: LocalSearch (vector‑similarity entity retrieval expanded to relations and chunks), GlobalSearch (LLM‑rated community summarization), NaiveSearch (standard vector RAG), MixSearch (parallel multi‑engine), and QueryPlanEngine (DAG decomposition). All support cross‑encoder reranking and hybrid dense+sparse retrieval via Qdrant.

Engineering and Deployment

• Three‑tier storage abstraction (graph/KV/vector) with lifecycle callbacks enables backend swapping (NetworkX → Neo4j, NanoVDB → Qdrant).
• Async‑first API with bounded concurrency.
• Structured LLM outputs validated through Pydantic v2, removing manual JSON post‑processing and preventing code injection.
• Incremental upsert/update/delete with deterministic hash‑based IDs and a consistency auditor.
• ∼374 tests and a deterministic mock LLM server enable CI without API keys.

Compact Model: Meno‑Lite‑0.1

Meno‑Lite‑0.1 (7 B parameters) is derived from RuadaptQwen2.5‑7B through continued pre‑training (1.3 B tokens, Russian+English educational/scientific texts) and supervised fine‑tuning (50 M tokens) covering NEREL‑based extraction, multi‑hop QA, and query logs. The critical distinction: instructions teach the model to use context rather than recall facts, investing compute in language skills, not world knowledge. Key properties: 128 K context window (passkey retrieval 0.98 at 128 K), 47% better tokenizer efficiency than vanilla Qwen2.5 on Russian text, and single‑consumer‑GPU deployment via vLLM.

Empirical Validation / Results

GraphRAG‑Bench (Medical Domain)

Table 1 and Figure 3 show a cross‑over by task complexity. On factoid levels, HippoRAG 2 leads (e.g., Fact Retrieval AC 72.4 vs. 54.2 for RAGU). As tasks demand broad synthesis, the gap closes and reverses: on Creative Generation, RAGU wins AC (59.0 vs. 56.9) and Faithfulness (34.2 vs. 26.6). RAGU leads on Coverage throughout (57.4 vs. 34.7 on Creative Generation). LightRAG is weakest at every level.

Retrieval results confirm the mechanism: RAGU attains the highest Evidence Recall at every factoid level (84 vs. ≤ 76% for competitors), directly supporting the consolidation hypothesis.

Table 1: Generation quality on GraphRAG‑Bench (Medical domain). AC = Answer Correctness, Cov = Coverage, Faith = Faithfulness (all ×100). All systems use bge-large-en-v1.5 for embeddings and gpt-4o-mini for answer generation.

Multi‑Hop QA Results

Table 2 reports two answer‑generation protocols. Under the verbose protocol (a), HippoRAG 2 dominates (AC up to 74.1 vs. 56.0 for RAGU on BioASQ). This gap is largely a format artifact: verbose answers mismatch terse gold references. Once format is controlled (b), RAGU ties HippoRAG 2 on BioASQ AC (72.9 vs. 72.4) and closes the 2WikiMultiHopQA gap from −19.3 pp to −5.5 pp. HippoRAG 2 retains a genuine lead only on MuSiQue (54.4 vs. 40.1), the hardest multi‑hop benchmark.

Table 2: Multi‑hop QA under two answer‑generation protocols. AC = Answer Correctness, RL = ROUGE‑L (×100). All systems use gte-multilingual-base for embeddings and gpt-4o-mini for answer generation.

Model Evaluation (IE Benchmark)

Table 3 confirms the prediction: Meno‑Lite‑0.1 achieves the highest harmonic mean on the IE benchmark, outperforming Qwen2.5‑32B by 12.5% relative—driven by relation extraction (F1 0.347 vs. 0.239), the sub‑task most dependent on language comprehension.

Table 3: IE benchmark (knowledge‑graph construction). NER = entity recognition (F1), RE = relation extraction (F1), Def = entity definition (chrF++), RDef = relation definition (chrF++), HM = harmonic mean of all four tasks.

Fine‑tuning payoff vs. pipeline robustness. Meno‑Lite‑0.1’s large standalone extraction edge compresses to ≤ 1 pp on end‑to‑end GraphRAG‑Bench QA—evidence that graph‑RAG QA quality is largely robust to extractor choice once consolidation is present.

Theoretical and Practical Implications

• Language skills scale weakly with model size. The scaling analysis (21.1× vs. 4×, slopes 0.65 vs. 0.26) supports the hypothesis that a compact, skill‑or