# K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs

> LLMs lack curriculum cognition (best 57.1% on K12-Bench), but knowledge-graph-guided SFT is highly sample-efficient for educational tasks.

- **Source:** [arXiv](https://arxiv.org/abs/2605.09635)
- **Published:** 2026-07-25
- **Permalink:** https://picx.dev/p/smAVs7
- **Whiteboard:** https://picx.dev/p/smAVs7/image

## Summary

## Summary (Overview)

- **K12-KGraph**: A curriculum-aligned knowledge graph extracted from official Chinese K–12 textbooks (People's Education Press) covering mathematics, physics, chemistry, and biology. It contains nine node types and fourteen relation types spanning both curriculum structure and visual grounding.
- **K12-Bench**: A 23,640-question multi-select benchmark across five graph-derived task families (Ground, Prereq, Neighbor, Evidence, Locate) that jointly probe "curriculum cognition"—the structured understanding of prerequisite chains, concept taxonomies, experiment–concept links, and pedagogical sequencing.
- **K12-Train**: A KG-guided supervised fine-tuning corpus of 7,335 samples, comprising 2,267 text-only QA pairs (K12-Train-Text) and 5,068 multimodal VQA pairs (K12-Train-MM).
- **Key finding**: Even strong LLMs struggle with curriculum cognition—Gemini-3-Flash reaches only 57.1% exact match on K12-Bench. However, domain-specific SFT with K12-Train is highly sample-efficient, consistently outperforming equally sized subsets of eight mainstream instruction-tuning corpora on GaokaoBench and EduEval.
- **Complementarity**: K12-Train-Full (text + multimodal) consistently outperforms both text-only and multimodal-only variants across three multimodal educational benchmarks.

## Introduction and Theoretical Foundation

Large language models (LLMs) have become proficient at answering K–12 exam questions, rivaling top human students on benchmarks such as C-Eval, CMMLU, GaokaoBench, and EduEval. However, these benchmarks measure only factual recall—whether a model can answer an exam question. Effective educational AI requires *curriculum cognition*: the structured understanding of how knowledge is organized and visually presented, including prerequisite chains, concept taxonomies, experiment–concept links, and pedagogical sequencing. Curriculum cognition is neither probed by current benchmarks nor explicitly taught by current instruction-tuning data.

The authors argue that a good teacher knows not just facts but also the relationships between concepts (e.g., linear equations require arithmetic operations as a prerequisite) and the location of each idea in the textbook. They also note that curriculum knowledge is communicated through textbook figures (diagrams, experimental setups, etc.), which current benchmarks ignore.

To close this gap, they build everything from a single resource: a curriculum-aligned knowledge graph extracted directly from official Chinese K–12 textbooks (People's Education Press). The graph mirrors how the curriculum is organized, enabling both evaluation and training.

## Methodology

**K12-KGraph Construction (5-stage pipeline):**
1. **OCR-based parsing**: Textbook PDFs are converted into structured Markdown using MinerU, preserving heading hierarchy, math formulas, text, and images.
2. **Table-of-contents parser**: Produces a section index and splits Markdown into per-section files, associating each image with its section text.
3. **LLM-based extraction**: For each section, GPT-5.2 is prompted with a schema-aware instruction to emit nodes and edges as structured JSON, along with evidence citations and confidence scores.
4. **Hierarchical merging**: Per-section graphs are merged bottom-up—book-level deduplication and ID assignment, then subject-level reconciliation across books.
5. **Topological validation**: Depth-first cycle detection on `is_a` and `prerequisites_for` subgraphs ensures valid DAGs.

**Schema**: Nine node types (*Book, Chapter, Section, Concept, Skill, Experiment, Exercise, Figure, VisualElement*) and fourteen directed edge types (e.g., `is_a`, `prerequisites_for`, `verifies`, `illustrates`, `requires_figure`). Each node carries typed properties (e.g., *Concept* includes name, definition, importance, formula).

**K12-Bench Construction:**
- Five task families (nine subtasks) derived from graph neighborhoods: *Ground* (knowledge grounding via `tests_concept`/`tests_skill`), *Prereq* (prerequisite reasoning via `prerequisites_for`), *Neighbor* (neighbor recommendation via `is_a`/`relates_to`), *Evidence* (experiment evidence chain via `verifies`), *Locate* (cross-chapter indexing via `appears_in`/`leads_to`).
- Questions are multi-select with 4 labeled options; correct answers are true graph neighbors; distractors are sampled from structurally proximate but non-answer nodes (e.g., 2-hop neighborhoods), filtered by 3-gram similarity and LLM-based pedagogical check.

**K12-Train Construction:**
Three paths: (i) node-grounded QA (LLM-prompted from *Concept*, *Skill*, *Experiment*, *Exercise* node properties); (ii) edge-grounded QA (LLM-prompted with templates for each relation type); (iii) exercise-assessment QA (deterministic templates for `tests_concept`/`tests_skill` edges). Quality control includes cropping irrelevant properties, filtering low-confidence edges, and validating JSON structure.

## Empirical Validation / Results

**Benchmarking on K12-Bench (Table 3):**

| Model | Ground EM/F1 | Prereq EM/F1 | Neighbor EM/F1 | Evidence EM/F1 | Locate EM/F1 | Overall EM/F1 |
|---|---|---|---|---|---|---|
| Random | 6.7/36.2 | 6.7/37.9 | 6.7/41.3 | 6.7/37.7 | 6.7/32.9 | 6.7/36.4 |
| Gemma-4-31B-IT | 50.6/79.0 | 28.3/62.6 | 15.0/60.7 | 43.3/73.9 | 73.4/73.5 | **46.4/69.5** |
| Gemini-3-Flash | 63.4/83.3 | 34.8/58.2 | 33.4/63.5 | 47.4/72.4 | 81.7/82.6 | **57.1/73.0** |

Key findings: Even strong models struggle, with Prereq and Neighbor being hardest. Meta-LLaMA-3-8B-Instruct (EM=7.2%) is near random (6.7%).

**Text-only SFT (Tables 4 & 5):** Under a matched 2,300-sample budget, K12-Train-Text consistently outperforms eight mainstream corpora (OpenHermes, Infinity, UltraChat, WizardLM, DataFlow, LMSYS, SmolTalk, Tulu-3) on both GaokaoBench and EduEval, across two base models (Qwen3-4B-Base, Llama3.1-8B-Base). Example: on GaokaoBench (Qwen3-4B-Base), K12-Train-Text achieves total 1009.96 vs. strongest baseline DataFlow 985.91 (+24.1).

**Multimodal SFT (Tables 6, 7, 8):** On Qwen3.5-2B-Base, K12-Train-Full achieves the best overall performance on Gaokao-MM (39.9%), MDK12-medium (52.94), and K12Vista (79.95), outperforming both K12-Train-Text and K12-Train-MM, and surpassing full DataFlow and WizardLM datasets despite using far fewer samples (7,335 vs. 10,000/142,759).

**Analysis**: Cross-subject transfer benefits (e.g., improved Chinese and Humanities Math despite no in-domain supervision) suggest K12-Train teaches transferable structural reasoning, not just content memorization.

## Theoretical and Practical Implications

The paper demonstrates a fundamental gap in current LLM capabilities: strong factual recall does not imply robust curriculum cognition. This has direct implications for building AI tutors and educational tools—merely answering exam questions is insufficient; models need to understand how knowledge is structured.

The K12-KGraph framework provides a principled way to:
- **Benchmark** structural understanding (curriculum cognition)
- **Generate** training data that explicitly teaches these relationships (sample-efficient, grounded in official curriculum)

The complementarity of text and visual supervision suggests that multimodal educational AI systems benefit from both modalities. The graph-grounded approach also offers a reusable resource: improvements to the KG propagate directly to both benchmark and training data.

## Conclusion

The authors introduce K12-KGraph, a curriculum-aligned knowledge graph from official Chinese K–12 textbooks, along with K12-Bench (23,640 questions across five task families) and K12-Train (7,335 SFT samples). Experimental results show that:
1. Current LLMs lack robust curriculum cognition (best EM 57.1% on K12-Bench).
2. KG-guided SFT is highly sample-efficient, outperforming much larger general-purpose corpora on educational benchmarks.
3. Textual and visual supervision are complementary for multimodal educational settings.

The resources (graph, benchmark, training data, and construction pipeline) are publicly released. Future work could extend to additional subjects, languages, or more fine-grained pedagogical reasoning tasks.

---

_Markdown view of https://picx.dev/p/smAVs7, served by PicX — AI-generated visual whiteboard summaries of research papers._