Summary (Overview)

• Problem: Visual generators excel at rendering but confidently fabricate unseen concepts (new characters, cultural symbols, post-cutoff events), revealing a world-knowledge bottleneck distinct from a visual-synthesis bottleneck. The paper shows frontier open generators score only 21–28/100 on knowledge-intensive prompts, a 40-point collapse invisible to existing benchmarks.

• Key Insight: The paper formalizes a generator-specific, evolving knowledge boundary — the partition between knowledge a generator can internalize through training ($\mathcal{K}_{\text{int}}$) and knowledge that must remain in external context ($\mathcal{K}_{\text{ctx}}$). This boundary is discoverable through a teach-then-search co-training framework.

• Naive Search Fails: Blind search degrades prompts the generator already handles (e.g., Qwen-Image-2 drops from 70.7 → 60.4 on no-search prompts), introducing concept corruption and copy effects. Two failure modes motivate a gate-filter-integrate protocol.

• Contributions: Release of SEARCHGEN-20K (20,839 prompts, 12 failure categories, 22 domains), SEARCHGEN-BENCH (751-prompt test set), and SEARCHGEN-CORPUS-1M (145,642 archived search sessions for offline replay).

• Result: A minimal recipe (8B reasoner + 4B generator, one DPO pass + one RFT pass) produces monotonic improvement across difficulty tiers, with the co-trained 8B reasoner matching or exceeding a frontier VLM oracle (31.8 vs. 31.2 on Klein-4B).

---

Introduction and Theoretical Foundation

Background and Motivation

Modern image generators saturate standard benchmarks (GenAI-Bench, T2I-CompBench) but fail on substantial real-world requests: the 2025 Osaka Expo mascot, historically accurate Spartan phalanxes, niche cultural symbols. The authors argue these failures reflect a world-knowledge bottleneck rather than a rendering bottleneck: generators are trained on fixed corpora with knowledge cutoffs, while user requests are unbounded, evolving, and deeply long-tailed.

The Twelve Failure Categories

Analysis of 20,840 production prompts revealed twelve recurring failure categories, split along a structural axis:

The Knowledge Boundary (Formal Definition)

The paper formalizes the central theoretical construct:

Definition 1 (Knowledge Boundary). Let $\mathcal{K}$ denote the space of world-knowledge units required by prompts in distribution $P$. For generator $G_\theta$, prompt $p$, and conditioning context $c$ (search-returned references), with bounded quality function $Q(G_\theta, p, c) \in [0,1]$ and tolerance $\epsilon > 0$:

$$\mathcal{K}_{\text{int}}(\theta) = \left\{k \in \mathcal{K}: \mathbb{E}_{p|k}\left[Q(G_\theta, p, \operatorname{SEARCH}(k)) - Q(G_\theta, p, \varnothing)\right] < \epsilon\right\}$$ $$\mathcal{K}_{\mathrm{ctx}}(\theta) = \mathcal{K} \setminus \mathcal{K}_{\text{int}}(\theta)$$

The pair $(\mathcal{K}_{\text{int}}(\theta), \mathcal{K}_{\text{ctx}}(\theta))$ forms a generator-specific partition; the boundary $\mathcal{B}(\theta)$ is generator-specific and shifts under training: $\mathcal{K}_{\text{int}}(\theta) \subseteq \mathcal{K}_{\text{int}}(\theta')$ when $\theta'$ results from DPO on search-augmented demonstrations.

Key Insight: Some knowledge is internalizable (search should not fire), other knowledge is contextual (search is structurally necessary). The boundary is discoverable through co-training, not knowable a priori.

---

Methodology

Dataset Construction (SEARCHGEN-20K)

• Seed database: 31,537 entities extracted from production requests, annotated with canonical names, training-set frequency estimates, ground-truth visual references, and distinguishing attributes. 93.1% of entities appear in only one prompt (extreme long-tail).
• Prompt synthesis: Human-brainstormed templates + LLM-assisted rewriting, using an answer-first strategy where a frontier model selects self-contained knowledge gaps.
• Statistics: 20,188 train rows, 751 test prompts; bilingual (58% English, 42% Chinese); mean 5.2 knowledge gaps per prompt (90.5% carry ≥3); 34,694 visual reference slots, 16,345 textual knowledge slots.

Evaluation Protocol (SEARCHGEN-BENCH)

Nine components scored 0–100, averaged to overall score:

Knowledge-sensitive (prompt-adaptive): Checklist Verification (3–10 binary checks), Rubric Scoring (3–5 weighted dimensions), Prompt Faithfulness, Visual Reference Fidelity, Textual Knowledge Fidelity.

Knowledge-invariant (rendering quality): Image Quality, Text Rendering, AI Naturalness, Composition & Aesthetics, Physical Plausibility.

The Noise-Resistant Agentic Reasoner: Gate–Filter–Integrate

Three-stage protocol suppressing noise at each entry point:

1. Gate: Identifies knowledge gaps, classifies by type and severity, proposes modality-labeled queries (image|web). Only critical/important gaps trigger search; ≤3 queries or SKIP.
2. Filter: Selects references that "fill the gap" while minimizing extraneous content, reducing copy effects.
3. Integrate: Routes visual references through natural language — the reasoner fuses original prompt + gap analysis + retrieved knowledge into enriched text specification with grounded citations (e.g., "following Image I, render the character in a teal-and-gold robe"), discarding pixel-level noise.

Co-Training: Teach, Then Search

Phase 0 — Supervised Warm-Start: SFT of Qwen3-VL-8B on ~10,000 expert-annotated Task A/B/C trajectories (corresponding to gate/filter/integrate).

Phase 1 — Online Iterative DPO (Generator): Samples $M$ images per prompt from search-augmented inputs, scores by evaluation protocol, constructs preference pairs from top/worst outputs. The DPO loss for flow-matching:

$$\mathcal{L}_{\mathrm{DPO}}(\theta) = -\mathbb{E}\left[\log \sigma\left(\beta\left(\log \frac{v_\theta(x_t^w | \tilde{p})}{v_{\theta_{\mathrm{ref}}}(x_t^w | \tilde{p})} - \log \frac{v_\theta(x_t^l | \tilde{p})}{v_{\theta_{\mathrm{ref}}}(x_t^l | \tilde{p})}\right)\right)\right]$$

where $v_\theta$ is the flow-matching velocity field, $\beta = 100$ is DPO temperature, $\theta_{ref}$ updated via EMA (decay 0.99). This builds noise-robustness: the generator learns to use imperfect references without being dominated by their noise.

Phase 2 — Rejection-Sampling Finetuning (Reasoner): Rolls out $N$ trajectories from Phase 0 reasoner paired with the DPO-strengthened generator, computes group-relative advantage:

$$A_n = \frac{s_n - \bar{s}}{\sigma_s + \delta}$$

Retains only positive-advantage trajectories for SFT. The reasoner learns the strengthened generator's boundary without explicit labels.

---

Empirical Validation / Results

The World-Knowledge Bottleneck (Finding 1)

Open generators score 67–75 on parametric-knowledge prompts but collapse to 22–28 on search-intensive prompts — a 40-point divergence invisible to existing benchmarks. Commercial systems with integrated search barely drop (GPT-Image-2: -0.1; Nano Banana Pro: -9.7).

Table 4 (excerpt): Knowledge bottleneck on Search-Intensive subset (651 prompts):

Knowledge-sensitive components collapse while rendering components hold steady, confirming the failure is knowledge absence, not rendering inability.

Naive Search Degrades Performance (Finding 2)

On the NoSearch stratum, BlindSearch strictly degrades every generator (Qwen-Image-2: 70.7 → 60.4, a 14.6% relative loss). Two failure modes:

• Concept corruption: Search fires on a prompt the generator handles; reference overrides accurate internal knowledge (gating failure).
• Copy effect: Reference carries too much raw information, becoming a copying template (filtering failure).

Co-Training Produces Monotonic, Selective Improvement (Finding 3)

Table 6 (excerpt): Co-training progression for Klein-4B:

Three predicted patterns confirmed:

1. Monotonicity: Phase 2 (31.8) exceeds frontier oracle (31.2) on same generator; each phase contributes independently (+2.8 from DPO, +2.6 from RFT).
2. Selectivity: Phase 2 scores 56.9 on NoSearch vs. 49.9 for no-search DPO (+7.0), proving the reasoner learned when to abstain.
3. Generator-specificity: Calibrated reasoner paired with base generator scores 26.8 (vs. 31.8 with its intended DPO generator), confirming the boundary is a joint property of the generator–reasoner pair.

Knowledge Boundary Shift Evidence

The CDF of per-prompt no-search quality shifts rightward after DPO (Figure 9b): fewer prompts score low, more score high from parametric knowledge alone. The shaded region represents newly internalized knowledge migrating from $\mathcal{K}_{\text{ctx}}$ to $\mathcal{K}_{\text{int}}$, consistent across both Klein-4B and Bagel-7B architectures.

---

Theoretical and Practical Implications

Theoretical Significance

1. The knowledge boundary is an operational design variable, not merely a conceptual framework. The formal definition (Def. 1) provides a principled basis for deciding when to search — a first-order design variable previous systems treat as always-on.

2. Co-training as boundary discovery: The boundary need not be specified a priori; it emerges from the interaction of generator DPO (pushing the boundary outward) and reasoner RFT (pulling the search policy inward). This mirrors self-rewarding and self-play paradigms but with structurally distinct roles for each player.

3. Noise-resistance as learned property: Generators trained only on clean inputs treat all conditioning as authoritative. Exposure to noisy search outputs during DPO builds robustness — a property essential for any tool-augmented system.

Practical Implications

1. Matched-compute efficiency: A co-trained 8B reasoner + 4B generator (31.8) slightly exceeds a frontier VLM oracle on the same generator (31.2), suggesting generator-specific calibration can approach costly frontier-scale reasoning at a fraction of compute, latency, and cost.

2. Replayable research infrastructure: The released assets (20,839 prompts, 90,452 reasoning traces, 281,925 images, 145,642 search sessions) enable fully offline reproducible research without live search API keys — democratizing search-augmented generation research.

3. Generalization beyond search: The gate–filter–integrate protocol and teach-then-search co-training extend to other tools (image editing, render-as-code, 3D retrieval, structural control), each supplying a different slice of $\mathcal{K}_{\text{ctx}}$.

---

Conclusion

The central question is not how to build a model that knows everything, but how to build a system that knows what it does not know. The paper demonstrates that this self-awareness emerges from co-training dynamics, not from scale alone. The deliberately minimal recipe (one DPO pass, one RFT pass, 4B generator, 8B reasoner) validates the principle at the smallest useful scale.

Key takeaways:

• The knowledge boundary may shift outward with scale but cannot disappear: training data is finite while the world is unbounded. No model can internalize post-cutoff events, ultra-rare entities, or evolving cultural knowledge.
• The monotonic progression (NO SEARCH → BLIND SEARCH → GENERATOR-ADAPTIVE SEARCH), per-stratum recovery, and generator-specific behavior provide converging evidence that co-training discovers the boundary.
• The 39-point gap to GPT-Image-2 reflects generator capacity at the 4B scale, not a framework limitation; scaling the generator along this axis is the clear path to closing the absolute gap.

Future directions:

• Scaling laws of the knowledge boundary: does the internalizable set expand uniformly across failure categories with generator size?
• Can the boundary be predicted from model internals (probing classifiers, uncertainty estimates) without full co-training?
• Does the internalizable/contextual split generalize to video, 3D, and music generation?
• Can a single agent learn a unified boundary across a toolbox of tools beyond search?