Summary (Overview)

• Knowledge bottleneck identified: Frontier open visual generators score only 21–28 out of 100 on the newly constructed SearchGen-Bench, a 40-point collapse on search-intensive prompts that remains invisible to existing benchmarks. This failure is due to world-knowledge absence, not rendering inability.
• SearchGen-20K dataset: A large-scale, bilingual (English/Chinese) dataset of 20,839 world-knowledge-grounded prompts spanning twelve failure categories and twenty-two domains, each annotated with structured knowledge gaps and evaluation checklists. A pre-executed multimodal corpus of 145,642 search sessions (SearchGen-Corpus-1M) enables fully offline, reproducible research.
• Naive search fails: Blind search degrades quality on prompts the generator already handles, introducing concept corruption and copy effects. The root cause is a generator-specific, evolving knowledge boundary between what a generator can internalize (knowledge that can be learned) and what must remain in external context.
• Co-training framework: A two-phase teach-then-search approach: (1) online iterative DPO teaches the generator to internalize stable knowledge and build noise-robustness; (2) rejection-sampling finetuning recalibrates the reasoner to search only what the strengthened generator cannot render. This produces monotonic gains from NoSearch through BlindSearch to Generator-Adaptive Search.
• Minimal recipe validates principle: An 8B agentic reasoner co-trained with a 4B generator (Flux.2-Klein-4B) achieves 31.8/100 on SearchGen-Bench, slightly exceeding a frontier VLM oracle (31.2) on the same generator, demonstrating that generator-specific calibration can approach frontier-scale reasoning at a fraction of the compute.

Introduction and Theoretical Foundation

Modern visual generators excel at rendering complex scenes with precise lighting and coherent structure, saturating standard benchmarks. Yet they fail dramatically on a substantial class of real-world user requests: new characters, post-cutoff events, cultural symbols, niche typography, and historical artifacts. These failures reflect a world-knowledge bottleneck rather than a visual-synthesis bottleneck. Generators are trained on fixed corpora with inherent knowledge cutoffs, while user requests are unbounded, evolving, and deeply long-tailed.

The paper formalizes the knowledge boundary:

Definition 1 (Knowledge Boundary). Let $K$ denote the space of world-knowledge units required by prompts in a distribution $P$. For a generator $G_\theta$ with parameters $\theta$, a prompt $p \in P$, and conditioning context $c$ (search-returned references and text), let $Q(G_\theta, p, c) \in [0, 1]$ be a bounded quality function. For a fixed tolerance $\epsilon > 0$, define the internalizable and contextual knowledge sets:

$$K_{\text{int}}(\theta) = \left\{ k \in K : \mathbb{E}_{p|k} \left[ Q(G_\theta, p, \text{SEARCH}(k)) - Q(G_\theta, p, \emptyset) \right] < \epsilon \right\}, \tag{1}$$ $$K_{\text{ctx}}(\theta) = K \setminus K_{\text{int}}(\theta). \tag{2}$$

The pair $(K_{\text{int}}(\theta), K_{\text{ctx}}(\theta))$ forms a generator-specific partition of $K$; we refer to this partition as the knowledge boundary $B(\theta)$. The boundary is generator-specific and shifts under training.

The key insight: some knowledge is internalizable and search should not fire for it; other knowledge is contextual and search is structurally necessary. The boundary is discoverable through co-training, not specifiable a priori.

Methodology

Dataset Construction (SearchGen-20K and SearchGen-Bench)

From 20,840 real production prompts, the authors identify twelve failure categories (Table 1) across twenty-two domains. The dataset is bilingual (58% English, 42% Chinese) with an extreme long-tail entity distribution: 93.1% of 31,537 unique entities appear in only one prompt. Each prompt is annotated with structured knowledge gaps (visual reference slots, textual knowledge slots, and failure mode labels) and evaluation checklists for automated assessment.

Table 1: Taxonomy of search-intensive visual generation

Table 2: Released assets

Three-Stage Agentic Reasoner (Gate–Filter–Integrate)

The reasoner (Qwen3-VL-8B) controls when and how search augments the generator:

1. Gate: Identifies knowledge gaps, classifies by type and severity, proposes modality-labeled search queries (image or web). Only critical or important gaps trigger search; otherwise SKIP.
2. Filter: Selects references that fill the specific gap while minimizing extraneous content, reducing copy effects.
3. Integrate: Routes visual references through natural language rather than raw pixel conditioning, producing grounded citations (e.g., "following Image 1, render the character in a teal-and-gold robe") to preserve missing knowledge while discarding pixel-level noise.

Co-Training Framework (Teach, Then Search)

Algorithm 1: Co-Training: Teach, Then Search

• Phase 0: Supervised Warm-Start. Fine-tune the reasoner (Qwen3-VL-8B) on ~10,000 expert-annotated gate/filter/integrate trajectories.

• Phase 1: Online Iterative DPO (Generator). For each episode, sample $M$ images per prompt using the search-augmented reasoner, score them, construct preference pairs from top/worst-scored generations, and update the generator via DPO loss adapted for flow-matching:

  $$\mathcal{L}_{\text{DPO}}(\theta) = -\mathbb{E}\left[ \log \sigma\left( \beta \left( \log \frac{v_\theta(x_w^t \mid \tilde{p})}{v_{\theta_{\text{ref}}}(x_w^t \mid \tilde{p})} - \log \frac{v_\theta(x_l^t \mid \tilde{p})}{v_{\theta_{\text{ref}}}(x_l^t \mid \tilde{p})} \right) \right) \right]$$

  where $v_\theta$ is the flow-matching velocity field, $t$ is the diffusion timestep, $\beta = 100$ is the DPO temperature, and $\theta_{\text{ref}}$ is updated via EMA (decay 0.99). This pushes the generator's knowledge boundary outward and builds noise-robustness to imperfect references.

• Phase 2: Rejection Finetuning (Reasoner). Roll out $N$ trajectories from the Phase 0 reasoner paired with the DPO-strengthened generator, compute group-relative advantages, and retain only positive-advantage trajectories for continued reasoner training. The reasoner learns to search only what the strengthened generator cannot render.

Empirical Validation / Results

Evaluation Protocol

The SearchGen-Bench judge uses nine components (Table 3), scored 0–100. Knowledge-sensitive components (Checklist, Rubric, Visual Reference Fidelity, Prompt Faithfulness, Textual Knowledge Fidelity) measure knowledge presence; knowledge-invariant components (Image Quality, Text Rendering, AI Naturalness, Composition & Aesthetics, Physical Plausibility) measure rendering competence.

Table 3: Evaluation components of the SearchGen-Bench judge

Key Findings

Finding 1: The world-knowledge bottleneck. On the NoSearch stratum (100 prompts), open and commercial generators score comparably (67–75). On the Search-Intensive stratum (651 prompts), open generators collapse to 22–28, while commercial systems with integrated search barely drop. Table 4 pinpoints the gap: knowledge-sensitive components collapse while rendering components hold steady.

Table 4: Knowledge bottleneck on SearchGen-Bench (Search-Intensive subset, 651 prompts)

Finding 2: Naive search is harmful. BlindSearch degrades every generator on the NoSearch stratum (e.g., Qwen-Image-2 drops from 70.7 to 60.4). ReasonedSearch improves selectively but requires generator-specific calibration.

Table 5: Search is a double-edged sword

Finding 3: Co-training discovers the knowledge boundary. Table 6 shows monotonic improvement across all three phases and all difficulty tiers. The calibrated 8B reasoner matches the frontier oracle on the same generator.

Table 6: Main results – co-training progression

Monotonicity: Each phase improves within every difficulty tier. Selectivity: Phase 2 improves NoSearch from 49.9 (no-search baseline) to 56.9, showing the reasoner learns when to abstain. Generator-specificity: The cross-check row (RFT-8B + base Klein-4B) scores only 26.8 vs. 31.8 when paired with the strengthened generator, confirming the boundary is a joint property of the generator–reasoner pair.

Figure 9b shows the cumulative distribution of per-prompt no-search quality shifting rightward after DPO, indicating newly internalized knowledge.

Theoretical and Practical Implications

• Shift from "knowing everything" to "knowing what it doesn't know": The knowledge boundary formalizes a structural partition that is generator-specific and evolving. Co-training provides a mechanism to discover this boundary without explicit labels.
• Noise-resistant agentic search: The gate–filter–integrate protocol addresses modality-specific failure modes (concept corruption, copy effects, spatial distortion, identity blending, stylistic leakage) that have no text-domain analogue.
• Minimal recipe validates scalability: A single iteration of co-training (one DPO pass, one RFT pass, 4B generator, 8B reasoner, ~256 GPU-hours) produces monotonic gains and matches frontier oracles. This suggests the principle can scale with larger models and more iterations.
• Evaluation gap: Existing benchmarks test only rendering within known concepts. SearchGen-Bench reveals a 40-point collapse invisible to standard evaluations, motivating a new class of knowledge-intensive benchmarks.
• Practical tool design: The co-training framework extends beyond search to any tool—image editing, render-as-code (SVG, matplotlib, CAD), 3D-asset retrieval, structural control—each supplying a different slice of the contextual knowledge set.

Conclusion

The central question is not how to build a model that knows everything, but how to build a system that knows what it does not know. This split creates a generator-specific knowledge boundary, and the paper shows that this boundary, though hard to specify a priori, is discoverable through co-training.

The resolution lies in co-training the generator and reasoner around their shared boundary. Online DPO serves a dual function: it pushes the generator's boundary outward by internalizing stable visual knowledge, and it teaches the generator to use