Summary (Overview)

• Problem Identification: The paper identifies the Context Gap — the mismatch between the user-provided context and the sufficient generation context required for text-to-image (T2I) models — as a fundamental challenge in real-world image generation where requests are often underspecified, implicit, or dependent on up-to-date knowledge.
• Proposed Framework: Qwen-Image-Agent, a unified agentic framework that integrates plan, reason, search, memory, and feedback in a context-centric manner, progressively constructing the full generation context through Context-Aware Planning and Context Grounding.
• New Benchmark: IA-Bench (Image Agent Bench), a benchmark covering four core image agent capabilities (Plan, Reason, Search, Memory) with 4 tasks, 17 subtasks, 730 test instances, and 1801 fine-grained binary checklist items.
• State-of-the-Art Results: Qwen-Image-Agent outperforms strong baselines on IA-Bench (IA-score of 45.4), WISE-Verified (0.9020 overall), and MindBench (0.42 overall), achieving state-of-the-art performance.
• Key Insight: The framework is training-free, compatible with existing image generators, and supports both multi-image and multi-turn interaction.

---

Introduction and Theoretical Foundation

Background and Motivation

Text-to-image (T2I) models have achieved remarkable progress in generating high-quality images from natural language prompts. However, as these systems move into real-world applications such as marketing, product design, and slide creation, they face a structural mismatch between training and deployment:

• Models are optimized for fully specified prompts, while real-world requests are often underspecified.
• Successful generation may require inferring implicit user intent, retrieving up-to-date knowledge or visual references from the web, and incorporating interaction history.

The Context Gap

The paper formally defines the Context Gap as the discrepancy between the provided user context and the generation context required for T2I models. This gap motivates a paradigm shift from traditional direct image generation to agentic image generation, where the system must:

1. Identify missing context
2. Acquire it
3. Use it effectively during generation

Mathematical Formulation

The paper formalizes image generation as a conditional rendering problem. Given a user context $c_u = (P, I_{ref})$ with prompt $P$ and optional reference images $I_{ref}$, direct image generation renders output image $y$ in a single forward pass:

$$y \sim p_{gen}(\cdot | c_u) \tag{1}$$

In real-world scenarios, the user context $c_u$ is often incomplete for the desired visual task. The paper distinguishes user context from generation context $c_g$, which denotes the complete context needed for successful rendering.

Agentic image generation addresses this gap by treating $p_{gen}$ as a renderer and introducing a context-construction process. At each step $t$, the agent maintains a state $s_t$, takes an action $a_t$, and receives an observation $o_t$, forming a trajectory:

$$\tau = \{(s_t, a_t, o_t)\}_{t=1}^{T} \tag{2}$$

The action space consists of basic operations to gather context: plan, reason, search, rewrite, and evaluate. The state is defined as $s_t = (c_t, O_{t-1})$ where $c_t$ is the current context under construction, and $O_{t-1} = \{o_1, \ldots, o_{t-1}\}$ is the set of accumulated intermediate results. Let $c(\tau)$ denote the final generation context induced by trajectory $\tau$. The agentic generation process is then:

$$p_{agent}(y | c_u) = \sum_{\tau} p(\tau | c_u) p_{gen}(y | c_g = c(\tau)) \tag{3}$$

---

Methodology

Qwen-Image-Agent Framework

The framework consists of two main modules: Context-Aware Planning and Context Grounding.

1. Context-Aware Planning

Operates at three levels:

Information-level Planning: Identifies the context gap and plans how to resolve it. The system raises explicit questions to characterize missing information, then routes each question to a suitable context grounding strategy (reasoning, web search, or image search).

Content-level Planning: Builds the generation context and plans image content. The system assembles context obtained during grounding and rewrites the user prompt into a detailed prompt specifying key generation elements: subject, attributes, layout, style, and textual elements.

Generation-level Planning: Allocates generation context in multi-image and multi-turn scenarios. For multi-turn settings, relevant information is selected from previous turns to avoid content drift. For multi-image settings, the generation context is distributed across individual images while accounting for dependencies (parallel, sequential, hybrid).

2. Context Grounding

A unified module that collects context through four channels:

Grounding via Reason: Makes implicit intents and requirements explicit using three forms of reasoning:

• Commonsense reasoning
• Logical reasoning
• Visual reasoning

For each question identified during Information-level Planning, a VLM infers the corresponding answer.

Grounding via Search: Handles requests dependent on up-to-date factual information or IP-related visual references:

• For factual knowledge: extract search keywords → web search → summarize retrieved results
• For visual references: retrieve candidate images → VLM ranking → retain most relevant

Grounding via Memory: Incorporates conversation history, extracts and updates user profiles, and supports external memory sources through a multimodal retriever.

Grounding via Feedback: After generation, plans a checklist of expected image attributes, uses a VLM to assess results against the checklist, and converts failures into feedback context for iterative refinement.

IA-Bench Benchmark

IA-Bench evaluates four core capabilities over 17 subtasks:

Evaluation Metrics

The benchmark uses a checklist-based evaluation protocol. For each test instance $i$, let $I^{gen}_i$ denote the generated image and $C_i = \{c_j^i\}_{j=1}^{K_i}$ its checklist. Two complementary metrics are reported:

Pass Rate (PR) — strict task success where all checklist items must be satisfied:

$$PR = \frac{1}{N} \sum_{i=1}^{N} \prod_{j=1}^{K_i} \text{VLM}(I^{gen}_i, c_j^i)$$

Checklist Accuracy (CA) — average proportion of checklist items satisfied:

$$CA = \frac{1}{N} \sum_{i=1}^{N} \frac{1}{K_i} \sum_{j=1}^{K_i} \text{VLM}(I^{gen}_i, c_j^i)$$

IA-score — weighted aggregate over the four dimensions:

$$\text{IA-score} = 0.3 \times \text{Plan} + 0.3 \times \text{Reason} + 0.3 \times \text{Search} + 0.1 \times \text{Memory}$$

---

Empirical Validation / Results

Implementation Details

• Image generation backbone: Qwen-Image-2.0
• MLLM backbone: GPT-5.5-0424
• Search tools: Google Search API (text limit: 5, image limit: 5), Jina API for web page processing
• Feedback loop: Up to 3 attempts on IA-Bench; disabled on WISE-Verified and MindBench

Results on IA-Bench

Key findings:

• Qwen-Image-Agent achieves the highest IA-score (45.4), improving over Qwen-Image-2.0 by 28 points (from 17.4).
• Particularly large improvement in the Memory dimension (49.0 Pass Rate vs. 11.0 for Qwen-Image-2.0).
• Agentic models consistently outperform direct generation models on Plan, Reason, and Search.

Results on WISE-Verified

Qwen-Image-Agent achieves state-of-the-art performance (0.9020 overall), surpassing previous SOTA Nano Banana Pro (0.8760):

Results on MindBench

Qwen-Image-Agent sets a new state of the art (0.42 overall), improving over Qwen-Image-2.0 by 82.6%:

Ablation Study

Key ablation findings:

• Removing any grounded context leads to clear drops in its corresponding dimension.
• Removing Reason Context degrades both Reason and Plan (implicit requirements resolved during reasoning feed into planning).
• Removing Feedback Context causes a relatively smaller drop due to Qwen-Image-2.0's strong rendering accuracy.
• Both MLLM backbone and Generation backbone are critical to overall system performance.

---

Theoretical and Practical Implications

Framework Contributions

1. Unified Context-Centric Perspective: Qwen-Image-Agent provides a unified lens for understanding why current T2I systems fail in practical settings, treating the context gap as the central challenge.

2. Training-Free Compatibility: The framework is training-free and compatible with existing image generators, making it practically deployable without retraining.

3. Modular Grounding: The four grounding channels (reason, search, memory, feedback) are complementary and can be individually ablated, verified through systematic experiments.

Identified Challenges and Failure Modes

The paper identifies several important challenges:

• Unidentified Context Gaps: Some gaps remain too implicit to be reliably identified (e.g., inferring a historical event from a date and location). These failures depend on MLLM intelligence, not generation backbone strength.

• Ambiguous Boundary between Reason and Search: The paper proposes a principled distinction: commonsense facts are solvable by internal reasoning, while Precise Facts (exact numbers, dates, names) and Dynamic Facts (change over time) require explicit search.

• Excessive Image Search: Overuse of image retrieval introduces distracting visual references and degrades output quality, especially when multi-reference editing is brittle.

• Context Explosion in Multiturn Generation: Accumulated multimodal context (user references, previously generated images, retrieved images) can exceed token limits. Relevance-based context selection mitigates this.

• Weak Feedback Supervision: Current prompt-based feedback is limited; future work should extend feedback to supervise context-gap identification and grounding earlier in the pipeline.

• High Latency and Cost: The full agentic pipeline is substantially more expensive than one-shot generation. DAG-based execution enables parallelism, but more efficient pipelines are needed.

---

Conclusion

This work identifies the Context Gap as a central challenge in real-world image generation and proposes Qwen-Image-Agent, a unified agentic framework that integrates plan, reason, search, memory, and feedback in a context-centric manner. The framework progressively constructs the generation context through Context-Aware Planning (operating at information, content, and generation levels) and Context Grounding (gathering context from four complementary sources).

The paper also introduces IA-Bench, a benchmark for systematically evaluating four core capabilities of agentic image generation: Plan, Reason, Search, and Memory, with 730 instances and 1801 checklist items.

Experiments demonstrate state-of-the-art performance across IA-Bench, WISE-Verified, and MindBench. The work highlights a paradigm shift from direct image generation to agentic image generation, providing a unified context-centric perspective for understanding this transition.

Future directions include:

1. Extending feedback beyond post-hoc critique to supervise earlier pipeline stages
2. Incorporating task-specific supervision (downstream metrics, reward models) for stronger test-time scaling
3. Developing more efficient agentic pipelines through training-based optimization or better tool-use policies
4. Improving MLLM intelligence for better context-gap identification and search-reason boundaries