Summary (Overview)

• AutoDesign is a meta-harness optimization framework that recursively improves a design harness (the system surrounding a fixed model) rather than individual artifacts, enabling long-horizon, human-aligned multimodal design (e.g., paper-to-poster generation).
• The framework operates via two nested loops: an inner loop (design harness) that iteratively generates and revises artifacts under critic feedback, and an outer loop (meta-harness) that aggregates rollouts, evaluates outputs, proposes bounded updates to one of five harness components, and gates acceptance using a train/dev split to prevent overfitting.
• The optimized system, DesignHarness, achieves the highest score (78.32) on the new PosterBench benchmark (100 papers across five disciplines), surpassing the closed-source commercial system Claude Design by 7.45 points under matched configurations.
• Across seven controlled code-agent–model configurations, attaching DesignHarness consistently improves performance, raising the average PosterBench Score from 54.99 to 67.39 (+12.4%).
• A system-blind human study (936 responses from 11 reviewers) confirms AutoDesign's superiority, with a Bradley–Terry preference estimate of 64.0% against a randomly sampled alternative, and demonstrates that PosterBench scores align with human preference, especially for large score gaps (74.4% agreement for gaps ≥20 points).

Introduction and Theoretical Foundation

Human communication often requires transforming multimodal sources into human-facing artifacts (webpages, slides, posters, videos). This is naturally a long-horizon agentic task requiring evidence extraction, reasoning over heterogeneous information, planning, and iterative refinement based on feedback.

Key limitation of existing systems: Current multimodal design systems treat human-aligned feedback (critiques, revisions) as transient signals rather than reusable design knowledge. Unlike human creators who accumulate experience from successes and failures, these systems do not convert evidence, structural constraints, and feedback into persistent design-aligned capabilities.

AutoDesign's core insight: Frame human-aligned design generation as a meta-harness optimization problem — an agentic system that recursively optimizes the design harness itself (the scaffolding around a fixed model) based on evaluation grounded in human preferences (annotated reference artifacts, natural language guidance).

Formal definition: A design harness $H$ transforms a multimodal source $x$ into an artifact $y$ given context $c$:

$$y \sim H(\pi_{\theta}, x, c) \tag{1}$$

The harness is decomposed into five components: (1) Context and Memory, (2) Tools and Specifications, (3) Execution Runtime, (4) Orchestration, and (5) Evaluation and Feedback.

The meta-harness optimizes expected quality:

$$J(H) = \mathbb{E}_{(x,c) \sim p_{\text{task}}, y \sim H(\pi_{\theta}, x, c)}[R_{\text{meta}}(y, x, c)] \tag{2}$$ $$H^{*} = \arg\max_{H} J(H) \tag{3}$$

The model parameters $\theta$ remain fixed throughout; optimization acts on the system surrounding the model.

Methodology

Inner Loop (Design Harness)

The inner loop consists of a designer $M_{\text{design}}$ and a critic $M_{\text{critic}}$:

$$\begin{array}{l} y_k = M_{\text{design}}(y_{k-1}, f_{k-1}; x, c), \\ f_k = M_{\text{critic}}(y_k; x, c), \end{array} \tag{4}$$

where $y_k$ is the artifact at step $k$ and $f_k$ is the feedback. The designer generates and revises; the critic evaluates and provides localized feedback.

Outer Loop (Meta-Harness)

Each outer-loop iteration has four stages:

1. Rollout: Execute current harness $H_t$ on training set $\mathcal{D}_{\text{train}}$, collecting trajectories $\tau_t$ and scores $s_t$.
2. Evaluation: A fixed evaluator $R_{\text{meta}}$ (constructed from human-annotated reference artifacts across seven quality dimensions) scores each artifact.
3. Update Proposal: A coding agent $P$ (acting as planner and code editor) analyzes trajectories, scores, and the optimization record $\mathcal{L}$ to propose a candidate harness:

$$H'_{t+1} = P(H_t, \tau_t, s_t, \mathcal{L}) \tag{5}$$

Each iteration modifies exactly one of the five harness components to maintain interpretable credit assignment.

4. Acceptance Gate: A candidate is accepted only if it improves on training and does not degrade on the development set:

$$\text{Accept}(H'_{t+1}) \iff J_{\text{train}}(H'_{t+1}) > J_{\text{train}}(H_t) \land J_{\text{dev}}(H'_{t+1}) \geq J_{\text{dev}}(H_t) \tag{6}$$

The dev set is never exposed to the optimizer, serving as a guard against overfitting.

Human-in-the-Loop: Optional natural language guidance $g_t$ can redirect the search when autonomous optimization stagnates, or revise the evaluator when systematic bias is detected.

DesignHarness Architecture

The optimized system has four stages:

1. Paper Ingestion: Extracts metadata, outlines, key passages, and figures with provenance tracking.
2. Artifact Generation/Revision: A coding agent generates/revises editable HTML with localized edits.
3. Validation: Rule-based blocking checks (missing assets, broken provenance, overflow) plus a VLM-based visual critic.
4. Finalization: Post-processing (rendering, math typesetting, asset inlining) with fallback mechanisms if the budget ($K=12$ attempts) is exhausted.

Empirical Validation / Results

PosterBench Benchmark

PosterBench comprises a 100-paper Main Track and a 10-paper mini subset across five disciplines (AI/ML, biomedicine, climate, economics, physics). The seven-dimensional rubric score:

$$R_{\text{rubric}}(p_i, A_i) = \sum_{j=1}^{7} \alpha_j q_{i,j}/10, \quad \boldsymbol{\alpha} = (10, 10, 15, 10, 20, 25, 10) \tag{7}$$

with final score capped by record-level ceilings:

$$R_{\text{poster}}(p_i, A_i) = \min\left(R_{\text{rubric}}(p_i, A_i), C_i^{\text{layout}}, C_i^{\text{viability}}, C_i^{\text{failure}}, C_i^{\text{gate}}\right) \tag{8}$$

Main Results (PosterBench Main Track)

Ablation: Effect of DesignHarness Attachment

Cost–Performance Trade-off

On PosterBench-mini, the Pareto frontier spans from LongCat-2.0 (55.13 at **$0.27/poster**) through Doubao Seed 2.1 Pro (71.83 at$2.75) and Claude 4.8 (74.56 at $7.63) to GPT-5.5 (81.46 at$10.02). Doubao achieves 88% of GPT-5.5's score at 27% of its cost.

Human Evaluation

Using a Bradley–Terry model:

$$\Pr(i \succ j) = \frac{\exp(\beta_i)}{\exp(\beta_i) + \exp(\beta_j)}$$

AutoDesign achieves the highest preference estimate at 64.0% (95% interval: 55.2–77.8%). Tie-adjusted preferences: 61.3% vs. Claude Code, 63.1% vs. OpenDesign, 67.6% vs. Claude Design.

Benchmark–human alignment: PosterBench Score correlates with human preference ($r = 0.34$), and agreement rises from 51.9% (0–3 point gaps) to 74.4% (≥20 point gaps), validating the benchmark's discriminative power.

Theoretical and Practical Implications

• Meta-harness optimization as a paradigm: AutoDesign demonstrates that persistent, recursive improvement of the system surrounding a model (rather than the model itself) is both feasible and effective for complex multimodal generation tasks. This aligns with recent taxonomies distinguishing model improvement from scaffold improvement.
• Credit assignment via component isolation: Restricting each update to one of five harness components makes gains/losses attributable to single coherent interventions, enabling interpretable, reproducible optimization trajectories.
• Practical accessibility: DesignHarness makes high-quality paper-to-poster generation affordable (under $3 per poster in fully autonomous mode, ~40 minutes, 253 tool calls, 11 editing turns), democratizing conference-quality poster production.
• Benchmark design: PosterBench's hybrid evaluation (rule-based + VLM) with record-level ceilings provides a robust, human-aligned protocol that could generalize to other design tasks (slides, webpages, videos).
• Generalization potential: The framework extends beyond posters to slides, webpages, and videos (pilot results shown), suggesting a path toward general multimodal-in/out agentic design systems.

Conclusion

AutoDesign transforms recurring design failures into persistent improvements to the system that generates future outputs. Through 7 days of evolving traces, it invokes 224 subagents, records at least 123 recursive iterations, and accumulates 54 harness updates — converting human-designed reference artifacts, rollout traces, rendering diagnostics, and evaluator feedback into durable design priors.

Key takeaways:

1. Meta-harness optimization with acceptance gating effectively converts transient feedback into reusable design knowledge.
2. DesignHarness achieves state-of-the-art paper-to-poster generation (78.32 on PosterBench) with broad model/coding-agent compatibility (+5.0 to +19.6 points across configurations).
3. PosterBench provides a comprehensive, human-aligned evaluation protocol for academic poster generation.

Future directions include: (a) extending to additional media with medium-specific evaluators and validation gates, (b) better component selection via failure attribution and uncertainty estimation, (c) versioned adaptive evaluators anchored by frozen reference tasks, and (d) model–harness co-evolution where long-horizon trajectories provide execution-time supervision for model post-training.