Aspire: Can Models Self-Evolve from Vague Goals?

Summary (Overview)

• Core Contribution: This paper introduces Aspire, a benchmark for studying vague-goal-driven self-evolution in LLMs—where agents receive only a broad natural-language capability goal (e.g., "become a better physicist") with no predefined tasks, datasets, or evaluation metrics, forcing them to operationalize the goal themselves.

• Key Finding: Vague goals redirect search effort toward goal interpretation and operationalization, but current agents routinely complete training and harness-editing loops while weight-level gains remain sparse, unstable, and rarely transfer to hidden evaluation sets.

• Benchmark Design: Aspire features a hidden, expert-authored evaluation set of 520 items spanning six goals, with sealed evaluation items that agents never observe; it supports both model-weight evolution and agent-harness evolution in a unified interactive environment.

• Main Results: Across three research questions, agents produce evaluated checkpoints far more often than retained improvements; only 1 of 12 final-only Avg@2 outcomes exceeds its base score, and the strongest evolved harness remains below the engineered Qwen-Agent reference (28.64 task macro vs. 27.22 best successor)..

• Failure Modes Identified: Agents often train on mismatched data (e.g., numeric-label SFT causing answer-format collapse), trust narrow self-evaluations, and continued search can erase earlier improvements, revealing a gap between closing the training loop andclosing the capability loop.

Introduction and Theoretical Foundation

Background and Motivation

Many important forms of human learning begin with a broad capability direction rather than a predefined benchmark, training set, or fully computable reward function. A student seeking to become a better physicist, for example, must:

1. Identify gaps in their knowledge
2. Prioritize which capabilities to develop
3. Determine whether learning has produced genuine progress

This involves three coupled decisions: what to improve, how to improve it, and how to verify the improvement.

The Gap in Existing Work

Existing work on LLM self-evolution focuses primarily on the second decision—how to improve. Given a concrete task, evaluation script, and success metric, LLM agents can already:

• Collect data
• Run training
• Revise post-training strategies from feedback

Systems like PostTrainBench, LaMDAgent, Evo-Memory, and SEAL show that modern LLMs can autonomously search for effective optimization paths toward a specified objective. However, these systems begin after humans have operationalized a broad capability request into a fixed task-level objective (e.g., "improve mathematical reasoning" becomes "improve performance on AIME").

The Missing Axis: Target Operationalization

The paper formalizes the missing axis asvague-goal-driven self-evolution. Key distinctions:

• "Vague" does not mean ambiguous or poorly specified—it denotes a broad capability direction that has not yet been operationalized as a fixed task-level objective and evaluation metric. . . . . . . . .
• The agent must diagnose capability gaps, form intermediate objectives, construct learning and validation signals, and jointly search over what to optimize andhow to optimize it.
• A deeper failure mode emerges: gains on an agent-constructed proxy may not translate into genuine improvements in the intended capability.

Connection to Real-World Practice

The paper draws an analogy to the forward-deployed engineer (FDE) role, popularized by Palantir and adopted by frontier-model companies. An FDE:

• Is embedded in the deployment environment
• Turns a general-purpose model into a system that works with a customer's data formats, workflows, and operational constraints
• Faces goals that are vague, feedback signals that may be absent or unreliable, and execution systems that may not exist in usable form

Aspire focuses on the first layer: when a deployment need specifies only a broad capability direction, can a model determine what it should learn, translate that direction into data, training plans, and validation signals, and achieve real capability growth?

Methodology

Task Definition and Evolution Surfaces

A campaign fixes the controller-side experiment contract:

$$\Gamma = (G, \mathcal{E}_{G}, J, \mathcal{A}, B, \Sigma), \qquad Y_{r}^{\mathrm{in}} = (M_{r}, H_{r}, D_{r}),$$

Where:

• $G \in \mathcal{G}$: a vague goal (natural-language capability objective)
• $\mathcal{E}_{G}$: the versioned evaluator bound to $G$
• $J$:the judge when model-based scoring is required
• $\mathcal{A}$:the typed action contract
• $B$:the campaign budget
• $\Sigma$:the predeclared terminal selection rule
• $M_{r}$: model weights
• $H_{r}$: agent harness (runtime instructions, tool policy, workflow, memory, validation logic)
• $D_{r}$: decision model that directs the search

Two evolution surfaces are defined:

$$\mathcal{S}_{M}(H_{0}, D_{r}) = \{(M, H_{0}, D_{r}): M \in \mathcal{M}\}, \quad \mathcal{S}_{H}(M_{0}, D_{r}) = \{(M_{0}, H, D_{r}): H \in \mathcal{H}\}.$$

Hidden Evaluation Set

The benchmark contains 520 expert-authored evaluation items covering six vague goals:

Construction and quality control:

• Domain experts author every candidate item from scratch
• GPQA, MMLU-Pro, and MedQA serve only as references for task format, domain coverage, and approximate difficulty—no item is copied
• Independent review removes incorrect, incomplete, underspecified, or unstably gradable items
• Surviving candidates pass blind multi-model difficulty calibration, exact and semantic deduplication, overlap auditing, scorer binding, and local end-to-end validation

Information boundary: The agent cannot access hidden evaluation items, reference answers, routing labels, rubrics, candidate outputs, or judge traces. Evaluation returns only aggregate scores permitted by the protocol.

Interactive Environment

Aspire exposes a single agent tool with typed, composable actions:

• Data operations: search for, download, import, or synthesize public datasets
• Training operations: launch SFT or GRPO with LoRA or other permitted configurations
• Status operations: query job state
• Validation operations: construct and run checks on the agent's own validation data
• Branching operations: branch or stop a lineage

Safe retention: For score-gated weight evolution, the controller retains the selected checkpoint $k^{\star}$ only when:

$$\Delta^{\mathrm{raw}}(k) = \widehat{s}_{\Gamma}(Y(k)) - \widehat{s}_{\Gamma}(Y^{\mathrm{in}}) > 0;$$

Otherwise it rolls back to $Y^{\mathrm{in}}$. Consequently:

$$\Delta^{\mathrm{ret}} = \widehat{s}_{\Gamma}(Y^{\mathrm{out}}) - \widehat{s}_{\Gamma}(Y^{\mathrm{in}}) \geq 0.$$

The nonnegative retained improvement follows from safety rollback and is therefore a selection outcome, not evidence that every attempted update improves capability.

Empirical Validation / Results

RQ1: How Vague Goals Change Post-Training

Question: How do vague goals change post-training outcomes and search trajectories relative to an explicit-task setting?

Main result: Vague-goal final scores are lower overall, but task-level changes vary:

• Claude Opus 4.8 under vague-goal prompting reaches 27.07, compared with the 32.90 official reference
• GPT-5.6 reaches 29.58, compared with the 36.23 official reference

Trajectory pattern 1: Vague goals are associated with different resource use:

• 2,109 more seconds of decision-model thinking
• 0.61 more GPU-idle hours per matched run pair
• Active training and evaluation time falls by 1.27 hours
• Accesses to agent-visible task materials become 2.98× and 2.39× as dense

Trajectory pattern 2: Vague-goal runs obtain less evaluation feedback per training start:

• Claude Opus 4.8 under vague goals: 0.57 evaluations per training start
• Official GPT-5.6: 2.63 evaluations per start—4.6× the feedback density

RQ2: Self-Directed Weight Evolution from Vague Goals

Final-only protocol (24 runs):

• All 24 runs complete their single final-checkpoint evaluation
• Only 1 of 12 model–goal pairs has a two-run mean above its base score
• The sole positive pair is scientific and academic reasoning with Qwen3.5-9B: both runs rise from 45.33 to ​48.00
• At the individual-run level, only 3/24 final checkpoints exceed their base scores

Adaptive-feedback protocol (30 configuration–goal cells):

• 28 produce an evaluated checkpoint; 21 satisfy eligibility requirements
• Only 2 cells have a best evaluated checkpoint whose score exceeds the base score
• Terra raises Qwen3.5-4B mathematics from 17.86 to ​20.10—the only cell whose checkpoint scores above base, passes eligibility, and remains after rollback

Key trajectories:

• Apparent progress can be recovery from regression: Qwen3.5-4B Self improves on mathematics from 0.79 to ​1.75 to ​2.38 while remaining far below its 17.86 base score
• More search does not mean better search: Sol searches most broadly (33 evaluated checkpoints, 76.56 GPU-hours) yet none exceeds its corresponding base score

Failure mode: Numeric-label SFT is associated with answer-format collapse:

• Five submitted checkpoints trained on numeric-label MMLU SFT—all 21,000 training targets are single-digit labels
• All 279 corresponding evaluation outputs are single digits
• Their scores are 0, 0, ​0, ​6.141, and 0

RQ3: Harness Evolution

Design: With Qwen3.5-4B as the fixed runtime model, four creators attempt one-step harness evolution for the academic and scientific writing goal.

Results (task macro / example micro):

Trajectory patterns:

1. Narrow agent validation can specialize the harness to the wrong proxy: Luna raises a creator-designed checklist from $7/8$ to $8/8$ by adding a fixed five-part research-answer template, but this template can recast requests for mechanisms, formulas, or engineering details as study-design problems.

2. Answer review does not guarantee output integrity: Terra correctly rejects a two-pass reviewer that invents unsupported results, but its final harness retains a different failure path—an empty final model response causes the pre-tool partial draft to be submitted.

Theoretical and Practical Implications

Theoretical Implications

1. Target operationalization is a missing axis of autonomous post-training: The paper formalizes the conversion of a broad capability goal into trainable objectives, learning signals, and validation criteria—a process previously implicit in human-mediated benchmark design.

2. Distinction between candidate progress and retained gain: The results reveal that producing an evaluated checkpoint is common, but producing an above-base goal score is rare—closing the training loop is not the same as closing the capability loop.

3. Safety rollback as selection, not evidence: The nonnegative retained improvement follows from rollback and is therefore a selection outcome, not evidence that every attempted update improves capability—a crucial distinction for evaluating self-evolution systems.

Practical Implications

1. Benchmark design: Aspire demonstrates that self-evolution benchmarks must keep evaluation items sealed while providing bounded aggregate feedback to approximate sparse deployment conditions without turning the evaluator into an agent-visible task specification.

2. Agent deployment: The findings mirror real-world FDE challenges—a capable model is not yet a working system, andthe gap between executing updates andretaining target-aligned improvements remains largely open.

3. Training data selection: The documented answer-format collapse from numeric-label SFT highlights concrete risks in autonomous data selection, suggesting that agents need better mechanisms for detecting distributional mismatch between training targets and evaluation requirements.

4. Harness design: The RQ3 results show that agents can generate functional successor harnesses for goal interpretation, tool use, and self-evaluation, but even the strongest observed successor remains below a well-engineered reference—indicating that harness engineering remains a bottleneck for autonomous system improvement.

Conclusion

Main Takeaways

1. Vague goals redirect search effort toward goal interpretation: In matched runs, vague-goal trajectories record more decision-model thinking and less active update time, but yield lower aggregate outcomes than explicit-task references.

2. Self-directed weight evolution rarely produces retained improvements: Only 1 of 12 final-only Avg@2 outcomes exceeds its base score, and adaptive feedback often produces locally rising trajectories that remain below base—agents complete the update loop more consistently than they preserve base performance.

3. Harness evolution remains below engineered references: One-step harness evolution can produce executable, behaviorally distinct successor harnesses, but all three valid successors remain numerically below Qwen-Agent, with bottlenecks in validation diversity and output integrity.

4. Progress must be judged against the base model, not only against the previous checkpoint: Continued search and training can erase earlier improvements, making checkpoint selection and rollback necessary.

Future Directions

• Broader goal coverage: The six goals represent a limited scope; broader coverage across more capability domains is needed
• Repeated runs: One canonical adaptive-feedback run per configuration–goal cell limits statistical power; repeated runs would strengthen conclusions
• Refreshed evaluation-set versions: New evaluation versions would reduce contamination risks over time
• Recursive evolution: Controlling drift and unrelated capability loss during recursive decision-model replacement remains an important direction
• Content-level trace evidence: Currently available only under controlled access; broader access could enable deeper failure-mode analysis