# The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks

> Taste-Bench reveals frontier LLM agents achieve only ~60% accuracy on long-horizon decision quality, but this taste is distillable, boosting end-to-end task success from 14.6% to 33.7%.

- **Source:** [arXiv](https://arxiv.org/abs/2609.25804)
- **Published:** 2026-09-24
- **Permalink:** https://picx.dev/p/bleQMu
- **Whiteboard:** https://picx.dev/p/bleQMu/image

## Summary

## Summary (Overview)

- **Novel Benchmark**: Introduces Taste-Bench, the first benchmark measuring "taste" in LLM agents—the ability to make good long-horizon decisions at decision forks where outcomes are not immediately visible.
- **Automatic Mining**: Constructs 502 taste questions automatically from existing agent trajectories (2,677 engineering rollouts on SWE-bench Pro and 1,132 research runs from RE-Bench/HCAST) without human annotation, using two complementary methods: parallel trajectories (divergent attempts at the same task) and detour trajectories (self-corrections within a single run).
- **Key Findings**: Frontier models struggle significantly—the best model (GPT-5.6 Sol) achieves only 59.7% accuracy (random guessing = 25% under dual-order protocol). Accuracy drops dramatically with time horizon (62.3% for in-prefix evidence to 21.0% for more-work evidence), and increased reasoning budget does not improve accuracy.
- **Trainable Taste**: Demonstrates that taste can be distilled—a student model trained via teacher distillation (with privileged outcome information) improves from 30.0% to 47.9% on held-out tasks and boosts end-to-end executor success on SWE-bench Pro from 14.6% to 33.7%.
- **Benchmark Quality**: Human review confirms 98.8% agreement with mined labels, and the benchmark is publicly released with code and dataset.

## Introduction and Theoretical Foundation

The paper addresses a critical gap in LLM agent evaluation: while agents increasingly work on long-horizon tasks (ML research, software engineering, scaffold refinement), existing benchmarks only measure **end-to-end success**—whether the agent finishes the task—but provide no measure of the **quality of decisions made along the way**.

**Definition**: The authors formalize "taste" as the ability to make good long-horizon decisions—choosing the better direction at points where multiple paths are available and the consequences of the choice only appear much later in the trajectory.

**Key Insight**: The later part of a trajectory provides *hindsight evidence* for earlier decisions. Agent systems routinely make multiple attempts at the same task; when attempts diverge at a common point (a **decision fork**), the recorded outcomes identify which direction was better. This allows automatic labeling without expert annotation.

**Formal Framework**: Given attempts at task $q$ sharing prefix $h_t = (o_0, a_0, \ldots, o_t)$ that diverge into candidates $c_1$ and $c_2$ at fork time $t$, each branch $i$ produces evidence $E_i$ mapped to a scalar outcome $U$. The label is:

$$y = \arg \max_{i \in \{1, 2\}} U(E_i) \tag{1}$$

The question is $x = (q, h_t, c_1, c_2)$ where everything after the fork is hidden. A model's taste is estimated as:

$$\widehat{T}(\pi) = \text{fraction of questions where } \pi(x) = y$$

## Methodology

### Fork Mining — Two Complementary Constructions

**1. Parallel Trajectories**: Aligns pairs of attempts at the same task that diverge at a fork. The shared prefix becomes the question context, and the two directions become candidates. The recorded outcomes (test passes, research scores) determine the label. Captures wrong judgments the agent never notices.

**2. Detour Trajectories**: A generator model reads a single trajectory with its outcome and locates three events: (a) the step where the agent takes a direction, (b) the observed failure ending that direction, (c) the step where the agent takes a different direction that completes the task. The fork is placed before the abandoned direction. Captures wrong judgments the agent corrects later.

### Filtering Pipeline

- **Trivial questions removed**: Questions answerable from candidate wording alone (judged by models seeing only candidates, no trajectory)
- **Undecidable questions removed**: Questions where the label is not clearly consistent with the full record (judged by models seeing the complete task record)
- **Human review**: 100 sampled questions reviewed by two independent reviewers; 98.8% agreement with mined labels (Cohen's κ = 0.973)

### Evaluation Protocol

- Each question evaluated **twice** (seeded order and exact reverse) to address position bias
- A question counts as correct **only when both orders are answered correctly**
- Random guessing accuracy = 25% (product of ½ per order); position-biased models score 0%

### Distillation Recipe

- **Base model**: Qwen3.6-27B with LoRA adapters
- **Task-disjoint folds**: 390 engineering questions split into two folds; each student trains on one and evaluates on the other (no task seen in both)
- **Teacher-student setup**: Privileged teacher (same frozen base model) sees the question *plus* a demonstration of the supported candidate; student sees only the benchmark-visible context
- **Loss**: Token-level forward KL from student to teacher, computed on teacher-sampled continuations (following SDPO framework)
- **End-to-end evaluation**: Student's judgment at each fork is written as advice into the task context; a fixed executor agent completes the task independently

## Empirical Validation / Results

### Model Performance on Taste-Bench

| Model | Average (%) | Research (%) | Engineering (%) |
|-------|-------------|--------------|-----------------|
| GPT-5.6 Sol | **59.7** | — | — |
| GPT-5.5 | 59.5 | — | — |
| (Other 12 models) | Widely dispersed below | — | — |

**Finding 1**: Current frontier models show limited taste—even the strongest cannot reliably identify the better direction on binary choices.

### Effect of Time Horizon

Forks are annotated with four ordinal time-horizon levels (in-prefix, inferable, next-step, more-work):

| Horizon Level | Mean Accuracy (14 models) |
|---------------|--------------------------|
| In-prefix | 62.3% |
| Inferable | — |
| Next-step | — |
| More-work | 21.0% (near random guessing) |

**Finding 2**: Model errors concentrate on forks with long time horizons—accuracy falls as horizon increases, confirming that answering requires predicting later work.

### Effect of Reasoning Budget

- Increasing reasoning effort (3 settings) changed GPT-5.6 Sol by −0.2 points and GPT-5.6 Luna by +2.2 points (statistically insignificant)
- Models produce *most* reasoning tokens at the *more-work* level—the level with lowest accuracy

**Finding 3**: Larger reasoning budget does not improve taste; models recognize hard forks but cannot resolve them.

### Comparison with End-to-End Benchmarks

- Pearson correlation with SWE-bench Verified: $r = +0.63$ ($R^2 = 0.39$)—only partial correlation
- Engineering subset correlation: $r = +0.37$ (despite being mined from SWE-bench Pro tasks)
- Top 4 SWE-bench Verified models are within 4.0 points there but 10.7 points apart on Taste-Bench

### Distillation Results

| Setting | Held-out Accuracy (%) | Mean Dual-Order Accuracy (%) |
|---------|----------------------|------------------------------|
| Base model | 30.0 | 42.7 |
| Distilled student | **47.9** | **62.4** |

**End-to-end SWE-bench Pro success (41 held-out tasks)**:

| Setting | Success Rate |
|---------|--------------|
| No advice | 14.6% |
| Correct advice (upper bound) | 39.0% |
| **Student advice** | **33.7%** |

**Finding 4**: Taste is distillable—the student gains 17.9 points on unseen tasks and achieves most of the upper bound on end-to-end success (+19.1 points over baseline).

## Theoretical and Practical Implications

- **Measurement without annotation**: The paper demonstrates that decision quality can be measured automatically from hindsight over existing trajectories, eliminating the need for expensive expert annotation and enabling scalable evaluation across new domains.

- **Complementary benchmark dimension**: Taste-Bench measures *process quality* rather than just *outcome success*, providing a new axis for model evaluation that is only partially correlated with end-to-end benchmarks ($r = 0.63$). This suggests that end-to-end success does not fully capture decision-making ability.

- **Time-horizon insight**: The finding that accuracy degrades with time horizon (from 62.3% to 21.0%) reveals a fundamental limitation in current models' ability to reason about long-term consequences—a critical capability for autonomous agents.

- **Trainability of judgment**: The successful distillation of taste (requiring only a privileged teacher with outcome knowledge) suggests that judgment can be improved without additional task-specific data, and that improved judgment directly translates to better task outcomes.

- **Practical framework**: The advice-injection paradigm (student judges forks, executor follows advice) provides a practical architecture for improving agent performance on complex tasks.

## Conclusion

The paper makes three primary contributions:

1. **Formalization**: Defines "taste" as the ability to choose the better direction at decision forks and shows it can be measured from hindsight over existing trajectories without human annotation.

2. **Benchmark**: Releases Taste-Bench (502 questions across software engineering and ML research), demonstrating that current frontier models achieve only ~60% accuracy, with errors concentrating on long-horizon forks where increased reasoning effort does not help.

3. **Training method**: Shows that taste is trainable via teacher distillation, producing students that generalize to unseen tasks and improve end-to-end task success.

**Future directions** include: extending the benchmark to more domains, exploring whether taste can be further improved through other training paradigms, and investigating whether the advice-injection framework can be integrated into production agent systems. The authors hope Taste-Bench provides "a practical basis for measuring and training the judgment of long-horizon agents."

---

_Markdown view of https://picx.dev/p/bleQMu, served by PicX — AI-generated visual whiteboard summaries of research papers._