# Evaluating Governed LLM-Driven ML Experimentation: Lifecycle, Provenance, and Failure Metrics

> Governed harnesses with append-only ledgers expose process failures invisible to task scores, achieving 100% provenance across 1,445 trials while ungoverned loops silently lose 86% of trials.

- **Source:** [arXiv](https://arxiv.org/abs/2611.23678)
- **Published:** 2026-11-21
- **Permalink:** https://picx.dev/p/PGYkzB
- **Whiteboard:** https://picx.dev/p/PGYkzB/image

## Summary

## Summary

- **Core contribution**: Proposes evaluating LLM-driven ML experimentation systems on their *process* (governance metrics) in addition to their *outcome* (task score), via a "governed harness" with a control plane that owns trial lifecycle, scope validation, and keep/discard decisions.
- **Key mechanism**: An append-only ledger records one immutable, typed record per trial (kept, discarded, or failed_invalid), guaranteeing 100% provenance completeness across all 1,445 trials on six ML tasks.
- **Main findings**: The ledger exposed failures invisible to task scores—a no-memory arm with 120 total failures, a bounded-proposal adapter that ran out, and a reset failure detected via reproducibility hashes—and a matched counterfactual showed an ungoverned loop silently lost 43 of 50 trials.
- **Memory ablation**: Memory's effect on repeated-bad proposals is task-dependent; summary memory gave the lowest repeated-bad rate in 2/3 ResNet replicates but reversed on the synthetic task.
- **Artifact release**: All 1,445 trial ledgers are released so every reported number can be recomputed without re-running training.

## Introduction and Theoretical Foundation

The paper addresses a fundamental gap in evaluating autonomous ML experimentation agents: **scores say nothing about process**. An agent reporting a 0.05 AUC improvement may have edited protected files, silently dropped crashed runs, or repeated the same bad idea many times. For scientific ML, auditability—"a traceable and reproducible record of every decision and its evidence"—matters as much as the score.

The theoretical framing follows the view of an agent as a **model plus a harness** forming a compound AI system. Three roles are separated:

- **Manager** (typically an LLM): proposes one bounded change per trial
- **Worker**: applies the patch and runs training
- **Control plane**: owns trial lifecycle, scope validation, metric parsing, keep/discard decisions, and ledger appends

**Five auditability desiderata** (Table 1):

| # | Desideratum | Operational meaning |
|---|---|---|
| D1 | Bounded edit scope | Agent cannot change protected benchmark/data/eval files |
| D2 | Complete attempt accounting | Every attempt recorded, including invalid proposals and crashes |
| D3 | Independent adjudication | Acceptance/rejection by fixed control-plane rules, not the agent |
| D4 | Provenance linkage | Each attempt links to patch, config, run, metric, decision evidence |
| D5 | Recomputability | Governance claims recomputable from ledger without re-running training |

The key distinction from prior work (AIDE, MLAgentBench, MLE-bench, MLflow, W&B): existing systems either couple search state with decision-making (AIDE), ask only "did the agent improve the task?" (MLAgentBench), or record runs post-hoc without governing them (trackers). This paper inverts the design—the control-plane contract is fixed and externally visible; managers and workers are replaceable.

## Methodology

### System Architecture

The governed harness (Figure 2) separates replaceable components (manager, worker, node) from the fixed control-plane contract. The control plane owns: scope validation, lifecycle state machine, keep/discard decisions, provenance ID assignment, and memory context updates.

### Trial Lifecycle State Machine

Each trial follows a fixed lifecycle (Table 4):

| From state | To state | Condition |
|---|---|---|
| pending | executing | Pending guard written; worker starts |
| executing | kept | Metric parses; beats current best; patch diff retained |
| executing | discarded | Metric parses; does not beat current best |
| executing | failed_invalid | Scope violation, runtime error, missing metric, no-op patch, or precondition failure |
| executing | failed_invalid | Mid-trial crash via stale pending guard at next campaign start |

**Lifecycle invariant**: For every budget slot entering executing, exactly one terminal record is appended before the next slot opens. Enforcement has two paths: normal (pending guard written/deleted) and crash-recovery (stale guard raises PendingTrialError, a failed_invalid record is appended).

### Benchmark Nodes

Six nodes share the same contract, differing only in NodeSpec YAML (Table 3):

| Node | Task type | Worker | Metric | Direction |
|---|---|---|---|---|
| resnet_trigger | Waveform classification | Claw | val_auc | max |
| lr_synthetic | Logistic regression | Local | val_score | max |
| openml_credit_g | Tabular classification | Local | val_auc | max |
| openml_bank_mkt | Tabular classification | Local | val_auc | max |
| mlagentbench_vec. | NumPy convolution | Local | speed_score | max |
| autoresearch_linux | LM pretraining | Claw | val_bpb | min |

### Governance Metric Suite (Table 5)

Given $N$ trials:

- **Acceptance rate (AR)**: $|\text{kept}| / N$
- **Invalid rate (IR)**: $|\text{failed\_invalid}| / N$
- **Repeated-bad rate (RBR)**: repeated-bad proposals / $N$
- **Provenance completeness**: trials with all five IDs / $N$
- **Artifact-reference coverage**: patch/log refs and hashes
- **Failure taxonomy**: counts per FailureCategory

### Memory Ablation Design

Three memory modes: no prior-trial context, append-only summary, and rationale-augmented summary. The behavioral hypothesis tested:

$$\text{RBR}(\text{none}) > \text{RBR}(\text{summary}) > \text{RBR}(\text{rationale})$$

### Metric Validation

Structural relabeling agreement: Cohen's $\kappa = 0.830$ (95% CI $[0.703, 0.958]$) on a stratified sample of 50 failed_invalid trials.

## Empirical Validation / Results

### Cross-Node Governance Summary (Table 7)

| Node | Trials | AR% | IR% | Lifecycle outcomes |
|---|---|---|---|---|
| resnet_trigger (case study) | 20 | 20.0 | 55.0 | kept, disc., failed_inv. |
| resnet_trigger (L40S, all arms) | 225 | 46.7 | 0.4 | kept, disc., failed_inv. |
| lr_synthetic | 450 | 8.4 | 46.2 | kept, disc., failed_inv. |
| openml_credit_g | 230 | 13.5 | 26.1 | kept, disc., failed_inv. |
| openml_bank_marketing | 230 | 13.0 | 32.2 | kept, disc., failed_inv. |
| mlagentbench_vec. | 50 | 28.0 | 48.0 | kept, disc., failed_inv. |
| autoresearch (none arm) | 120 | 0.0 | 100.0 | failed_inv. only |
| autoresearch (summary) | 120 | 11.7 | 40.8 | kept, disc., failed_inv. |

**Provenance completeness: 100% on every row.**

### Governed vs. Ungoverned Counterfactual (Table 9)

| Node | N | Governed | Ungoverned | Drops |
|---|---|---|---|---|
| autoresearch_linux (IR=100%) | 30 | 1.00 | 0.00 | 30 (100%) |
| openml_bank_mkt (IR=55%) | 20 | 1.00 | 0.35 | 13 (65%) |

**Combined: 43/50 ungoverned trials absent vs. 0/50 governed.**

### Large-Scale Memory Replication (225 trials, NVIDIA L40S)

Across 5 clean seeds with verified reset integrity (shared node_state_hash at trial 1):

- Mean best AUC: none < summary ≈ rationale (0.899 vs. 0.939 and 0.941)
- Cross-seed variability: $\sigma_{\text{none}} = 0.009$ vs. $\sigma_{\text{summary}} = \sigma_{\text{rationale}} = 0.002$ (4–5× smaller)
- Memory arms diverge from no-memory by trial 5, hold gap through trial 15
- Provenance completeness: 100% across all 225 trials

### Real LM Training Node (autoresearch_linux)

- None arm: IR = 1.00 across all 120 trials (4 seeds)
- Summary arm: 3/4 seeds produce valid improving trials (mean val_bpb reduction $-0.062$); seed s2 suffered total worker failure
- Across 240 trials including 151 runtime-error crashes: provenance completeness 240/240 (100%)

## Theoretical and Practical Implications

### Theoretical Implications

1. **Process-based evaluation paradigm**: The paper argues that autonomous ML experimentation should be evaluated on governance metrics (provenance, lifecycle classification, scope enforcement, failure taxonomy) alongside task metrics—a shift from outcome-only benchmarking.

2. **Structural vs. empirical guarantees**: The lifecycle invariant holds "by construction of the control-plane code, not because nothing crashed." This distinguishes structural guarantees from empirical observations—a key methodological contribution.

3. **Replaceability principle**: Separating proposal (manager), execution (worker), and authority (control plane) means the LLM manager is a replaceable component, enabling controlled experiments on manager behavior without changing governance.

### Practical Implications

1. **Auditability for scientific ML**: Every decision can be traced to its evidence (patch, log, metric, decision), enabling reproducible scientific experimentation.

2. **Failure visibility**: The ledger exposes failures that task scores hide—total failures, bounded-proposal exhaustion, and reset failures detectable via reproducibility hashes.

3. **Memory diagnostics**: Repeated-bad rate serves as a per-node diagnostic for memory sensitivity, but the task-dependence warns against general claims about memory formats.

4. **Reproducibility**: Released ledgers allow recomputation of all reported numbers without re-running training (no GPU required).

## Conclusion

The paper demonstrates that "autonomous ML experimentation should be evaluated on governance metrics—provenance, lifecycle classification, scope enforcement, and failure taxonomy—not only task metrics." Across 1,445 governed trials on six ML nodes, the control plane classifies every outcome without silent corruption, and provenance is complete by construction.

**Key limitations** acknowledged by the authors:
- Short budgets and limited node diversity
- Structural relabeling is a consistency check, not a human inter-rater study
- ResNet AUC gains fall within baseline seed noise
- Human-auditor utility not yet tested

**Future directions**: A human-auditor utility study (do reviewers with governed ledgers detect seeded errors faster?), two-rater validation, and holdout-node replication.

**Evidence strength summary** (Table 10): Demonstrated properties include control-plane classification, 100% provenance, guard failure visibility, and protocol transfer across node types. Observed-but-not-claimed findings include memory ordering on AUC (5 clean seeds) and smaller cross-seed spread with memory. The claim that "rationale reduces repeated-bad rate" is explicitly **not claimed** due to non-monotonic results.

---

_Markdown view of https://picx.dev/p/PGYkzB, served by PicX — AI-generated visual whiteboard summaries of research papers._
