# The Critic Can Hurt: On the Safety of Critics in LLM-Guided AutoML Search

> A naive LLM critic can drive AutoML search below the no-critic random-search floor on 7 of 8 tabular datasets, with losses up to 0.245 AUC.

- **Source:** [arXiv](https://arxiv.org/abs/2611.19037)
- **Published:** 2026-11-21
- **Permalink:** https://picx.dev/p/n5Xya6
- **Whiteboard:** https://picx.dev/p/n5Xya6/image

## Summary

# The Critic Can Hurt: On the Safety of Critics in LLM-Guided AutoML Search

## Summary (Overview)

- **Core finding**: A naive 8B LLM critic can actively *harm* iterative search in AutoML, driving performance **below the no-critic random-search floor** on 7 of 8 tabular datasets, with worst-case losses up to **0.245 AUC**.
- **Key contribution**: Introduces the concept of **critic safety** and a **four-arm audit** protocol (no-critic floor, random feedback, real critic, oracle ceiling) that isolates a critic's marginal effect on a shared cached configuration frontier.
- **Mechanistic explanation**: The loop's best-so-far (elitist) dynamics make it resilient to *non-directional* (random) errors but highly vulnerable to *systematically adversarial* (directional) errors. An idealized model yields the necessary condition for net benefit: $$q(m-1) > \beta$$ (competence $q$, lookahead $m$, adversarial rate $\beta$).
- **Prevention**: A training-free **validation gate** and a **learned detector** (packaged as the CRITFAULT benchmark) can reduce realized loss, but neither guarantees preservation of the full-budget no-critic baseline on every dataset.
- **Practical takeaway**: A critic is not a free addition—audit it against a no-critic control at the intended budget before trusting it; gate or omit it when capability or verification is insufficient.

---

## Introduction and Theoretical Foundation

### Background and Motivation

Automated machine learning (AutoML) searches over preprocessing, feature transformations, model families, and hyperparameters. Recent agentic systems organize this as a **propose–evaluate–correct loop**: a proposer selects a candidate pipeline, it is scored on held-out data, and an LLM critic converts the score and search history into recommendations that condition the next proposal, while the system retains a best-so-far incumbent.

The authors argue that critics and proposers fail **asymmetrically**:
- A weak **proposal** is evaluated, scores poorly, and is discarded by the best-so-far rule (verified by construction).
- A **critique** receives no such check—it is neither scored nor filtered, yet it conditions every subsequent proposal.

The cost of a bad critique is not a degraded incumbent (elitist selection protects what is found), but **sustained misdirection of a finite evaluation budget** away from configurations unguided search would otherwise reach.

### Theoretical Foundation

The paper builds on several strands of prior work:
- **Classical AutoML**: Bayesian/evolutionary optimizers that optimize validation metrics directly without reasoning about *why* a configuration failed.
- **LLM-based AutoML**: Systems like SELA and AIDE use LLMs primarily as *proposers*, assessed at outcome level—the critic role is not isolated.
- **Self-critique literature**: Reflexion, Self-Refine, and ReAct show verbal critique can help on coding/reasoning, but intrinsic self-correction can be net-negative (degrades accuracy once oracle labels are removed, turns correct answers incorrect).
- **LLM-as-judge reliability**: Documented biases and instability in LLM-as-judge setups; verifier-gated selection is an active theme.
- **Noise tolerance**: Robust preference optimization (rDPO) tolerates flip rates below $1/2$; this paper's resilience threshold is structurally different—it arises from elitist best-so-far dynamics rather than gradient preference training.

---

## Methodology

### 3.1 Critic-Guided Search Loop

A critic-guided AutoML agent tunes a tabular pipeline over $T$ rounds. Each round:
1. A **proposer** selects an untried configuration $a_t$ from a finite space $\mathcal{A}$.
2. The configuration is trained and scored on a frozen validation split.
3. An **LLM critic** turns the score and history into a natural-language correction.
4. The agent returns the best-so-far validation incumbent $\hat{a}_T$, whose quality is the hidden test utility $s_{\text{test}}(\hat{a}_T)$.

For dataset $D$ and condition $r$, the expected test utility is $J_D(r) = \mathbb{E}[s_{\text{test}}(\hat{a}^{(r)}_T)]$.

### 3.2 Shared Cached Frontier

- Each dataset is partitioned once into train/validation/test splits.
- Every configuration in a 96-configuration pipeline space is scored *before* search, with validation and test AUC cached.
- All conditions share the same frozen frontier (candidate space, split, budget, outcomes), differing only in the trajectory-selecting mechanism.
- The critic acts **only through the feedback channel** (never trains/executes code, alters candidate space, or sees test utilities).

### 3.3 Four-Arm Audit

Four budget-matched conditions on the shared frontier:

| Arm | Description | Role |
|-----|-------------|------|
| (i) No critic | Uniform sampling over untried configurations | Unguided-search floor |
| (ii) Random feedback | Independently seeded uniform random search | Second draw of floor |
| (iii) Real critic | Feedback from LLM under evaluation | Object of study |
| (iv) Oracle | Directly selects highest-validation-utility untried config | Unattainable ceiling |

Paired contrasts:
$$\Delta_{NC}(D) = J_D(\text{real}) - J_D(\text{no-critic}), \quad \Delta_{RF}(D) = J_D(\text{real}) - J_D(\text{random-feedback})$$

A critic is **below floor** when $\Delta_{NC}(D) < 0$.

### 3.4 Controlled Fault Injection

Corrupts a clean reference policy $\pi^\star$: each round follows $\pi^\star$ with probability $1-p$ and a corrupted choice with probability $p$. Three error structures:

- **RANDOM**: Uniform untried (non-directional)
- **WORST**: Lowest-utility untried (globally adversarial)
- **MISLEAD**: Lowest-utility incumbent neighbor (locally plausible but directionally harmful)

### 3.5 Experimental Scope

- 12 public OpenML classification datasets (Table 12 in paper).
- In-space headroom $h_D = J_D(\text{oracle}) - J_D(\text{no-critic})$ is below 0.01 AUC on all 12 datasets.
- Primary audit: $T=8$, 8 matched seeds, deterministic decoding.
- Primary critic: Llama-3.1-8B; capability ladder spans 3B–70B open-weight models and Claude Haiku 4.5.

---

## Empirical Validation / Results

### 4.1 Limited Upside of Guidance

- Clean hill-climbing reference achieves +0.001 to +0.005 AUC over no-critic (30 seeds; suite-level Wilcoxon $p = 0.001$).
- The in-space oracle matches or beats AutoGluon-Tabular on 5 of 8 datasets—so floors are SOTA-competitive.
- **Asymmetric opportunity**: Attainable upside is small, but downside from misleading guidance can be considerably larger.

### 4.2 Weak Critics Fall Below the Floor

A naive Llama-3.1-8B critic (temperature 0) has a **negative No-Critic delta on 7 of 8 binary datasets**, ranging from −0.009 AUC (PHONEME) to **−0.245 (ADULT)**. Only CREDIT-G is positive (+0.006).

An exact one-sided sign-flip test over all $2^8$ sign assignments gives $p = 0.0078$ for a negative suite-level mean.

### 4.3 Harm Attenuates with Setting and Capability

**Larger frontier, longer horizon** (250 configs, $T=20$): Below-floor effect decreases by 1–2 orders of magnitude:
- ADULT: −0.245 → +0.0003
- ELECTRICITY: −0.072 → +0.0003
- JM1: −0.080 → −0.004

**Capability ladder** (Table 1): The weak 2024 8B critic is negative on 5 of 6 datasets. At 14B tier, all point estimates are within 0.012 of zero (3 of 6 remain negative). The 14B critic improves on 5 of 6 datasets relative to 8B (paired sign-flip test, $p = 0.047$).

**Key finding**: The pattern is *not monotone* in size or release date—Llama-3.3-70B is negative on 5 of 6 datasets (up to −0.031). This supports **task-dependent attenuation, not a capability threshold**.

### 5.1 Fault Injection: Wide but Asymmetric Safety Envelope

| Error Structure | Finding |
|-----------------|---------|
| RANDOM ($p=1$) | Nearly costless: final degradation < 0.005 AUC on every dataset |
| WORST/MISLEAD | Resilience threshold $p^\star = 0.6$–$0.9$ on primary binary datasets (LETTER at 1.0) |

### 5.2 Theoretical Results

**Proposition 1 (Uniform corruption preserves the random-search floor)**: If every selected configuration is uniformly distributed over remaining untried configurations, then for every $p \in [0,1]$ and horizon $T$:
$$\mathbb{E}[b^{\text{uniform}(p)}_T] = \mathbb{E}[b^{\text{no-critic}}_T]$$

This follows from without-replacement exchangeability.

**Proposition 2 (Resilience under adversarial corruption)**: Let $K \sim \text{Binomial}(T, 1-p)$ be the number of effective rounds. Under Assumption 1 (adversarial rounds never raise the incumbent), the loop is $(\delta + \epsilon)$-robust when $\Pr[\text{Binomial}(T, 1-p) < T_{\text{sat}}] \leq \epsilon$. The approximation:
$$p^\star \approx 1 - \frac{T_{\text{sat}}}{T}$$
With $T=8$, $T_{\text{sat}} \in [1,3]$ predicts $p^\star \in [0.63, 0.88]$, matching measured 0.6–0.9.

### 5.5 Net-Benefit Criterion

**Lemma 5.1 (Conditional effective sample size)**: Conditioned on $K_c$ competent and $K_u$ uniform rounds, the final incumbent has the same distribution as the maximum of $N_{\text{eff}} = mK_c + K_u$ i.i.d. score draws.

**Proposition 3 (Effective-horizon criterion)**: The critic can be net-beneficial only if:
$$q(m-1) > \beta$$
equivalently $T_{\text{eff}} > T$. In the rare-improvement regime, the per-round advantage is $\epsilon[q(m-1) - \beta] + O(\epsilon^2)$ as $\epsilon \to 0$.

**Important caveat**: The condition is *necessary, not sufficient*. With Bernoulli($1/2$) scores, $q=0.3$, $m=3$, $\beta=0.3$: $q(m-1) = 0.6 > \beta$, yet a critic round reaches the top score with probability 0.4625 < 0.5—worse at every budget $T$.

### 5.6 Cross-Domain Test of Mismatch Hypothesis

| Setting | $\Delta$ | 95% CI | Verdict |
|---------|----------|--------|---------|
| Tabular AutoML (96) | −0.245 | [−0.27, −0.22] | harm |
| Credit scoring (96) | +0.000 | [−0.001, +0.002] | neutral |
| 20NG text-pipe (36) | −0.003 | [−0.008, +0.001] | neutral |
| 20NG weak→strong (50) | +0.003 | [−0.005, +0.010] | neutral |
| M4 forecasting (13) | +0.001 | [+0.001, +0.002] | helps |
| RAG retrieval (54) | +0.007 | [+0.006, +0.008] | helps |

Holding the critic fixed, model size cannot explain this variation—consistent with the **prior–reality mismatch hypothesis**.

### 6.1 Prevention Motivation

- The failure **reproduces in the published CAAFE system**: negative on 3 datasets (ADULT −0.030, BANK-MARKETING −0.054, CREDIT-G −0.090), mean −0.021.
- **Prompting does not reliably remove harm**: Chain-of-thought lifts ADULT from −0.278 to −0.041, yet 3 of 4 datasets stay below floor. Five-sample self-consistency spends 5× generations while doing worse than plain chain-of-thought.

### 6.6 End-to-End Prevention Results

Across six datasets:
- Raw policy: −0.00541 AUC relative to uniform
- Simple non-improvement gate: **+0.00046**
- Learned detector: **−0.00015**

The simple gate significantly improves over raw on bank-marketing and kc1; the detector on bank-marketing and electricity. **Neither guarantees floor preservation on every dataset.**

### CRITFAULT Benchmark

~3,840 labeled loop transitions across 12 datasets. Leave-one-dataset-out AUC = 0.95; leave-one-model-out mean = 0.988. Fully deployable f0–f2 detector (evaluated utility, incumbent-relative change, round fraction) attains 0.982 on six datasets.

---

## Theoretical and Practical Implications

### Theoretical Implications

1. **Critic safety is a measurable property**: The four-arm audit provides a rigorous protocol for isolating a critic's marginal effect, controlling for search randomness, training noise, and pool differences.

2. **Error structure, not error rate, determines harm**: The elitist best-so-far dynamics create an asymmetric safety envelope—random errors are nearly free, but systematically adversarial ones accumulate cost. This differs fundamentally from symmetric flip-rate thresholds in robust preference optimization (rDPO's $\epsilon < 1/2$).

3. **The necessary condition $q(m-1) > \beta$** organizes the design space:
   - Stronger critic → raises $q$, $m$
   - Verification → lowers $\beta$
   - Breadth → raises $m$
   
   It also predicts that parallel $k$-wide search is a partial structural defense (falls below floor on only 1/12 datasets vs. 9/12 for sequential, tree-search, and multi-agent backbones).

### Practical Implications

1. **Audit before trusting**: Practitioners should run a critic against a no-critic control at the intended budget before deployment.

2. **Asymmetric risk**: On small, fixed frontiers, the upside of guidance is bounded (~0.005 AUC) while the downside is not (up to 0.245 AUC)—roughly 6× the largest reference gain typically, up to 50× in the worst case.

3. **Gate or omit**: When critic capability or verifier reliability is insufficient, gate the critique (VGC) or omit it entirely.

4. **High-stakes domains**: Tabular ML is the production default in credit scoring, fraud detection, and clinical risk. Even a ~0.03 AUC drop can shift a non-trivial slice of accept/reject boundaries.

---

## Conclusion

**Main takeaways**:
- Adding an LLM critic to an iterative search loop is **not guaranteed to be harmless**—a naive 8B critic can drive search below the no-critic random-search floor on 7 of 8 datasets.
- **When**: Harm attenuates with critic capability and varies with frontier and budget, without a universal capability threshold.
- **Why**: Best-so-far search tolerates non-directional errors but not directional ones; the idealized necessary condition is $q(m-1) > \beta$.
- **How**: Observable gates targeting $\beta$ reduce average loss in deployable reruns, without guaranteeing the no-critic baseline on every dataset.
- Cross-domain audits are consistent with the hypothesis that harm tracks **mismatch between the critic's priors and the task**.

**Future directions**:
- Extending the quantitative boundary $q(m-1) > \beta$ to self-refinement, verifier-guided decoding, or preference-based optimization requires setting-specific assumptions.
- Directly identifying $(q, m, \beta)$ for real critics remains open.
- The authors will release **CRITFAULT** and all artifacts upon publication—the first benchmark for detecting harmful critic feedback in tabular AutoML.

---

_Markdown view of https://picx.dev/p/n5Xya6, served by PicX — AI-generated visual whiteboard summaries of research papers._
