Full text not available for this paper

The Critic Can Hurt: On the Safety of Critics in LLM-Guided AutoML Search

Summary (Overview)

  • Core finding: A naive 8B LLM critic can actively harm iterative search in AutoML, driving performance below the no-critic random-search floor on 7 of 8 tabular datasets, with worst-case losses up to 0.245 AUC.
  • Key contribution: Introduces the concept of critic safety and a four-arm audit protocol (no-critic floor, random feedback, real critic, oracle ceiling) that isolates a critic's marginal effect on a shared cached configuration frontier.
  • Mechanistic explanation: The loop's best-so-far (elitist) dynamics make it resilient to non-directional (random) errors but highly vulnerable to systematically adversarial (directional) errors. An idealized model yields the necessary condition for net benefit: q(m−1)>βq(m-1) > \beta (competence qq, lookahead mm, adversarial rate β\beta).
  • Prevention: A training-free validation gate and a learned detector (packaged as the CRITFAULT benchmark) can reduce realized loss, but neither guarantees preservation of the full-budget no-critic baseline on every dataset.
  • Practical takeaway: A critic is not a free addition—audit it against a no-critic control at the intended budget before trusting it; gate or omit it when capability or verification is insufficient.

Introduction and Theoretical Foundation

Background and Motivation

Automated machine learning (AutoML) searches over preprocessing, feature transformations, model families, and hyperparameters. Recent agentic systems organize this as a propose–evaluate–correct loop: a proposer selects a candidate pipeline, it is scored on held-out data, and an LLM critic converts the score and search history into recommendations that condition the next proposal, while the system retains a best-so-far incumbent.

The authors argue that critics and proposers fail asymmetrically:

  • A weak proposal is evaluated, scores poorly, and is discarded by the best-so-far rule (verified by construction).
  • A critique receives no such check—it is neither scored nor filtered, yet it conditions every subsequent proposal.

The cost of a bad critique is not a degraded incumbent (elitist selection protects what is found), but sustained misdirection of a finite evaluation budget away from configurations unguided search would otherwise reach.

Theoretical Foundation

The paper builds on several strands of prior work:

  • Classical AutoML: Bayesian/evolutionary optimizers that optimize validation metrics directly without reasoning about why a configuration failed.
  • LLM-based AutoML: Systems like SELA and AIDE use LLMs primarily as proposers, assessed at outcome level—the critic role is not isolated.
  • Self-critique literature: Reflexion, Self-Refine, and ReAct show verbal critique can help on coding/reasoning, but intrinsic self-correction can be net-negative (degrades accuracy once oracle labels are removed, turns correct answers incorrect).
  • LLM-as-judge reliability: Documented biases and instability in LLM-as-judge setups; verifier-gated selection is an active theme.
  • Noise tolerance: Robust preference optimization (rDPO) tolerates flip rates below 1/21/2; this paper's resilience threshold is structurally different—it arises from elitist best-so-far dynamics rather than gradient preference training.

Methodology

3.1 Critic-Guided Search Loop

A critic-guided AutoML agent tunes a tabular pipeline over TT rounds. Each round:

  1. A proposer selects an untried configuration ata_t from a finite space A\mathcal{A}.
  2. The configuration is trained and scored on a frozen validation split.
  3. An LLM critic turns the score and history into a natural-language correction.
  4. The agent returns the best-so-far validation incumbent a^T\hat{a}_T, whose quality is the hidden test utility stest(a^T)s_{\text{test}}(\hat{a}_T).

For dataset DD and condition rr, the expected test utility is JD(r)=E[stest(a^T(r))]J_D(r) = \mathbb{E}[s_{\text{test}}(\hat{a}^{(r)}_T)].

3.2 Shared Cached Frontier

  • Each dataset is partitioned once into train/validation/test splits.
  • Every configuration in a 96-configuration pipeline space is scored before search, with validation and test AUC cached.
  • All conditions share the same frozen frontier (candidate space, split, budget, outcomes), differing only in the trajectory-selecting mechanism.
  • The critic acts only through the feedback channel (never trains/executes code, alters candidate space, or sees test utilities).

3.3 Four-Arm Audit

Four budget-matched conditions on the shared frontier:

ArmDescriptionRole
(i) No criticUniform sampling over untried configurationsUnguided-search floor
(ii) Random feedbackIndependently seeded uniform random searchSecond draw of floor
(iii) Real criticFeedback from LLM under evaluationObject of study
(iv) OracleDirectly selects highest-validation-utility untried configUnattainable ceiling

Paired contrasts:

ΔNC(D)=JD(real)−JD(no-critic),ΔRF(D)=JD(real)−JD(random-feedback)\Delta_{NC}(D) = J_D(\text{real}) - J_D(\text{no-critic}), \quad \Delta_{RF}(D) = J_D(\text{real}) - J_D(\text{random-feedback})

A critic is below floor when ΔNC(D)<0\Delta_{NC}(D) < 0.

3.4 Controlled Fault Injection

Corrupts a clean reference policy π⋆\pi^\star: each round follows π⋆\pi^\star with probability 1−p1-p and a corrupted choice with probability pp. Three error structures:

  • RANDOM: Uniform untried (non-directional)
  • WORST: Lowest-utility untried (globally adversarial)
  • MISLEAD: Lowest-utility incumbent neighbor (locally plausible but directionally harmful)

3.5 Experimental Scope

  • 12 public OpenML classification datasets (Table 12 in paper).
  • In-space headroom hD=JD(oracle)−JD(no-critic)h_D = J_D(\text{oracle}) - J_D(\text{no-critic}) is below 0.01 AUC on all 12 datasets.
  • Primary audit: T=8T=8, 8 matched seeds, deterministic decoding.
  • Primary critic: Llama-3.1-8B; capability ladder spans 3B–70B open-weight models and Claude Haiku 4.5.

Empirical Validation / Results

4.1 Limited Upside of Guidance

  • Clean hill-climbing reference achieves +0.001 to +0.005 AUC over no-critic (30 seeds; suite-level Wilcoxon p=0.001p = 0.001).
  • The in-space oracle matches or beats AutoGluon-Tabular on 5 of 8 datasets—so floors are SOTA-competitive.
  • Asymmetric opportunity: Attainable upside is small, but downside from misleading guidance can be considerably larger.

4.2 Weak Critics Fall Below the Floor

A naive Llama-3.1-8B critic (temperature 0) has a negative No-Critic delta on 7 of 8 binary datasets, ranging from −0.009 AUC (PHONEME) to −0.245 (ADULT). Only CREDIT-G is positive (+0.006).

An exact one-sided sign-flip test over all 282^8 sign assignments gives p=0.0078p = 0.0078 for a negative suite-level mean.

4.3 Harm Attenuates with Setting and Capability

Larger frontier, longer horizon (250 configs, T=20T=20): Below-floor effect decreases by 1–2 orders of magnitude:

  • ADULT: −0.245 → +0.0003
  • ELECTRICITY: −0.072 → +0.0003
  • JM1: −0.080 → −0.004

Capability ladder (Table 1): The weak 2024 8B critic is negative on 5 of 6 datasets. At 14B tier, all point estimates are within 0.012 of zero (3 of 6 remain negative). The 14B critic improves on 5 of 6 datasets relative to 8B (paired sign-flip test, p=0.047p = 0.047).

Key finding: The pattern is not monotone in size or release date—Llama-3.3-70B is negative on 5 of 6 datasets (up to −0.031). This supports task-dependent attenuation, not a capability threshold.

5.1 Fault Injection: Wide but Asymmetric Safety Envelope

Error StructureFinding
RANDOM (p=1p=1)Nearly costless: final degradation < 0.005 AUC on every dataset
WORST/MISLEADResilience threshold p⋆=0.6p^\star = 0.6–0.90.9 on primary binary datasets (LETTER at 1.0)

5.2 Theoretical Results

Proposition 1 (Uniform corruption preserves the random-search floor): If every selected configuration is uniformly distributed over remaining untried configurations, then for every p∈[0,1]p \in [0,1] and horizon TT:

E[bTuniform(p)]=E[bTno-critic]\mathbb{E}[b^{\text{uniform}(p)}_T] = \mathbb{E}[b^{\text{no-critic}}_T]

This follows from without-replacement exchangeability.

Proposition 2 (Resilience under adversarial corruption): Let K∼Binomial(T,1−p)K \sim \text{Binomial}(T, 1-p) be the number of effective rounds. Under Assumption 1 (adversarial rounds never raise the incumbent), the loop is (δ+ϵ)(\delta + \epsilon)-robust when Pr⁡[Binomial(T,1−p)<Tsat]≤ϵ\Pr[\text{Binomial}(T, 1-p) < T_{\text{sat}}] \leq \epsilon. The approximation:

p⋆≈1−TsatTp^\star \approx 1 - \frac{T_{\text{sat}}}{T}

With T=8T=8, Tsat∈[1,3]T_{\text{sat}} \in [1,3] predicts p⋆∈[0.63,0.88]p^\star \in [0.63, 0.88], matching measured 0.6–0.9.

5.5 Net-Benefit Criterion

Lemma 5.1 (Conditional effective sample size): Conditioned on KcK_c competent and KuK_u uniform rounds, the final incumbent has the same distribution as the maximum of Neff=mKc+KuN_{\text{eff}} = mK_c + K_u i.i.d. score draws.

Proposition 3 (Effective-horizon criterion): The critic can be net-beneficial only if:

q(m−1)>βq(m-1) > \beta

equivalently Teff>TT_{\text{eff}} > T. In the rare-improvement regime, the per-round advantage is ϵ[q(m−1)−β]+O(ϵ2)\epsilon[q(m-1) - \beta] + O(\epsilon^2) as ϵ→0\epsilon \to 0.

Important caveat: The condition is necessary, not sufficient. With Bernoulli(1/21/2) scores, q=0.3q=0.3, m=3m=3, β=0.3\beta=0.3: q(m−1)=0.6>βq(m-1) = 0.6 > \beta, yet a critic round reaches the top score with probability 0.4625 < 0.5—worse at every budget TT.

5.6 Cross-Domain Test of Mismatch Hypothesis

SettingΔ\Delta95% CIVerdict
Tabular AutoML (96)−0.245[−0.27, −0.22]harm
Credit scoring (96)+0.000[−0.001, +0.002]neutral
20NG text-pipe (36)−0.003[−0.008, +0.001]neutral
20NG weak→strong (50)+0.003[−0.005, +0.010]neutral
M4 forecasting (13)+0.001[+0.001, +0.002]helps
RAG retrieval (54)+0.007[+0.006, +0.008]helps

Holding the critic fixed, model size cannot explain this variation—consistent with the prior–reality mismatch hypothesis.

6.1 Prevention Motivation

  • The failure reproduces in the published CAAFE system: negative on 3 datasets (ADULT −0.030, BANK-MARKETING −0.054, CREDIT-G −0.090), mean −0.021.
  • Prompting does not reliably remove harm: Chain-of-thought lifts ADULT from −0.278 to −0.041, yet 3 of 4 datasets stay below floor. Five-sample self-consistency spends 5× generations while doing worse than plain chain-of-thought.

6.6 End-to-End Prevention Results

Across six datasets:

  • Raw policy: −0.00541 AUC relative to uniform
  • Simple non-improvement gate: +0.00046
  • Learned detector: −0.00015

The simple gate significantly improves over raw on bank-marketing and kc1; the detector on bank-marketing and electricity. Neither guarantees floor preservation on every dataset.

CRITFAULT Benchmark

~3,840 labeled loop transitions across 12 datasets. Leave-one-dataset-out AUC = 0.95; leave-one-model-out mean = 0.988. Fully deployable f0–f2 detector (evaluated utility, incumbent-relative change, round fraction) attains 0.982 on six datasets.


Theoretical and Practical Implications

Theoretical Implications

  1. Critic safety is a measurable property: The four-arm audit provides a rigorous protocol for isolating a critic's marginal effect, controlling for search randomness, training noise, and pool differences.

  2. Error structure, not error rate, determines harm: The elitist best-so-far dynamics create an asymmetric safety envelope—random errors are nearly free, but systematically adversarial ones accumulate cost. This differs fundamentally from symmetric flip-rate thresholds in robust preference optimization (rDPO's ϵ<1/2\epsilon < 1/2).

  3. The necessary condition q(m−1)>βq(m-1) > \beta organizes the design space:

    • Stronger critic → raises qq, mm
    • Verification → lowers β\beta
    • Breadth → raises mm

    It also predicts that parallel kk-wide search is a partial structural defense (falls below floor on only 1/12 datasets vs. 9/12 for sequential, tree-search, and multi-agent backbones).

Practical Implications

  1. Audit before trusting: Practitioners should run a critic against a no-critic control at the intended budget before deployment.

  2. Asymmetric risk: On small, fixed frontiers, the upside of guidance is bounded (~0.005 AUC) while the downside is not (up to 0.245 AUC)—roughly 6× the largest reference gain typically, up to 50× in the worst case.

  3. Gate or omit: When critic capability or verifier reliability is insufficient, gate the critique (VGC) or omit it entirely.

  4. High-stakes domains: Tabular ML is the production default in credit scoring, fraud detection, and clinical risk. Even a ~0.03 AUC drop can shift a non-trivial slice of accept/reject boundaries.


Conclusion

Main takeaways:

  • Adding an LLM critic to an iterative search loop is not guaranteed to be harmless—a naive 8B critic can drive search below the no-critic random-search floor on 7 of 8 datasets.
  • When: Harm attenuates with critic capability and varies with frontier and budget, without a universal capability threshold.
  • Why: Best-so-far search tolerates non-directional errors but not directional ones; the idealized necessary condition is q(m−1)>βq(m-1) > \beta.
  • How: Observable gates targeting β\beta reduce average loss in deployable reruns, without guaranteeing the no-critic baseline on every dataset.
  • Cross-domain audits are consistent with the hypothesis that harm tracks mismatch between the critic's priors and the task.

Future directions:

  • Extending the quantitative boundary q(m−1)>βq(m-1) > \beta to self-refinement, verifier-guided decoding, or preference-based optimization requires setting-specific assumptions.
  • Directly identifying (q,m,β)(q, m, \beta) for real critics remains open.
  • The authors will release CRITFAULT and all artifacts upon publication—the first benchmark for detecting harmful critic feedback in tabular AutoML.

Related papers