Full text not available for this paper
The Critic Can Hurt: On the Safety of Critics in LLM-Guided AutoML Search
Summary (Overview)
- Core finding: A naive 8B LLM critic can actively harm iterative search in AutoML, driving performance below the no-critic random-search floor on 7 of 8 tabular datasets, with worst-case losses up to 0.245 AUC.
- Key contribution: Introduces the concept of critic safety and a four-arm audit protocol (no-critic floor, random feedback, real critic, oracle ceiling) that isolates a critic's marginal effect on a shared cached configuration frontier.
- Mechanistic explanation: The loop's best-so-far (elitist) dynamics make it resilient to non-directional (random) errors but highly vulnerable to systematically adversarial (directional) errors. An idealized model yields the necessary condition for net benefit: (competence , lookahead , adversarial rate ).
- Prevention: A training-free validation gate and a learned detector (packaged as the CRITFAULT benchmark) can reduce realized loss, but neither guarantees preservation of the full-budget no-critic baseline on every dataset.
- Practical takeaway: A critic is not a free addition—audit it against a no-critic control at the intended budget before trusting it; gate or omit it when capability or verification is insufficient.
Introduction and Theoretical Foundation
Background and Motivation
Automated machine learning (AutoML) searches over preprocessing, feature transformations, model families, and hyperparameters. Recent agentic systems organize this as a propose–evaluate–correct loop: a proposer selects a candidate pipeline, it is scored on held-out data, and an LLM critic converts the score and search history into recommendations that condition the next proposal, while the system retains a best-so-far incumbent.
The authors argue that critics and proposers fail asymmetrically:
- A weak proposal is evaluated, scores poorly, and is discarded by the best-so-far rule (verified by construction).
- A critique receives no such check—it is neither scored nor filtered, yet it conditions every subsequent proposal.
The cost of a bad critique is not a degraded incumbent (elitist selection protects what is found), but sustained misdirection of a finite evaluation budget away from configurations unguided search would otherwise reach.
Theoretical Foundation
The paper builds on several strands of prior work:
- Classical AutoML: Bayesian/evolutionary optimizers that optimize validation metrics directly without reasoning about why a configuration failed.
- LLM-based AutoML: Systems like SELA and AIDE use LLMs primarily as proposers, assessed at outcome level—the critic role is not isolated.
- Self-critique literature: Reflexion, Self-Refine, and ReAct show verbal critique can help on coding/reasoning, but intrinsic self-correction can be net-negative (degrades accuracy once oracle labels are removed, turns correct answers incorrect).
- LLM-as-judge reliability: Documented biases and instability in LLM-as-judge setups; verifier-gated selection is an active theme.
- Noise tolerance: Robust preference optimization (rDPO) tolerates flip rates below ; this paper's resilience threshold is structurally different—it arises from elitist best-so-far dynamics rather than gradient preference training.
Methodology
3.1 Critic-Guided Search Loop
A critic-guided AutoML agent tunes a tabular pipeline over rounds. Each round:
- A proposer selects an untried configuration from a finite space .
- The configuration is trained and scored on a frozen validation split.
- An LLM critic turns the score and history into a natural-language correction.
- The agent returns the best-so-far validation incumbent , whose quality is the hidden test utility .
For dataset and condition , the expected test utility is .
3.2 Shared Cached Frontier
- Each dataset is partitioned once into train/validation/test splits.
- Every configuration in a 96-configuration pipeline space is scored before search, with validation and test AUC cached.
- All conditions share the same frozen frontier (candidate space, split, budget, outcomes), differing only in the trajectory-selecting mechanism.
- The critic acts only through the feedback channel (never trains/executes code, alters candidate space, or sees test utilities).
3.3 Four-Arm Audit
Four budget-matched conditions on the shared frontier:
| Arm | Description | Role |
|---|---|---|
| (i) No critic | Uniform sampling over untried configurations | Unguided-search floor |
| (ii) Random feedback | Independently seeded uniform random search | Second draw of floor |
| (iii) Real critic | Feedback from LLM under evaluation | Object of study |
| (iv) Oracle | Directly selects highest-validation-utility untried config | Unattainable ceiling |
Paired contrasts:
A critic is below floor when .
3.4 Controlled Fault Injection
Corrupts a clean reference policy : each round follows with probability and a corrupted choice with probability . Three error structures:
- RANDOM: Uniform untried (non-directional)
- WORST: Lowest-utility untried (globally adversarial)
- MISLEAD: Lowest-utility incumbent neighbor (locally plausible but directionally harmful)
3.5 Experimental Scope
- 12 public OpenML classification datasets (Table 12 in paper).
- In-space headroom is below 0.01 AUC on all 12 datasets.
- Primary audit: , 8 matched seeds, deterministic decoding.
- Primary critic: Llama-3.1-8B; capability ladder spans 3B–70B open-weight models and Claude Haiku 4.5.
Empirical Validation / Results
4.1 Limited Upside of Guidance
- Clean hill-climbing reference achieves +0.001 to +0.005 AUC over no-critic (30 seeds; suite-level Wilcoxon ).
- The in-space oracle matches or beats AutoGluon-Tabular on 5 of 8 datasets—so floors are SOTA-competitive.
- Asymmetric opportunity: Attainable upside is small, but downside from misleading guidance can be considerably larger.
4.2 Weak Critics Fall Below the Floor
A naive Llama-3.1-8B critic (temperature 0) has a negative No-Critic delta on 7 of 8 binary datasets, ranging from −0.009 AUC (PHONEME) to −0.245 (ADULT). Only CREDIT-G is positive (+0.006).
An exact one-sided sign-flip test over all sign assignments gives for a negative suite-level mean.
4.3 Harm Attenuates with Setting and Capability
Larger frontier, longer horizon (250 configs, ): Below-floor effect decreases by 1–2 orders of magnitude:
- ADULT: −0.245 → +0.0003
- ELECTRICITY: −0.072 → +0.0003
- JM1: −0.080 → −0.004
Capability ladder (Table 1): The weak 2024 8B critic is negative on 5 of 6 datasets. At 14B tier, all point estimates are within 0.012 of zero (3 of 6 remain negative). The 14B critic improves on 5 of 6 datasets relative to 8B (paired sign-flip test, ).
Key finding: The pattern is not monotone in size or release date—Llama-3.3-70B is negative on 5 of 6 datasets (up to −0.031). This supports task-dependent attenuation, not a capability threshold.
5.1 Fault Injection: Wide but Asymmetric Safety Envelope
| Error Structure | Finding |
|---|---|
| RANDOM () | Nearly costless: final degradation < 0.005 AUC on every dataset |
| WORST/MISLEAD | Resilience threshold – on primary binary datasets (LETTER at 1.0) |
5.2 Theoretical Results
Proposition 1 (Uniform corruption preserves the random-search floor): If every selected configuration is uniformly distributed over remaining untried configurations, then for every and horizon :
This follows from without-replacement exchangeability.
Proposition 2 (Resilience under adversarial corruption): Let be the number of effective rounds. Under Assumption 1 (adversarial rounds never raise the incumbent), the loop is -robust when . The approximation:
With , predicts , matching measured 0.6–0.9.
5.5 Net-Benefit Criterion
Lemma 5.1 (Conditional effective sample size): Conditioned on competent and uniform rounds, the final incumbent has the same distribution as the maximum of i.i.d. score draws.
Proposition 3 (Effective-horizon criterion): The critic can be net-beneficial only if:
equivalently . In the rare-improvement regime, the per-round advantage is as .
Important caveat: The condition is necessary, not sufficient. With Bernoulli() scores, , , : , yet a critic round reaches the top score with probability 0.4625 < 0.5—worse at every budget .
5.6 Cross-Domain Test of Mismatch Hypothesis
| Setting | 95% CI | Verdict | |
|---|---|---|---|
| Tabular AutoML (96) | −0.245 | [−0.27, −0.22] | harm |
| Credit scoring (96) | +0.000 | [−0.001, +0.002] | neutral |
| 20NG text-pipe (36) | −0.003 | [−0.008, +0.001] | neutral |
| 20NG weak→strong (50) | +0.003 | [−0.005, +0.010] | neutral |
| M4 forecasting (13) | +0.001 | [+0.001, +0.002] | helps |
| RAG retrieval (54) | +0.007 | [+0.006, +0.008] | helps |
Holding the critic fixed, model size cannot explain this variation—consistent with the prior–reality mismatch hypothesis.
6.1 Prevention Motivation
- The failure reproduces in the published CAAFE system: negative on 3 datasets (ADULT −0.030, BANK-MARKETING −0.054, CREDIT-G −0.090), mean −0.021.
- Prompting does not reliably remove harm: Chain-of-thought lifts ADULT from −0.278 to −0.041, yet 3 of 4 datasets stay below floor. Five-sample self-consistency spends 5× generations while doing worse than plain chain-of-thought.
6.6 End-to-End Prevention Results
Across six datasets:
- Raw policy: −0.00541 AUC relative to uniform
- Simple non-improvement gate: +0.00046
- Learned detector: −0.00015
The simple gate significantly improves over raw on bank-marketing and kc1; the detector on bank-marketing and electricity. Neither guarantees floor preservation on every dataset.
CRITFAULT Benchmark
~3,840 labeled loop transitions across 12 datasets. Leave-one-dataset-out AUC = 0.95; leave-one-model-out mean = 0.988. Fully deployable f0–f2 detector (evaluated utility, incumbent-relative change, round fraction) attains 0.982 on six datasets.
Theoretical and Practical Implications
Theoretical Implications
-
Critic safety is a measurable property: The four-arm audit provides a rigorous protocol for isolating a critic's marginal effect, controlling for search randomness, training noise, and pool differences.
-
Error structure, not error rate, determines harm: The elitist best-so-far dynamics create an asymmetric safety envelope—random errors are nearly free, but systematically adversarial ones accumulate cost. This differs fundamentally from symmetric flip-rate thresholds in robust preference optimization (rDPO's ).
-
The necessary condition organizes the design space:
- Stronger critic → raises ,
- Verification → lowers
- Breadth → raises
It also predicts that parallel -wide search is a partial structural defense (falls below floor on only 1/12 datasets vs. 9/12 for sequential, tree-search, and multi-agent backbones).
Practical Implications
-
Audit before trusting: Practitioners should run a critic against a no-critic control at the intended budget before deployment.
-
Asymmetric risk: On small, fixed frontiers, the upside of guidance is bounded (~0.005 AUC) while the downside is not (up to 0.245 AUC)—roughly 6× the largest reference gain typically, up to 50× in the worst case.
-
Gate or omit: When critic capability or verifier reliability is insufficient, gate the critique (VGC) or omit it entirely.
-
High-stakes domains: Tabular ML is the production default in credit scoring, fraud detection, and clinical risk. Even a ~0.03 AUC drop can shift a non-trivial slice of accept/reject boundaries.
Conclusion
Main takeaways:
- Adding an LLM critic to an iterative search loop is not guaranteed to be harmless—a naive 8B critic can drive search below the no-critic random-search floor on 7 of 8 datasets.
- When: Harm attenuates with critic capability and varies with frontier and budget, without a universal capability threshold.
- Why: Best-so-far search tolerates non-directional errors but not directional ones; the idealized necessary condition is .
- How: Observable gates targeting reduce average loss in deployable reruns, without guaranteeing the no-critic baseline on every dataset.
- Cross-domain audits are consistent with the hypothesis that harm tracks mismatch between the critic's priors and the task.
Future directions:
- Extending the quantitative boundary to self-refinement, verifier-guided decoding, or preference-based optimization requires setting-specific assumptions.
- Directly identifying for real critics remains open.
- The authors will release CRITFAULT and all artifacts upon publication—the first benchmark for detecting harmful critic feedback in tabular AutoML.
Related papers
- UpLoop: Training Looped Transformers via Upcycling at No Extra Pretraining Compute
UpLoop trains looped Transformers with identical parameters, tokens, and FLOPs to dense baselines, improving reasoning and language modeling across scales via progressive depth growth and shared-weight looping.
- GUARD: Stabilizing Recursive Self-Improvement with Regret Guidance
GUARD achieves non-regressive agent harness self-improvement without external graders by separating edit-induced regressions from capability gaps via paired regret, guaranteeing monotonic accuracy gains.
- Inside the Loop: A Mechanistic Study of Looped Transformers on Permutation Traversal
Looped transformers solve sequential algorithmic tasks untied models of equal effective depth cannot, by executing one algorithm step per iteration without intermediate supervision.