Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems

Summary

• AgentX-Model is a novel dual-agent framework for industrial recommender system research that enables long-horizon, autonomous model experimentation by connecting proposal development with sandboxed experimentation.
• The system employs a Research Agent (which develops proposals from papers and experimental findings) and a Model Agent (which conducts multi-round investigations and returns code, measurements, and unresolved questions).
• Research is organized around four actions: Reproduce, Follow-up, Composition, and Diagnose, enabling iterative model improvement across experiment lineages.
• In production evaluation, 560 of 636 completed model-changing experiments recorded AUC above their business baselines, with five latest online A/B tests showing gains of 10–15% in acquisition efficiency, 15–20% in target-segment advertising spend, and 0.3–0.8% in watch time.
• The watch-time model achieved gains while using ~10% fewer FLOPs and parameters, and a dependency-aware historical-replay benchmark showed no consistent efficiency gain from more complex scheduling.

---

Introduction and Theoretical Foundation

The paper addresses a fundamental challenge in industrial recommendation research: sustaining research progress requires using the results of one experiment to decide what to investigate next. Unlike isolated model training, industrial recommender systems demand continuous, cumulative research where findings from one experiment inform subsequent investigations.

The motivation stems from the limitations of existing approaches:

• Human-driven research is slow and difficult to scale across the many business settings and prediction tasks in a production recommender system.
• Existing autoML and search-based methods typically optimize within a fixed search space but do not formulate new research questions or build on experimental findings over extended horizons.
• Prior AgentX work established a framework for online experimentation but lacked a dedicated mechanism for model research — the iterative process of proposing, testing, and refining model ideas.

The key theoretical insight is that model research in production settings operates within sandboxes defined by business inputs and prediction tasks. This constraint creates a structured environment where autonomous agents can operate safely while still exploring a rich space of model improvements.

The authors propose a dual-agent architecture to separate two distinct research horizons:

1. Research Agent: Handles the long horizon — developing proposals, synthesizing findings, and deciding what to investigate next.
2. Model Agent: Handles the short horizon — conducting concrete experiments, returning code and measurements.

This separation mirrors how human research teams divide responsibilities between senior researchers (who set direction) and engineers (who run experiments).

---

Methodology

2.1 Research Sandbox Definition

Each research effort is confined to a sandbox defined by:

• A specific business setting (e.g., a recommendation surface)
• A set of prediction tasks (e.g., watch-time prediction, ad conversion)
• Predefined business baselines for comparison

This structure ensures that all experiments are evaluated against meaningful production metrics while allowing safe autonomous exploration.

2.2 Dual-Agent Architecture

Research Agent responsibilities:

• Reads papers and prior experimental findings
• Develops independently reviewed proposals
• Selects starting implementations
• Formulates the next research question based on returned results

Model Agent responsibilities:

• Executes multi-round investigations within the sandbox
• Returns code, measurements (e.g., AUC, PCOC), and unresolved questions
• Implements the concrete experimental changes

2.3 Four Research Actions

The research cycle is organized around four actions:

The first three actions drive routine research, while Diagnose acquires the evidence needed to address issues raised by business feedback and online evaluation — such as prediction bias measured by PCOC (Predicted Click vs. Observed Click ratio).

2.4 Continuing from Experimental Evidence

A critical design feature is the feedback loop: the Research Agent uses returned experimental results to:

1. Select a starting implementation for the next experiment
2. Formulate the next research question

This enables lineages of experiments where each new experiment builds on the findings of its ancestors, creating cumulative research progress rather than isolated trials.

2.5 Selection of the Next Investigation

The Research Agent employs a dependency-aware approach to select which experiment to run next, considering the lineage relationships between candidate experiments and the information value of each potential investigation.

---

Empirical Validation / Results

4.1 Experimental Setting

The system was evaluated in production across multiple business settings in a large-scale industrial recommender system. Implementation details include integration with existing training infrastructure and online evaluation pipelines.

4.2 Key Results: Production Evaluation

Performance Metrics:

• 560 of 636 completed model-changing experiments recorded AUC above their business baselines
• Success rate of approximately 88% across all experiments

Research Continuity:

• As research continued, some experiments recorded AUC above every comparable ancestor in their lineages, demonstrating cumulative improvement

4.3 Online A/B Evaluation Results

The five latest online A/B evaluations across different business settings reported:

Efficiency finding: The watch-time model used approximately 10% fewer FLOPs and parameters while still achieving gains — indicating that the research process discovered more efficient architectures, not just larger ones.

4.4 From Ranking Gains to Calibration Repair

The paper documents a specific case where ranking gains (AUC improvements) were accompanied by calibration issues (PCOC bias). The Diagnose action was triggered by business feedback and online evaluation, leading to targeted repairs that restored proper calibration without sacrificing ranking quality.

4.5 Knowledge Transfer Across Settings

The framework demonstrated that insights and model components discovered in one business setting could be transferred to other settings, accelerating research in new sandboxes.

4.6 Benchmarking Research Allocation

A dependency-aware historical-replay benchmark was used to evaluate research allocation strategies. Initial results showed:

No consistent efficiency gain from more complex scheduling when agents already analyze and select concrete candidates.

This suggests that the bottleneck in long-horizon research is not scheduling efficiency but the quality of candidate proposal and selection.

---

Theoretical and Practical Implications

Theoretical Implications

1. Long-horizon autonomy is achievable in industrial ML research through structured sandboxes and dual-agent separation of concerns.
2. Cumulative research progress can be formalized as lineage-based experiment graphs, where each experiment's value is measured not just by its own metrics but by its contribution to subsequent discoveries.
3. The finding that complex scheduling doesn't help when candidate selection is already strong has implications for the design of autonomous research systems — the intelligence should be invested in what to try, not when to try it.

Practical Implications

1. Scalable research capacity: The framework enables far more experiments than a human team could conduct, expanding the research frontier.
2. Safe exploration: Sandboxes defined by business inputs ensure that autonomous research stays within meaningful constraints.
3. Calibration as a first-class concern: The Diagnose action explicitly handles prediction bias (PCOC), addressing a common failure mode in production recommender systems.
4. Efficiency gains: The discovery of models with ~10% fewer FLOPs and parameters demonstrates that autonomous research can find Pareto improvements, not just accuracy gains.

---

Conclusion

The paper presents AgentX-Model as a successful demonstration of long-horizon autonomous model research in an industrial recommender system. Key takeaways:

1. Dual-agent architecture works: Separating research direction (Research Agent) from experiment execution (Model Agent) enables sustained, cumulative progress.
2. Four-action research cycle is sufficient: Reproduce, Follow-up, Composition, and Diagnose cover the essential modes of model research.
3. Production-validated gains: The framework delivered consistent AUC improvements and meaningful business metric gains across diverse settings.
4. Efficiency and effectiveness are compatible: The research process discovered models that were both better and more efficient.

Future Directions

The authors identify several open questions:

• How to further improve the quality of proposal generation (the apparent bottleneck)
• How to extend the framework to broader research questions beyond model architecture
• How to better measure and optimize research allocation in dependency-aware settings
• Whether the framework's principles generalize to other domains beyond recommender systems

The work establishes that with the right structure — sandboxes, dual-agent separation, and lineage-based continuity — autonomous systems can conduct meaningful, long-horizon research that produces real-world value.