# NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness > NeoHorse-1 shows a deployed routing harness can operationalize recursive self-improvement, converting execution trajectories and routing signals into training data that lifts 4B and 9B models across ten benchmarks. - **Source:** [arXiv](https://arxiv.org/abs/2609.08183) - **Published:** 2026-09-10 - **Permalink:** https://picx.dev/p/nNG7IX - **Whiteboard:** https://picx.dev/p/nNG7IX/image ## Summary # NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness ## Summary (Overview) - **Core contribution**: NeoHorse-1 introduces a family of agent-native models (4B and 9B scales) that demonstrate how a deployed routing harness can serve as a concrete mechanism for recursive self-improvement (RSI), where execution trajectories, routing signals, and recorded outcomes are converted into the next round of training data. - **Key mechanism**: The system couples a heterogeneous model pool with intelligent routing that records capability demand predictions, service tier selections, and interaction outcomes for every turn, which are then transformed into user-turn training examples preserving interleaved reasoning, tool calls, and harness context - **Methodological innovations**: Routing-guided curriculum learning organizes SFT into a three-stage progression based on capability demand scores, extended to routing-guided on-policy distillation(OPD) where a teacher supervises student-generated responses under the same staged progression - **Quantitative results**: Post-training lifts macro-average scores from 58.94 to 64.87 for the 4B model and from 65.60 to 69.04 for the 9B model across ten benchmarks, substantially narrowing the gap between the post-trained 4B model and the 9B base model - **RSI validation**: The post-trained 4B model matches or exceeds Qwen3.5-9B on several benchmarks, providing preliminary evidence that the evaluation–selection–update loop can be operationalized across model generations ## Introduction and Theoretical Foundation The paper addresses recursive self-improvement(RSI), the vision in which AI systems take a growing part in their own improvement process. The authors argue that RSI requires a **concrete mechanism** through which a system observes its own capabilities and converts that evidence into the next round of learning. **Key theoretical position**: A deployed routing harness already contains such a mechanism. Beyond producing task outputs, agentic interaction leaves: - **Execution trajectories** that ground training in real interaction - **Routing signals** that characterize capability demand - **Recorded outcomes** that reveal where the model still falls short The work builds on several established research threads: - **Agentic model post-training**: Trajectory-based SFT (FireAct, AgentTuning, Agent-FLAN, AgentBank), on-policy distillation [1], and harness design research (SWE-agent, Terminal-Lego) - **LLM routing**: FrugalGPT, RouteLLM, and Agentic Routing [33],which extends routing to multi-agent LLM systems where a decision layer selects models based on requests - **Curriculum learning**: Training strategies presenting examples according to estimated difficulty, with routing systems providing alternative difficulty signals The central insight is that the **prediction–action–outcome separation** [33] in routing records enables the harness to function both as a serving-time decision mechanism and as a source of data feedback within harness-mediated RSI ## Methodology ### Data Pipeline (Section 3) The corpus is organized at three linked granularities: | Granularity | Definition | Purpose | |---|---|---| | **Trajectory** | Complete interaction executed by deployment harness | Preserves full execution history | | **User turn** | User request through next user request or termination | Basic serialized training unit | | **Subscene** | Adjacent user turns sharing a local goal | Unit for semantic characterization | **Quality control** involves three stages: 1. **Deduplication and decontamination**: Exact and near-duplicate removal; evaluation overlap screening 2.. **Structural validation**: Rule-based verification of payload readability, causal event order, tool-call/result closure, producing three outcomes: internally complete, partially recoverable, quarantined 3.. **Semantic evaluation**: Six independent quality dimensions—goal attainment, instruction adherence, tool use, evidence consistency, error recovery, termination—each assigned PASS, WARN, FAIL, or NOT_EVALUATED **Data characterization** uses three axes at the subscene level: - **Scene**: What the user is doing and context (task type, application domain, use context, asking/doing) - **Goal**: What the user expects to achieve (acceptance criteria, cross-turn relations: new, continued, modified, resumed) - **Outcome**: Verifiable result of the attempt ### Agentic Routing Signals The harness router estimates capability demand from current request, recent dialogue, previous routing decisions, and available execution state, assigning each turn to one of four service tiers: $$C0 \text{ (bounded low-risk)}, \quad C1 \text{ (general-purpose default)}, \quad C2 \text{ (multi-step reasoning/execution)}, \quad C3 \text{ (maximum capability)}$$ For each turn, the corpus retains: the router's raw prediction, the policy-adjusted decision, and the tier actually served ### Agentic Post-Training (Section 4 **Agentic supervision**: User turns are serialized using the Qwen3.5 chat template, retaining reasoning within the current turn while omitting reasoning from earlier turns. The SFT objective uses a binary token-level loss mask $m_{i,t}$: $$ \mathcal{L}_{\mathrm{SFT}}(\theta; \mathcal{B}) = - \frac{\sum_{i \in \mathcal{B}} \sum_{t=2}^{T_{i}} m_{i,t} \log p_{\theta}(x_{i,t} \mid x_{i,