Summary (Overview)
- Xing 4 is a series of large language models extending the TeleChat lineage, trained entirely on domestic Ascend NPUs using the MindSpore framework, spanning tens to hundreds of billions of parameters with expansion toward trillion-parameter scale.
- Architectural innovations: The series combines Multi-head Latent Attention (MLA) with DeepSeek Sparse Attention (DSA) and cross-layer IndexShare for long-context serving, adopts a wider-and-shallower mixture-of-experts (MoE) topology, and pairs a block-wise draft model with lightweight correction heads for speculative decoding.
- Systems engineering: The pretraining infrastructure builds on MindSpore HyperParallel, combining topology-aware hybrid sharded data parallelism (HSDP), owner-based distributed Muon optimizer execution, and coordinated management of long-sequence workloads and activation memory, supporting pretraining of a 285B-parameter model on 8,192 Ascend 910C cards.
- Performance gains: Measurements show up to 4.68× prefill throughput over an MLA-only baseline, 2.67×–3.77× decoding throughput from speculative decoding, and a 16.1% throughput improvement from eliminating pipeline parallelism.
- Post-training framework: The paper introduces TeleGym, a unified agent execution platform connecting execution, evaluation, and trajectory construction to policy optimization, with disaggregated asynchronous execution and partial rollout to handle heterogeneous task durations.
Introduction and Theoretical Foundation
Background and Motivation
The TeleChat family has evolved through successive advances in model capability and training infrastructure:
- TeleChat established an open foundation spanning pretraining, instruction tuning, and preference alignment
- TeleChat2, TeleChat2.5, and T1 expanded with stronger reasoning, coding, and long-context capabilities
- TeleChat3-MoE extended to models with hundreds of billions to over a trillion parameters
Xing 4 continues this lineage with a focus on long-context reasoning, code generation, and agentic applications while advancing efficient training on Ascend. The open-sourced Xing4.0-29B-A4B (29B total parameters, 4B activated per token, 256K-token context support) combines:
- Manifold-Constrained Hyper-Connections (mHC)
- Multi-head Latent Attention (MLA)
- Multi-token prediction (MTP)
Key Design Challenges
The paper identifies two primary bottlenecks:
-
Long-context inference bottleneck: MLA reduces KV Cache size, but in ultra-long-context scenarios, the main attention still needs to access a large number of historical KV states, causing decode latency to increase with context length.
-
Ascend hardware efficiency bottleneck: Overly deep networks amplify non-matrix-computation overheads (normalization, routing, indexing, scheduling, data movement), reducing effective Cube utilization on Ascend 910C.
Hardware Context
The Ascend 910C provides strong Cube matrix-computation capability, while Vector compute and HBM bandwidth are relatively more constrained. This guides architecture choices toward:
- Leveraging Cube with matrix-dominant computation
- Mitigating Vector bottlenecks through structural optimization
- Reducing HBM pressure through sparse memory access
Methodology
1. Long-Context Attention Design
The paper studies three attention variants for the 285B model:
- MLA (baseline): Compresses KV representations
- MLA + DSA: DSA uses a lightning indexer to select Top-K historical KV states, so the main attention only attends to a sparse subset of historical context
- MLA + DSA + IndexShare: IndexShare reuses sparse indices across layers, reducing repeated indexer computation
Inference evaluation setup (Table 1):
| Item | Configuration |
|---|---|
| Inference environment | 32 Ascend 910C cards |
| Deployment mode | P/D disaggregation |
| Prefill-side parallelism | TP4DP4 |
| Decode-side parallelism | TP2DP8 |
| Evaluation context length | 256K |
2. Depth-Width Trade-off Study
Two candidate topologies were compared for the 285B model:
| Configuration | Layers | Hidden Size | Heads | Routed Experts | Expert Intermediate Size | Experts per Token | Shared Experts |
|---|---|---|---|---|---|---|---|
| Baseline | 60 | 4096 | 128 | 256 | 1536 | 6 | 1 |
| Wider-shallower | 45 | 4096 | 128 | 256 | 2048 | 8 | 1 |
3. Speculative Decoding with DSpark
DSpark combines:
- A dedicated block-wise draft model (five-layer DFlash backbone computing seven speculative positions in parallel)
- A lightweight Markov head (captures dependencies between adjacent proposed tokens)
- An acceptance confidence head (estimates per-position acceptance probabilities)
4. Training Infrastructure (HyperParallel)
Hybrid Sharded Data Parallelism (HSDP):
- Parameters FSDP-sharded within each SuperPoD, replicated across SuperPoDs
- Frequent weight-gathering confined to high-bandwidth intra-SuperPoD domain
- Only coarse-grained gradient reduction crosses SuperPoD boundaries
- Zero-copy communication: Per-parameter AllGather/ReduceScatter eliminates flattened buffer overhead
- Three-way communication overlap: ReduceScatter, AllGather, and inter-SuperPoD AllReduce overlap with backward computation
Owner-Based Muon Execution:
- Each matrix assigned a single owner rank within its shard group
- Non-owner ranks send input shards to the owner via point-to-point transfers
- Owner assembles full input, runs Newton-Schulz iteration, and scatters update slices back
Fine-Grained Selective Recomputation:
- Object-based exclusion: operators assigned as object members during initialization
- Memory-budgeted selection formulated as a 0–1 knapsack problem:
Where is the forward execution time of operator , is its output-memory cost, and is the activation-memory budget.
DSA Long-Sequence Optimizations:
- Layer-wise indexer backward: Precomputed gradients from fused KL operator allow early backward propagation
- Q-block pairing: Head-tail complementary pairing of causal workload blocks; dynamic cost-based pairing replaces fixed folding when predicted peak decreases
- Document-aware KV compact: For Q block , only KV from is needed rather than
5. Post-Training Framework (TeleGym)
Three cooperating layers:
- TeleGym: Shared execution layer managing task environments and agent sessions
- Intermediate server: Connects model requests to policy service, converts interactions into training trajectories
- Slime: Coordinates rollout scheduling and policy optimization
Key mechanisms:
- Disaggregated asynchronous execution: Generation and policy optimization on separate device pools
- Bounded oversampling: Finite lookahead window limits task production beyond training consumption
- Partial rollout: Unfinished trajectories resume after policy updates, preserving sampling-time log probabilities
6. CPU Affinity and Core Isolation
- Coarse-grained pinning: Threads in a training process share a device CPU pool
- Fine-grained pinning: Partitions the pool by thread role (PyNative, MindData, Main)
- Core isolation: Reserves a separate system CPU pool for services, logging, monitoring, and movable IRQs
Empirical Validation / Results
1. Attention Variant Results (Table 2)
Simulation-based inference efficiency of attention variants in the 285B study (normalized against MLA baseline):
| Architecture | KV HBM Access | Index Computation | Relative TTFT | Relative TPOT | Prefill Throughput | Decode Throughput |
|---|---|---|---|---|---|---|
| MLA | 1.00× | None | 1.00× | 1.00× | 1.00× | 1.00× |
| MLA + DSA | 0.85× | Per-DSA-layer | 0.37× | 0.84× | 2.67× | 1.18× |
| MLA + DSA + IndexShare | 0.80× | Cross-layer | 0.21× | 0.79× | 4.68× | 1.27× |
2. Depth-Width Study Results (Table 4)
| Configuration | Relative TTFT | Relative TPOT | Prefill Throughput | Decode Throughput | Decision |
|---|---|---|---|---|---|
| Baseline | 1.00× | 1.00× | 1.00× | 1.00× | Reference |
| Wider-shallower | 0.83× | 0.90× | 1.20× | 1.11× | Selected |
The wider-and-shallower topology also improved overall training throughput by approximately 10% compared with the deeper baseline.
3. Speculative Decoding Results (Table 5)
| Benchmark | Output Throughput (tokens/s) | MTP3/Base | DSpark/Base | DSpark/MTP3 |
|---|---|---|---|---|
| GSM8K | 35.4 / 63.7 / 115.8 | 1.80× | 3.28× | 1.82× |
| MATH-500 | 36.8 / 70.0 / 128.1 | 1.90× | 3.49× | 1.83× |
| HumanEval | 35.7 / 64.1 / 134.7 | 1.80× | 3.77× | 2.10× |
| MBPP | 37.1 / 70.0 / 132.0 | 1.89× | 3.56× | 1.89× |
| AIME25 | 37.0 / 71.5 / 109.2 | 1.93× | 2.95× | 1.53× |
| LiveCodeBench | 35.6 / 64.5 / 95.2 | 1.81× | 2.67× | 1.48× |
4. Training Performance (Table 7, 1,024-NPU profiling cluster)
| Scheme | Pipeline | DP (Replica × Shard) | mbs | Step Time | Speedup |
|---|---|---|---|---|---|
| PP-1F1B (Baseline) | pp = 4, vpp = 1 | 2 × 1 | 8 | 36.4 s | 1.00× |
| PP-Interleaved | pp = 4, vpp = 2 | 2 × 1 | 4 | 34.2 s | 1.06× |
| HSDP (Ours) | pp = 1 | 4 × 2 | 2 | 31.8 s | 1.14× |
The theoretical pipeline bubble fraction for stages and micro-batches in 1F1B scheduling is:
5. Incremental Optimization Gains (vs. HSDP + Muon baseline of 28.9s)
| Optimization | Step Time | Speedup | Step-time Reduction |
|---|---|---|---|
| HSDP + Muon (baseline) | 28.9 s | 1.00× | — |
| + Owner-based Muon | — | 1.10× | 9.1% |
| + Selective recomputation | 26.9 s | 1.07× | 6.9% |
| + Layer-wise indexer backward | 24.5 s | 1.18× | 15.2% |
6. CPU Resource Isolation Gains (Table 5-1)
| Optimization | Cluster size | Throughput gain |
|---|---|---|
| CPU resource isolation | Small (512 accelerators) | 1–2% |
| CPU resource isolation | Large (8,192 accelerators) | 5–10% |
7. Kernel Profiling (Table 9)
Compute kernels of one forward Transformer layer on rank 0 (256K-token run):
| Kernel | Duration (ms) | Output size (MiB) |
|---|---|---|
| SparseLightningIndexerGradKLLoss | 38.32 | 193.0 |
| LightningIndexer | 21.51 | 48.0 |
| AllGatherMatmul | 15.90 | 656.0 |
| SparseFlashAttention | 13.47 | 516.0 |
| MatmulReduceScatter | 6.00 | 64.0 |
Theoretical and Practical Implications
Architectural Implications
-
Sparse attention is essential for long contexts: The combination of MLA + DSA + IndexShare achieves a 4.68× prefill throughput improvement over MLA alone, demonstrating that sparse KV access is critical for ultra-long-context deployment on Ascend hardware.
-
Wider-and-shallower topology suits Ascend: The 45-layer wider-FFN configuration outperforms the 60-layer narrower-FFN baseline in both inference (1.11× decode throughput) and training (~10% throughput gain), validating the design principle that reducing layer-wise sequential overhead improves Cube utilization on Ascend 910C.
-
Speculative decoding bridges the throughput gap: DSpark achieves 2.67×–3.77× decoding throughput over the base configuration, with the largest gains on code generation benchmarks (HumanEval: 3.77×), highlighting the importance of inference-time efficiency for agentic workloads.
Systems Implications
-
Pipeline parallelism can be eliminated: HSDP with zero-copy communication and three-way overlap achieves 1.14× speedup over PP-1F1B, demonstrating that hybrid sharding in the data-parallel dimension can replace layer-wise memory partitioning.
-
Distributed optimizer optimization matters: Owner-based Muon execution removes redundant full-matrix orthogonalization, achieving 9.1% step-time reduction without changing the distributed state layout.
-
Host scheduling is a first-class concern: CPU pinning and core isolation yield 5–10% throughput gains on the 8,192-accelerator cluster, showing that host contention can significantly impact distributed training at scale.
Practical Implications
- Domestic chip viability: The successful training of a 285B-parameter model entirely on Ascend 910C cards demonstrates that domestic hardware and frameworks can support frontier-scale model development.
- Unified agent training: TeleGym provides a reusable execution substrate for data synthesis, evaluation, and reinforcement learning across heterogeneous task harnesses, enabling a common training workflow for reasoning, coding, and multi-turn agents.
Conclusion
Main Takeaways
-
Coordinated architectural and systems design is essential for scaling models on domestic chips: model capacity, sparse computation, attention, depth-width balance, and decoding must be considered jointly with compute, memory, and communication costs.
-
MindSpore HyperParallel infrastructure successfully combines topology-aware hybrid sharding, owner-based Muon execution, long-sequence workload balancing, KV compaction, and activation-memory management to support production-scale training on Ascend.
-
TeleGym provides a unified agent execution platform that connects task synthesis, environment interaction, and reward evaluation to policy optimization, with disaggregated asynchronous execution and partial rollout accommodating heterogeneous task durations.
-
The open-sourced Xing4.0-29B-A4B demonstrates the practical viability of the architectural choices, combining mHC with MLA and MTP while supporting native 256K-token contexts.
Future Directions
- Trillion-parameter training: The authors are currently training trillion-parameter models in the Xing 4 series on Ascend, extending toward larger model capacity while investigating training efficiency and system scalability at that scale.
- Longer contexts and complex agentic workloads: Further development will address the interaction between model scale, longer contexts, and increasingly complex agentic workloads.
- Continued ecosystem development: The work establishes an Ascend-oriented training foundation for the continued development of Xing 4 across model scales within the Ascend and MindSpore ecosystem.
Related papers
- VerifierBench: Benchmarking the Generation of Executable Verifiers for LLM Agents
VerifierBench shows LLM-generated verifiers reject invalid solutions reliably but accept valid ones poorly, with most failures from unjustified checks that self-testing fails to expose.
- Envisioning Agentic Robotics: A New Era of Robotic Systems
Agentic robotics replaces task-specific policies with a foundation model plus embodied harness as the decision-making core, enabling broad generalization but facing capped scaling and latency-safety tradeoffs.
- Depth as a Candidate Source: Portfolio Allocation for Recurrent Test-Time Scaling in Large Language Models
Deeper recurrent computation can lose correct answers found at shallower depths, so allocating inference budgets across multiple depths outperforms any single stopping depth.