Summary (Overview)

• SciReasoner is a multimodal scientific foundation model that performs native structural reasoning across proteins, small molecules, and inorganic crystals, using a unified structure-aware vocabulary.
• It discretizes 3D coordinates, topologies, and periodic connectivities into addressable evidence units (structural tokens) that are interleaved with language in autoregressive reasoning trajectories.
• In homology-controlled Gene Ontology prediction, it improves Cellular Component F_max from 0.42 to 0.55 for low-homology proteins. In retrosynthesis, it raises single-step accuracy from 0.63 to 0.72.
• Across 86 benchmarks, SciReasoner achieves state-of-the-art performance on 67 tasks, and its reasoning traces are preferred or tied with a frontier LLM in 98% of double-blind expert evaluations.
• The model demonstrates that structural information is not a peripheral input but a necessary substrate for scientific inference, with explicit evidential grounding in its reasoning traces.

Introduction and Theoretical Foundation

Structure–property relationships are foundational to biology, chemistry, and materials science. In proteins, structure and active-site geometry govern biological function; in small molecules, bonding and stereochemistry determine reactivity; in crystalline materials, lattice symmetry and bonding networks dictate electronic and thermodynamic properties. Mechanistically explaining these relationships requires interpreting structural evidence through scientific principles and physical constraints.

Current AI systems face a critical gap:

• Large language models (LLMs) compress structural organization into text, leading to explanations based on linguistic associations rather than directly addressable physical evidence.
• Domain-specific models (e.g., graph neural networks) output scores or labels without exposing intermediate evidence.
• Agentic systems are constrained by the structural competence of the foundation models they orchestrate.

The paper proposes native structural reasoning as a new paradigm, where structural tokens are addressable evidence units that can be combined, cited, and checked within a generated reasoning trajectory.

Methodology

Model Architecture

SciReasoner is a unified causal language model (initialized from Qwen3-14B) with three main components:

1. Offline Structure Encoders: Domain-specific discrete encoders convert raw structures into structure-aware tokens:

   • Foldseek for protein 3D structures (3Di tokens)
   • ConfSeq for small molecule 3D conformers
   • SLICES for periodic crystal structures

2. Structure-Aware Vocabulary Embedding: A dedicated learnable embedding matrix $W_v \in \mathbb{R}^{|V_v| \times d_{\text{LLM}}}$ projects structural tokens directly into the LLM's latent space:

   $$H_v = \text{Embedding}(X_v, W_v) \in \mathbb{R}^{L_v \times d_{\text{LLM}}}$$

3. Unified Autoregressive Generation: Structural embeddings $H_v$ and language embeddings $H_q$ are concatenated and processed by the LLM backbone $f_\phi$ to generate responses conditioned on both structural evidence and textual instruction.

Training Pipeline

Pre-training (3 stages):

• Stage 1 (Warm-up): Freeze backbone, train only structure-aware vocabulary, text embeddings, and prediction head.
• Stage 2 (Full-parameter): Unfreeze all parameters, train on diverse structure-text datasets with a Warmup-Stable-Decay (WSD) scheduler.
• Stage 3 (Annealing): Increase proportion of QA-style data while continuing full-parameter training.

The training objective is next-token prediction:

$$\mathcal{L}_{\text{NTP}} = -\sum_{t=1}^T \log P_\phi(x_{a,t} | x_{a,<t}, H_v, H_q)$$

Post-training (Self-Bootstrapped Native Structural Reasoning):

1. Intra-domain structural evidence grounding: Partition tasks by domain, coldstart specialized experts via supervised fine-tuning (SFT) and reinforcement learning (RL) using DAPO.
2. Cross-domain reasoning consolidation: Use expert-generated traces to coldstart and reinforce a unified model across all tasks.

The RL objective uses a clipped PPO-style formulation:

$$J_{\text{DAPO}}(\theta) = \mathbb{E}_{(q,a) \sim \mathcal{D}, \{o_i\}_{i=1}^G \sim \pi_{\theta_{\text{old}}}(\cdot|q)} \left[ \frac{1}{\sum_{i=1}^G |o_i|} \sum_{i=1}^G \sum_{t=1}^{|o_i|} \min\left(r_{i,t}(\theta)\hat{A}_{i,t}, \text{clip}(r_{i,t}(\theta), 1-\varepsilon_{\text{low}}, 1+\varepsilon_{\text{high}})\hat{A}_{i,t}\right) \right]$$

Empirical Validation / Results

Key Findings

1. Protein GO Term Prediction

• Overall F_max = 0.59, outperforming BLAST (0.55), Foldseek (0.54), ESM2 (0.53), SaProt (0.52), and general LLMs.
• Largest gains in low-homology regime: For Cellular Component at ≤30% identity, SciReasoner achieves F_max = 0.55 vs. BLAST (0.34) and ESM2 (0.42).
• Attention analysis: In DNA-binding proteins, high-attention residues localize to protein-DNA interfaces (AUROC up to 0.91, enrichment up to 4.2×).
• Reasoning quality: LLM-as-judge scores of 8.33/10 vs. 6.96 for DeepSeek-V4-Pro.

2. Retrosynthesis (USPTO-50K)

• Exact Match accuracy = 0.72, surpassing RSGPT (0.63) and Opus-4.7 (0.48).
• Chain-of-thought traces follow a canonical workflow: Analysis → Disconnection → Verification → Feasibility.
• Recovers correct reactants in 5/5 representative cases vs. 2/5 for RSGPT and Opus-4.7.

3. 3D Molecular Similarity (DUD-E)

• AUC = 0.76 (matching best prior) and 5.0% EF = 7.70 (new best).
• Embedding clusters actives by binding-pocket geometry rather than 2D scaffold.

4. Materials Science

• Outperforms CGCNN across all 10 tasks and exceeds LLM-Prop on most numerical properties.
• Formation energy R² = 0.895, band gap R² = 0.785.
• Latent space separates C, Si, SiC into compositional clusters with smooth band-gap gradients.

Ablation Studies

Removing structural inputs consistently degrades performance, especially for:

• Protein tasks (folded geometry, binding sites)
• Materials tasks (periodic topology)
• Molecular similarity tasks (3D shape)

Specialist Comparison (Table A1)

Human Expert Evaluation

• Double-blind comparison of SciReasoner vs. DeepSeek-V4-Pro across 3 tasks.
• 98% of cases rated SciReasoner as tie-or-better (73% strongly prefer, 21% prefer, 4% tie).
• Per-axis scores: SciReasoner averages 8.7/10 vs. 4.3/10 for DeepSeek-V4-Pro.
• All per-axis differences significant (Wilcoxon signed-rank, p < 0.001).

Theoretical and Practical Implications

Theoretical Contributions

1. Native structural reasoning establishes a new paradigm for scientific AI, where structural tokens serve as inspectable, addressable evidence rather than opaque input descriptors.
2. The model demonstrates that structure-grounded reasoning generalizes across diverse scientific domains (proteins, small molecules, crystals) within a single architecture.
3. The self-bootstrapped post-training framework shows how domain-specific reasoning patterns can be consolidated into unified capabilities without destructive interference.

Practical Implications

1. Drug discovery: Accurate retrosynthesis planning, virtual screening, and scaffold hopping with explicit chemical reasoning.
2. Functional annotation: Improved GO term prediction for low-homology and orphan proteins, enabling functional characterization of previously unannotated sequences.
3. Materials design: Reliable prediction of formation energies, band gaps, and pore geometries with interpretable structural evidence.
4. Scientific auditing: The inspectable chain-of-thought allows domain experts to verify reasoning against physical structures, reducing reliance on black-box predictions.

Conclusion

SciReasoner demonstrates that a single autoregressive foundation model can unify sequence, structure, and natural-language reasoning across proteins, small molecules, and inorganic crystals while achieving specialist-level accuracy and interpretable scientific explanation.

The central premise is that structure–property relationships require structures to be represented as primary objects of inference, not as text strings, low-dimensional descriptors, or black-box inputs. SciReasoner's design—discretizing coordinates, topologies, and connectivities into a unified structure-aware vocabulary and integrating them with language—enables residues, molecular fragments, and crystal descriptors to function as addressable evidence units within generated reasoning trajectories.

Future directions include:

• Extending the approach to additional scientific modalities (e.g., nucleic acid structures, complex materials interfaces).
• Scaling the model to larger backbone architectures and more diverse training data.
• Deploying SciReasoner in interactive scientific discovery workflows where human experts can query and probe the model's structural reasoning.