# Accurate, Interdisciplinary and Transparent Structure-property Understanding with Deep Native Structural Reasoning

> SciReasoner unifies structural reasoning across proteins, molecules, and crystals, achieving state-of-the-art on 67 of 86 benchmarks via addressable structural tokens.

- **Source:** [arXiv](https://arxiv.org/abs/2607.07708)
- **Published:** 2026-07-10
- **Permalink:** https://picx.dev/p/0qxnse
- **Whiteboard:** https://picx.dev/p/0qxnse/image

## Summary

## Summary (Overview)

- **SciReasoner** is a multimodal scientific foundation model that performs **native structural reasoning** across proteins, small molecules, and inorganic crystals, using a unified structure-aware vocabulary.
- It discretizes 3D coordinates, topologies, and periodic connectivities into **addressable evidence units** (structural tokens) that are interleaved with language in autoregressive reasoning trajectories.
- In **homology-controlled Gene Ontology** prediction, it improves Cellular Component F_max from 0.42 to 0.55 for low-homology proteins. In **retrosynthesis**, it raises single-step accuracy from 0.63 to 0.72.
- Across **86 benchmarks**, SciReasoner achieves state-of-the-art performance on **67 tasks**, and its reasoning traces are preferred or tied with a frontier LLM in **98%** of double-blind expert evaluations.
- The model demonstrates that structural information is not a peripheral input but a **necessary substrate** for scientific inference, with explicit evidential grounding in its reasoning traces.

## Introduction and Theoretical Foundation

Structure–property relationships are foundational to biology, chemistry, and materials science. In proteins, structure and active-site geometry govern biological function; in small molecules, bonding and stereochemistry determine reactivity; in crystalline materials, lattice symmetry and bonding networks dictate electronic and thermodynamic properties. Mechanistically explaining these relationships requires interpreting structural evidence through scientific principles and physical constraints.

Current AI systems face a critical gap:
- **Large language models (LLMs)** compress structural organization into text, leading to explanations based on linguistic associations rather than directly addressable physical evidence.
- **Domain-specific models** (e.g., graph neural networks) output scores or labels without exposing intermediate evidence.
- **Agentic systems** are constrained by the structural competence of the foundation models they orchestrate.

The paper proposes **native structural reasoning** as a new paradigm, where structural tokens are **addressable evidence units** that can be combined, cited, and checked within a generated reasoning trajectory.

## Methodology

### Model Architecture
SciReasoner is a unified causal language model (initialized from Qwen3-14B) with three main components:

1. **Offline Structure Encoders**: Domain-specific discrete encoders convert raw structures into structure-aware tokens:
   - **Foldseek** for protein 3D structures (3Di tokens)
   - **ConfSeq** for small molecule 3D conformers
   - **SLICES** for periodic crystal structures

2. **Structure-Aware Vocabulary Embedding**: A dedicated learnable embedding matrix $W_v \in \mathbb{R}^{|V_v| \times d_{\text{LLM}}}$ projects structural tokens directly into the LLM's latent space:
   $$H_v = \text{Embedding}(X_v, W_v) \in \mathbb{R}^{L_v \times d_{\text{LLM}}}$$

3. **Unified Autoregressive Generation**: Structural embeddings $H_v$ and language embeddings $H_q$ are concatenated and processed by the LLM backbone $f_\phi$ to generate responses conditioned on both structural evidence and textual instruction.

### Training Pipeline
**Pre-training** (3 stages):
- **Stage 1 (Warm-up)**: Freeze backbone, train only structure-aware vocabulary, text embeddings, and prediction head.
- **Stage 2 (Full-parameter)**: Unfreeze all parameters, train on diverse structure-text datasets with a Warmup-Stable-Decay (WSD) scheduler.
- **Stage 3 (Annealing)**: Increase proportion of QA-style data while continuing full-parameter training.

The training objective is next-token prediction:
$$\mathcal{L}_{\text{NTP}} = -\sum_{t=1}^T \log P_\phi(x_{a,t} | x_{a,<t}, H_v, H_q)$$

**Post-training** (Self-Bootstrapped Native Structural Reasoning):
1. **Intra-domain structural evidence grounding**: Partition tasks by domain, coldstart specialized experts via supervised fine-tuning (SFT) and reinforcement learning (RL) using DAPO.
2. **Cross-domain reasoning consolidation**: Use expert-generated traces to coldstart and reinforce a unified model across all tasks.

The RL objective uses a clipped PPO-style formulation:
$$J_{\text{DAPO}}(\theta) = \mathbb{E}_{(q,a) \sim \mathcal{D}, \{o_i\}_{i=1}^G \sim \pi_{\theta_{\text{old}}}(\cdot|q)} \left[ \frac{1}{\sum_{i=1}^G |o_i|} \sum_{i=1}^G \sum_{t=1}^{|o_i|} \min\left(r_{i,t}(\theta)\hat{A}_{i,t}, \text{clip}(r_{i,t}(\theta), 1-\varepsilon_{\text{low}}, 1+\varepsilon_{\text{high}})\hat{A}_{i,t}\right) \right]$$

## Empirical Validation / Results

### Key Findings

#### 1. Protein GO Term Prediction
- **Overall F_max = 0.59**, outperforming BLAST (0.55), Foldseek (0.54), ESM2 (0.53), SaProt (0.52), and general LLMs.
- **Largest gains in low-homology regime**: For Cellular Component at ≤30% identity, SciReasoner achieves F_max = 0.55 vs. BLAST (0.34) and ESM2 (0.42).
- **Attention analysis**: In DNA-binding proteins, high-attention residues localize to protein-DNA interfaces (AUROC up to 0.91, enrichment up to 4.2×).
- **Reasoning quality**: LLM-as-judge scores of 8.33/10 vs. 6.96 for DeepSeek-V4-Pro.

#### 2. Retrosynthesis (USPTO-50K)
- **Exact Match accuracy = 0.72**, surpassing RSGPT (0.63) and Opus-4.7 (0.48).
- Chain-of-thought traces follow a canonical workflow: **Analysis → Disconnection → Verification → Feasibility**.
- Recovers correct reactants in 5/5 representative cases vs. 2/5 for RSGPT and Opus-4.7.

#### 3. 3D Molecular Similarity (DUD-E)
- **AUC = 0.76** (matching best prior) and **5.0% EF = 7.70** (new best).
- Embedding clusters actives by **binding-pocket geometry** rather than 2D scaffold.

#### 4. Materials Science
- Outperforms CGCNN across all 10 tasks and exceeds LLM-Prop on most numerical properties.
- **Formation energy R² = 0.895**, band gap R² = 0.785.
- Latent space separates C, Si, SiC into compositional clusters with smooth band-gap gradients.

### Ablation Studies
Removing structural inputs consistently degrades performance, especially for:
- **Protein tasks** (folded geometry, binding sites)
- **Materials tasks** (periodic topology)
- **Molecular similarity tasks** (3D shape)

### Specialist Comparison (Table A1)

| Task | Metric | Specialist | SciReasoner |
|------|--------|------------|-------------|
| Retrosynthesis USPTO-50K | Exact Match ↑ | RSGPT: 0.63 | **0.72** |
| Fluorescence | Spearman ↑ | SaprotHub: 0.70 | **0.77** |
| Isoform | R² ↑ | APARENT: 0.59 | **0.86** |
| GO-BP | F_max ↑ | SaprotHub: 0.49 | **0.52** |
| GO-CC | F_max ↑ | SaprotHub: 0.48 | **0.58** |
| GO-MF | F_max ↑ | SaprotHub: 0.67 | 0.66 |
| Subcellular localization | ACC ↑ | ESM2: 0.84 | **0.88** |
| DUD-E | 5.0% EF ↑ | ConfSeq: 7.12 | **7.70** |
| RNA protein interaction | MCC ↑ | RPI-Pred: 0.74 | **0.81** |
| Non-coding RNA family | ACC ↑ | RNA-MSM: 0.89 | **0.90** |

### Human Expert Evaluation
- **Double-blind** comparison of SciReasoner vs. DeepSeek-V4-Pro across 3 tasks.
- **98%** of cases rated SciReasoner as tie-or-better (73% strongly prefer, 21% prefer, 4% tie).
- Per-axis scores: SciReasoner averages **8.7/10** vs. **4.3/10** for DeepSeek-V4-Pro.
- All per-axis differences significant (Wilcoxon signed-rank, p < 0.001).

## Theoretical and Practical Implications

### Theoretical Contributions
1. **Native structural reasoning** establishes a new paradigm for scientific AI, where structural tokens serve as inspectable, addressable evidence rather than opaque input descriptors.
2. The model demonstrates that **structure-grounded reasoning** generalizes across diverse scientific domains (proteins, small molecules, crystals) within a single architecture.
3. The **self-bootstrapped post-training framework** shows how domain-specific reasoning patterns can be consolidated into unified capabilities without destructive interference.

### Practical Implications
1. **Drug discovery**: Accurate retrosynthesis planning, virtual screening, and scaffold hopping with explicit chemical reasoning.
2. **Functional annotation**: Improved GO term prediction for low-homology and orphan proteins, enabling functional characterization of previously unannotated sequences.
3. **Materials design**: Reliable prediction of formation energies, band gaps, and pore geometries with interpretable structural evidence.
4. **Scientific auditing**: The inspectable chain-of-thought allows domain experts to verify reasoning against physical structures, reducing reliance on black-box predictions.

## Conclusion

SciReasoner demonstrates that a single autoregressive foundation model can unify **sequence, structure, and natural-language reasoning** across proteins, small molecules, and inorganic crystals while achieving **specialist-level accuracy** and **interpretable scientific explanation**.

The central premise is that structure–property relationships require structures to be represented as **primary objects of inference**, not as text strings, low-dimensional descriptors, or black-box inputs. SciReasoner's design—discretizing coordinates, topologies, and connectivities into a unified structure-aware vocabulary and integrating them with language—enables residues, molecular fragments, and crystal descriptors to function as **addressable evidence units** within generated reasoning trajectories.

**Future directions** include:
- Extending the approach to additional scientific modalities (e.g., nucleic acid structures, complex materials interfaces).
- Scaling the model to larger backbone architectures and more diverse training data.
- Deploying SciReasoner in interactive scientific discovery workflows where human experts can query and probe the model's structural reasoning.

---

_Markdown view of https://picx.dev/p/0qxnse, served by PicX — AI-generated visual whiteboard summaries of research papers._