# Meshy T2: Fast Native Mesh Generation with Flow Matching

> Meshy T2 generates production-ready polygonal meshes from a single image in 6 seconds via flow matching, achieving state-of-the-art geometric fidelity with native face-count control.

- **Source:** [arXiv](https://arxiv.org/abs/2607.28675)
- **Published:** 2026-08-04
- **Permalink:** https://picx.dev/p/GmVldH

## Summary

## Summary (Overview)

- **Meshy T2** is a fast, native mesh generation framework built on **flow matching** that generates compact, artist-style polygonal meshes directly from a single reference image, completing end-to-end generation in a **median of 6 seconds** — over an order of magnitude faster than autoregressive baselines.
- The core innovation is a **nearly lossless vertex-set mesh VAE** (extending SpaceMesh) that encodes a mesh into one continuous latent token per vertex, decoding vertices, edge connectivity, and face winding order jointly in a single pass — without vertex quantization or welding.
- Generation proceeds via a **coarse-to-fine cascade** of two flow-matching models: an image-conditioned **voxel flow** sketches a coarse 64³ occupancy scaffold, and a **mesh flow** populates it with per-vertex latent tokens, conditioned on the image, scaffold, and a requested vertex budget.
- The system provides **effective face-count control** (via the vertex budget, leveraging Euler's formula $F = 2V - 4 \approx 2V$) and **native multi-part asset support**, where connected components emerge directly from the generated connectivity.
- Meshy T2 achieves **state-of-the-art geometric fidelity** on both retopology and image-to-mesh tasks, with a non-manifold edge count of just 0.14 and 70.1% tri-to-quad mergeability.

## Introduction and Theoretical Foundation

### Background and Motivation
Polygonal meshes are the standard surface representation for modern 3D pipelines (game engines, AR/VR, robotics, digital content). Production-ready assets must capture shape with minimal primitives while preserving sharp edges, thin structures, and semantically meaningful part boundaries. However, such meshes are predominantly hand-authored — a slow, costly workflow.

### Limitations of Prior Approaches
1. **Implicit/volumetric methods** (3DShape2VecSet, TRELLIS, LATTICE): Generate neural fields and extract meshes via Marching Cubes. These produce hundreds of thousands of near-uniform triangles that poorly respect sharp features and part structure.
2. **Autoregressive sequence models** (MeshGPT, EdgeRunner, TreeMeshGPT, BPT, DeepMesh): Serialize meshes into token sequences. Inference is inherently sequential and expensive, with error accumulation causing incomplete surfaces, inconsistent winding, or cracks.
3. **Existing diffusion/flow methods**: Face-level tokens (MeshCraft, MeshFlow) duplicate shared vertices; sparse-voxel latents (TriFlow, LATO) re-derive primitives at decode time; per-vertex tokens (Nexus) still decouple geometry and topology. Gaps include vertex quantization artifacts, heuristic face reconstruction, and lack of native image-to-mesh conditioning.

### Theoretical Foundation: Flow Matching
The framework uses **flow matching** with the **linear interpolation schedule of Rectified Flow**. Given a clean target $x_0$ and Gaussian noise $x_1$, the interpolated state at time $t$ is:
$$x_t = (1-t)x_0 + t x_1$$
The model learns a velocity field $v_\theta(x_t, t, c)$ to map noise to data, enabling **parallel generation** of all primitives rather than sequential token-by-token decoding.

## Methodology

### 2.1 Vertex-Set Mesh VAE

**Representation**: A mesh $M = (V, F)$ with vertices $V = \{x_i\}_{i=1}^{V}$, $x_i \in \mathbb{R}^3$ is encoded into a latent set $Z = \{z_i\}_{i=1}^{V}$ with exactly one token $z_i \in \mathbb{R}^C$ per vertex.

**Encoder**: 
- A local **PointNet** pools surface samples (positions and normals, drawn preferentially along mesh edges) into features on occupied cells of a 256³ sparse voxel grid.
- Each vertex query is initialized from **Fourier features** of its continuous position; all attention uses **3D RoPE**.
- Queries are refined by cross-attention into voxel context, followed by interleaved **graph-attention** (restricted to ground-truth edges) and **self-attention** layers.

**Decoder**: A pure set decoder consuming latent tokens without positional encoding, predicting:
- Continuous position $\hat{x}_i$ (vertex branch)
- Edge embedding $e_i = (e^{\text{space}}_i, e^{\text{time}}_i) \in \mathbb{R}^{d_e} \times \mathbb{R}^{d_e}$ (topology branch)
- Face embedding $f_i = (f^{\text{root}}_i, f^{\text{prev}}_i, f^{\text{next}}_i)$ (topology branch)

**Edge prediction** uses the SpaceMesh spacetime logit:
$$A_{ij} = \|e^{\text{time}}_i - e^{\text{time}}_j\|^2_2 - \|e^{\text{space}}_i - e^{\text{space}}_j\|^2_2$$
An edge is predicted when temporal separation exceeds spatial separation. The edge loss is class-balanced binary cross-entropy:
$$\mathcal{L}_{\text{edge}} = \frac{1}{Z}\left[\sum_{\{i,j\} \in E} \text{softplus}(-A_{ij}) + \lambda \sum_{\substack{i<j \\ \{i,j\} \notin E}} \text{softplus}(A_{ij})\right]$$

**Face prediction** extends SpaceMesh with a **NULL element** $\emptyset$ to handle open (non-watertight) boundaries. Each vertex's fan defines a successor mapping $\pi_i$ on neighbors $N(i)$:
$$\pi_i(p) = n \text{ for every triangle } (p, i, n) \in F$$
The face head predicts a soft permutation via Sinkhorn iterations:
$$\Phi_i[p, q] = \mathbf{1}^\top \left(f^{\text{root}}_i \odot f^{\text{prev}}_p \odot f^{\text{next}}_q\right)$$
trained with negative log-likelihood:
$$\mathcal{L}_{\text{face}} = -\frac{1}{\sum_i (D_i + 1)} \sum_i \sum_{p \in N(i) \cup \{\emptyset\}} \log P_i\left[p, \pi_i(p)\right]$$

**Training objective**:
$$\mathcal{L}_{\text{VAE}} = w_v \mathcal{L}_{\text{vertex}} + w_e \mathcal{L}_{\text{edge}} + w_f \mathcal{L}_{\text{face}}$$
with weights $w_v = 100$, $w_e = w_f = 10$, latent channels $C = 32$, and embedding sizes $d_e = d_f = 16$.

### 2.2 Stage I: Image-Conditioned Voxel Flow

- A **Voxel VAE** compresses 64³ binary occupancy grids into an 8-channel Gaussian posterior at 16³ resolution via two stride-2 3D convolutional stages, decoded with 3D pixel-shuffle upsampling.
- The flow model is a **Transformer velocity field** over the flattened 4096 latent tokens, with:
  - 3D positional self-attention
  - Cross-attention to frozen **DINOv3** image features
  - Timestep injection via **AdaLN modulation**
- Hidden dimension 1536, 28 blocks, 12 heads, SwiGLU feed-forward.

### 2.3 Stage II: Image-and-Voxel-Conditioned Mesh Flow

**Conditions**:
- **Image**: Frozen DINOv3 at 768×768, patch size 16 → 2304 image tokens
- **Voxel**: 64³ occupancy grid → Voxel VAE encoder → 4096 voxel tokens (fine-tuned at 0.25× learning rate)
- **Vertex count**: Fourier embedding of $N/N_{\max}$, added to time embedding

**Count control via padding**: Zero-valued pad tokens are randomly appended up to ratio $p$, with an extra existence channel (+1 real, −1 pad). The flow operates on $C+1 = 33$ channels. At inference, tokens with positive existence are kept as real vertices.

**Latent PE via Optimal Transport**: Latent tokens are an unordered set, so they receive positions from a **deterministic Sobol point set**. During training, real vertex coordinates are matched to Sobol candidates via **optimal transport** (minimum squared Euclidean cost). At inference, the Sobol set provides positions directly.

**Training**: Velocity-prediction flow matching with logit-normal timestep sampling. Conditions are dropped with probabilities 0.2 (image), 0.3 (voxel), 0.2 (count), and 0.05 (all) for **classifier-free guidance**.

## Empirical Validation / Results

### Position Encoding / Optimal Transport Ablation

| Position encoding | Chamfer ↓ | Hausdorff ↓ | Non-manifold edge ratio ↓ |
|---|---|---|---|
| No OT (index PE) | 0.0127 | 0.0783 | 0.0102 |
| Sobol + Morton | 0.0091 | 0.0534 | 0.0064 |
| **Sobol OT (Ours)** | **0.0070** | **0.0225** | **0.0043** |

Sobol OT reduces Chamfer distance by 45% vs. no OT, Hausdorff by over 3×, and non-manifold edge ratio by 58% vs. Sobol + Morton.

### High-Poly Mesh Retopology (Table 2)

| Method | CD ↓ | HD ↓ | NC ↑ | Non-manifold ↓ | Tri-to-Quad ↑ | Time (s) ↓ | Success ↑ |
|---|---|---|---|---|---|---|---|
| **Meshy T2** | **0.020** | **0.044** | **0.860** | **0.14** | 70.1% | **3** | **100%** |
| MeshFlow | 0.319 | 0.534 | 0.595 | 304.5 | 45.2% | 94 | 100% |
| MeshAnything V2 | 0.037 | 0.151 | 0.833 | 69.6 | 75.0% | 49 | 100% |
| BPT | 0.061 | 0.136 | 0.797 | 39.0 | 60.9% | 210 | 95.7% |
| DeepMesh | 0.453 | 0.719 | 0.537 | 233.0 | 80.8% | 636 | 28.7% |
| MeshSilksong | 0.309 | 0.525 | 0.629 | 19823.0 | 45.7% | 1210 | 49.6% |
| FastMesh | 0.049 | 0.175 | 0.774 | 17234.7 | 17.4% | 80 | 100% |

Meshy T2 achieves the best geometric fidelity (CD 0.020, HD 0.044, NC 0.860) and runs in median **3 seconds** — more than an order of magnitude faster than all baselines.

### Image-to-Mesh Generation (Table 3)

| Method | FD (Inception) ↓ | FD (DINOv2) ↓ | Time (s) ↓ | Success ↑ |
|---|---|---|---|---|
| Meshy 6 (reference) | 240.65 | 2021.68 | — | — |
| **Meshy T2** | 255.77 | **2312.01** | **6** | **100%** |
| Tripo P1 | 255.76 | 2442.27 | 12 | 100% |
| MeshFlow | **254.06** | 2577.00 | 94 | 100% |
| MeshAnything V2 | 257.02 | 2499.05 | 49 | 100% |
| BPT | 257.95 | 2457.99 | 210 | 95.7% |
| DeepMesh | 281.68 | 2752.54 | 636 | 28.7% |
| MeshSilksong | 301.09 | 2525.77 | 1210 | 49.6% |
| FastMesh | 268.21 | 2405.57 | 80 | 100% |

Meshy T2 leads on DINOv2 FD (2312.01), indicating strongest semantic and structural alignment, with 100% success in median 6 seconds.

## Theoretical and Practical Implications

### Theoretical Contributions
- **Nearly lossless mesh representation**: By avoiding vertex quantization and welding, and recovering vertices, edges, and face winding jointly, the VAE preserves high-precision geometry and artist-authored topology exactly — addressing a fundamental gap in prior latent mesh representations.
- **Optimal transport for unordered latent sets**: The Sobol OT position assignment resolves the permutation ambiguity inherent to set-based generation, providing a principled solution applicable to any set-based generative model.
- **Unified flow-matching cascade**: The coarse-to-fine split (voxel scaffold → mesh latent) decomposes the ill-conditioned image-to-mesh problem into tractable sub-problems.

### Practical Implications
- **Interactive speed**: Median 6-second end-to-end generation makes Meshy T2 practical for interactive asset creation workflows.
- **Face-count control**: Users can directly specify vertex budgets (with $F \approx 2V$), enabling level-of-detail pipelines without post-hoc simplification.
- **Multi-part assets**: Connected components emerge naturally from generated connectivity, eliminating separate part-wise generation or stitching stages.
- **Production-ready topology**: Low non-manifold edge counts (0.14) and high tri-to-quad mergeability (70.1%) indicate outputs are closer to artist-quality than prior automated methods.

## Conclusion

Meshy T2 demonstrates that **flow-based parallel generation** over a carefully designed vertex-set latent representation can match or exceed the geometric fidelity of autoregressive methods while being over an order of magnitude faster. The key insights are:

1. **Vertex-set latent representation** with continuous coordinates and explicit connectivity prediction enables nearly lossless mesh encoding.
2. **Coarse-to-fine flow cascade** (voxel scaffold → mesh flow) effectively handles the ambiguity of image-to-mesh generation.
3. **Optimal transport position assignment** resolves the permutation ambiguity of unordered latent sets.

**Future directions** identified by the authors include:
- Improving topological robustness on highly irregular source meshes
- Extending to richer material and part-level controls
- Scene-level generation with multiple compact meshes under a shared global budget

The code and weights will be released at https://github.com/meshy-dev/meshy-t2.

---

_Markdown view of https://picx.dev/p/GmVldH, served by PicX — AI-generated visual whiteboard summaries of research papers._