# WorldClaw: Agentic 3D Open-World Generation at Scale

> WorldClaw generates large-scale explorable 3D worlds from text prompts via a coarse-to-fine agentic pipeline, producing editable meshes with terrain and placed objects.

- **Source:** [arXiv](https://arxiv.org/abs/2608.05248)
- **Published:** 2026-08-08
- **Permalink:** https://picx.dev/p/4nIoDN
- **Whiteboard:** https://picx.dev/p/4nIoDN/image

## Summary

## Summary (Overview)

- **WorldClaw** is a fully agentic, coarse-to-fine framework for generating large-scale, freely explorable 3D worlds from open-ended text prompts, developed by Tencent Hunyuan.
- The framework follows a **global-to-regional principle**: it first establishes scene-wide semantics, regional organization, and a terrain foundation, then selectively generates instance-level content only in regions requiring further development.
- The pipeline consists of three sequential stages: **intent analysis and planning**, **global terrain generation**, and **regional object generation and placement**, coordinated by specialized agents communicating through shared structured intermediate representations.
- WorldClaw produces scenes as **independently editable textured meshes** with explicit terrain placements, supporting free-viewpoint rendering, asset reuse, and game-engine workflows.
- Key contributions include a semantic-layout-guided procedural terrain generator, translation of global/local constraints into executable regional plans, and an agentic refinement loop that corrects object scale, pose, and object–terrain contacts.

## Introduction and Theoretical Foundation

Generating explorable 3D worlds from a single sentence remains a compelling yet difficult goal in generative content creation. Unlike images or videos locked to predefined camera paths, such worlds must be **walkable, structurally consistent, and editable**—properties that transform generated content from a fixed visual artifact into a persistent, reusable environment supporting exploration, interaction, simulation, and iterative creation.

The paper categorizes existing methods into four paradigms:

1. **Procedural Content Generation (PCG)**: Builds terrain, vegetation, and buildings with rules and executable programs (e.g., Infinigen). Strong controllability and scalability, but limited by hard-coded rule expressiveness.
2. **Image/Video-Lifting**: Synthesizes scene observations via image/video generators and lifts them to 3D (e.g., Marble). Rich content but lacks global consistency and geometric fidelity; computationally inefficient for complete scenes.
3. **Native 3D Diffusion**: Learns scene distributions directly over 3D voxels, distance fields, or Gaussians. Good geometric fidelity and cross-view consistency, but content diversity limited by scarce 3D scene datasets.
4. **Multi-Modal LLM Agents (MLLM-Agents)**: Leverage world knowledge of multimodal LLMs for intent understanding, spatial planning, and tool orchestration. Excel at global planning but lack precise 3D spatial control.

**Central insight**: A globally coherent world need not be generated everywhere at once. Shared constraints (scene semantics, spatial organization, terrain foundation) are established globally first, while objects and local relationships are progressively realized where needed.

The overall construction process is formalized as:

$$
\mathcal {P} = F _ {\mathrm{plan}} (q), \qquad \mathcal {T} = F _ {\mathrm{terrain}} (\mathcal {P}), \qquad \mathcal {O} = F _ {\mathrm{region}} (\mathcal {P}, \mathcal {T}),\tag{1}
$$

where $q$ is the user prompt, $\mathcal{P}$ is the structured scene specification, $\mathcal{T}$ is the global terrain representation, and $\mathcal{O}$ is the set of generated and placed regional object instances. The final world is composed as:

$$
\mathcal {S} = \operatorname{Compose} (\mathcal {T}, \mathcal {O}).\tag{2}
$$

## Methodology

### Stage 1: Intent Analysis and Planning

An **intent analysis agent** extracts and normalizes explicit constraints from the user prompt (scene type, theme, visual style, key regions/objects, spatial relationships) without introducing new content. A **scene planning agent** then resolves ambiguities and completes unspecified information according to a predefined schema, producing:

$$
\mathcal {P} = \left(\mathcal {R}, \mathcal {C} _ {\text { terrain }}, \mathcal {C} _ {\text { object }}\right),\tag{3}
$$

where $\mathcal{R}$ describes major scene regions and their spatial relationships, $\mathcal{C}_{\text{terrain}}$ describes terrain types and landform characteristics, and $\mathcal{C}_{\text{object}}$ specifies object categories, appearance attributes, and densities.

### Stage 2: Global Terrain Generation

**Terrain Planning**: A terrain planning agent converts high-level constraints into a structured terrain specification:

$$
\mathcal {P} _ {\mathrm{terrain}} = (\mathbf {p} _ {\mathrm{layout}}, \mathbf {p} _ {\mathrm{asset}}, \mathbf {p} _ {\mathrm{material}}, \boldsymbol {\theta} _ {\mathrm{terrain}}),\tag{4}
$$

where $\mathbf{p}_{\mathrm{layout}}$ specifies region categories and adjacency, $\mathbf{p}_{\mathrm{asset}}$ describes terrain-associated asset categories and densities, $\mathbf{p}_{\mathrm{material}}$ defines surface types and styles, and $\boldsymbol{\theta}_{\mathrm{terrain}}$ contains numerical parameters (world scale, elevations, noise frequencies, geomorphic operator weights). The agent may also invoke search tools and generate a concept image $\mathbf{I}_{\mathrm{concept}}$ as visual conditioning.

**Terrain Asset Generation**: Produces a semantic layout map $\mathbf{I}_{\mathrm{layout}}$ (encoding region categories via distinct colors), reusable 3D asset prototypes $\mathcal{O}_{\mathrm{asset}}$ (via GPT-Image-2 and Hunyuan3D image-to-3D), and terrain materials $\mathcal{M}_{\mathrm{terrain}}$ via two pathways: generative texture synthesis (albedo, normal, roughness maps) and procedural Blender material-node generation.

**Terrain Generation & Refinement**: The global height field is constructed as:

$$
H (\mathbf{x}) = \sum_ {r} \widetilde {m} _ {r} (\mathbf{x}) \left[ h _ {r} + \sum_ {k} w _ {r, k} N _ {r, k} (\mathbf{x}) + \sum_ {j} \alpha_ {r, j} G _ {r, j} (\mathbf{x}) \right],\tag{6}
$$

where $\mathbf{x}$ is a 2D position, $h_r$ is the base elevation of region $r$, $N_{r,k}$ is a noise component at spatial frequency $k$, $G_{r,j}$ is a geomorphic operator (peak, dune, terrace, erosion), and $w_{r,k}$, $\alpha_{r,j}$ control noise and geomorphic contributions. Soft region weights $\widetilde{m}_r$ enable continuous composite height fields with irregular boundaries. Global terrain asset scattering then instantiates reusable prototypes (rocks, vegetation clusters) with scale/orientation adapted to local surface. A terrain refinement agent (powered by BlenderMCP) renders, inspects, and locally corrects geometry, materials, and scattering.

### Stage 3: Regional Object Generation and Placement

**Regional Planning**: A regional planning agent selects a subset $\mathcal{R}^+ \subseteq \mathcal{R}$ of regions requiring instance-level content and produces:

$$
\mathcal {P} _ {\text {regional }} = \left\{\left(r, \phi_ {r}, \mathcal {C} _ {r} ^ {\text {object }}, \mathbf {p} _ {r} ^ {\text {spatial }}, \mathbf {p} _ {r} ^ {\text {appearance }}\right) \mid r \in \mathcal {R} ^ {+} \right\},\tag{7}
$$

where $\phi_r$ is the region's functional role, $\mathcal{C}_r^{\text{object}}$ specifies object categories/counts/densities, and $\mathbf{p}_r^{\text{spatial}}$, $\mathbf{p}_r^{\text{appearance}}$ define spatial relationships and appearance requirements.

**Region Composition**: The agent renders the existing terrain with camera parameters $\kappa_r = (K_t, E_t)$, producing a terrain image $\mathbf{I}_r^{\text{terrain}}$, then generates a region composition image conditioned on the terrain rendering:

$$
\mathbf {I} _ {r} ^ {\mathrm{comp}} = \mathcal {G} _ {\mathrm{image}} \left(\mathbf {I} _ {r} ^ {\mathrm{terrain}}, \mathcal {P} _ {r}, \mathbf {I} _ {\mathrm{concept}}\right).\tag{9}
$$

**Object Generation**: Text-guided SAM3 extracts individual 2D object instances (with sliding-window inference for multi-scale recall). For each instance $i$, an object-centric image is cropped and enlarged via affine transformation $A_i$, giving equivalent intrinsics:

$$
\widehat {K} _ {i} = A _ {i} K _ {t}.\tag{10}
$$

SAM3D predicts an object mesh $M_i$, appearance $\mathcal{U}_i$, and local-to-camera transform $T_{l2c}^i$. Scale calibration iteratively adjusts a factor $\lambda_i$ until:

$$
- \epsilon_ {i} ^ {-} \leq \mathcal {B} (\Pi (K _ {i} ^ {o}, \lambda_ {i} M _ {i} ^ {c})) - b _ {i} ^ {\mathrm{ref}} \leq \epsilon_ {i} ^ {+},\tag{11}
$$

where $\Pi(\cdot)$ is rendering projection and asymmetric tolerances suppress oversized reconstructions.

**Object Placement**: Using corresponding rays from the object reconstruction camera and terrain camera, the initial scale preserving apparent object size is:

$$
s _ {i} = \frac {Z _ {t}}{Z _ {o}} \frac {f _ {i} ^ {o}}{\widehat {f _ {i}}},\tag{12}
$$

where $Z_t$, $Z_o$ are depths at terrain anchor and object reference points, and $f_i^o$, $\widehat{f}_i$ are focal lengths. The complete placement transformation is:

$$
T _ {\mathrm{place}} ^ {i} = \left[ \begin{array}{c c} s _ {i} R _ {i} & P _ {t} - s _ {i} R _ {i} P _ {o} \\ \mathbf {0} ^ {\top} & 1 \end{array} \right] T _ {l 2 c} ^ {i}.\tag{13}
$$

A joint search over anchor depth and isotropic scale minimizes floating/penetration by maximizing object–terrain contact ratio.

**Scene Refinement**: An agentic loop (via MCP) performs object refinement (evaluating pose, mesh quality, scale; re-conditioning Hunyuan3D on coarse mesh + image for improved geometry/textures) and terrain refinement (object–terrain co-deformation for floating, penetration, unstable support), with all modifications restricted to local support regions.

## Empirical Validation / Results

**Implementation**: WorldClaw uses Claude Opus 4.8 as the agent model, with GPT-Image-2, SAM3, SAM3D, and Hunyuan3D as foundation models. Experiments run on 4 NVIDIA H20 GPUs with Blender 5.1.1. PBR textures at 2048×2048 for large objects and 1024×1024 for small objects.

**Qualitative Results**: Four representative worlds are demonstrated:
- **Tropical pirate stronghold** (One Piece-inspired): irregular coastlines, dense vegetation, settlements, docks, ships
- **Canyon with tribal settlements**: continuous river, substantial elevation changes, villages on cliffs and valley floor
- **Desert battlefield** (PUBG-inspired): layered rocky landforms, open combat areas, buildings, defensive structures, vehicles
- **Snow-covered mountain valley** (Command & Conquer: Red Alert style): enclosed valley with futuristic facilities

Each scene shows global orbit views, regional close-ups, and local walk views with instance/depth/normal renderings, demonstrating coherent global structure with region-specific content.

**Qualitative Comparison** (vs SynCity, Marble, MajutsuCity, WorldGen, GPT-5.6 Sol):

| Method | Terrain Expressiveness | Content Richness | Representation |
|--------|----------------------|------------------|-----------------|
| SynCity | Block-wise latents, weaker long-range organization | Captures theme, but coarse/repeated local structures | Persistent 3D, coarse local geometry |
| Marble | Lacks explicit region-level organization | Visually rich but dominated by buildings/vegetation | Degrades over long distances; discrete Gaussians |
| MajutsuCity | Regular ground geometry, city-oriented | Well-arranged buildings, urban-focused | Controllable assets, regular layouts |
| WorldGen | Flat, homogeneous terrain | Detailed village content, limited variation | Explicit meshes, stable traversal (closest baseline) |
| GPT-5.6 Sol | Simple geometric forms, coarse transitions | Captures main semantics, simplified geometry | Explicit 3D, blockout-like appearance |
| **WorldClaw** | **Semantic layout map, pronounced elevation variation, connected regions** | **Rich content variation, aligned with regional functions** | **Explicit terrain + independent object meshes** |

## Theoretical and Practical Implications

- **Decoupling global/local generation** provides a scalable alternative to monolithic single-pass generation, avoiding the content-diversity limits of native 3D diffusion and the global-inconsistency problems of video-lifting approaches.
- The **semantic layout map** bridges the gap between LLM-level spatial planning and executable terrain geometry, addressing the planar-ground assumption that limits many existing pipelines.
- The **two-camera ray correspondence** method for object placement (Eqs. 10–13) enables high-resolution object reconstruction without sacrificing placement accuracy—a practical solution to the scale/pose ambiguity problem in image-to-3D lifting.
- The **agentic refinement loop** (render → inspect → edit → re-render) provides a generalizable mechanism for quality control that does not require end-to-end training.
- The explicit representation (independently editable meshes with terrain placements) enables downstream workflows in game engines, animation authoring, and simulation—a significant practical advantage over monolithic scene representations.

## Conclusion

WorldClaw demonstrates that a coarse-to-fine agentic pipeline can generate large-scale, explorable 3D worlds with coherent spatial organization, visually compelling local content, and editable instance-level assets. The key design principle—decoupling global world organization from local instance-level generation—enables the system to achieve an effective balance among scene scale, visual quality, and editability.

**Limitations** acknowledged by the authors:
1. **High dependency on underlying models**: The pipeline requires capable models (Claude Opus 4.8, GPT-Image-2, Hunyuan3D); open-source alternatives often fail at procedural terrain generation or layout-map production.
2. **Stability risks in code generation**: LLM-generated Blender programs can contain scale errors, numerical parameter mistakes, or node-connectivity issues requiring multiple refinement iterations.
3. **Efficiency overhead**: Long-horizon pipelines with per-object reconstruction and multi-round refinement incur substantial latency and computational cost, especially for simpler scenes.

**Future directions**:
- **Code-native 3D modeling**: Replacing selected object-generation stages with executable modeling programs (already explored for terrain materials via Blender node graphs), enabling explicit part hierarchies, parametric structures, and articulation definitions.
- **Integration with production engines**: Extending beyond Blender to Unreal Engine's procedural content generation, physics, navigation, and interaction systems.
- **Executable world representations**: Moving toward worlds whose geometry, materials, internal structures, animation capabilities, and interactive behaviors are jointly defined through executable representations—narrowing the gap between generative models and production workflows.

---

_Markdown view of https://picx.dev/p/4nIoDN, served by PicX — AI-generated visual whiteboard summaries of research papers._