Scaling Laws & Training Methods

This direction addresses core scientific questions in large-scale model pretraining, including scaling laws and compute-optimal allocation, parameterization and hyperparameter transfer, optimizer and training stability, and the design and comparison of pretraining objectives. We prioritize work with multi-scale validation, reliable extrapolation, and transparent training details, aiming to advance predictable and reproducible training methodologies.