Scaling laws are well established for language models in the physical world, but do they hold for embodied agents acting in virtual worlds with different graphics, physics, controls, and objectives? This paper runs a controlled study of continued pre-training across three axes, with 120 configurations spanning four model sizes (0.8B to 27B), six data budgets (50 to 1,600 hours), and five data mixtures from Minecraft-only to diverse multi-world corpora. The findings: more data helps within the training worlds, data diversity transfers to related worlds but barely to distant ones, and larger models use data more efficiently yet generalize worse to distant worlds under single-world training. Training also splits into an early interface phase with broad cross-world gains and a later world phase where gains narrow to familiar worlds. The authors fit an empirical scaling law relating validation loss to volume, diversity, and size, and note that persistent gaps in distant worlds mean generalist embodied agents will need training stages beyond continued pre-training.
- 411 stars/7d
- 0 citations
- 56 upvotes
arXiv · THUSI-Lab/GameScaling