CWM: Code Generation with World Models

This Meta September 24 2025 paper provides an extensive overview of Code World Model (CWM), a 32-billion-parameter dense decoder-only Transformer designed for coding and reasoning tasks, highlighting its architecture and multi-stage training process. Training involves pre-training, mid-training, and post-training stages which include supervised fine-tuning (SFT) and joint reinforcement learning (RL) across environments like software engineering (SWE) tasks, coding problems, and mathematics. A core feature is CWM's ability to process and predict Python execution traces and perform agentic interactions using a minimal set of tools within containerized environments. The document details the model's competitive performance against other large language models on benchmarks like SWE-bench Verified and discusses infrastructure choices, such as asynchronous RL and fp8 matrix multiplication, used to achieve training efficiency.