xAI Grok 3.5 Renamed Grok 4 and Has Specialized Coding Model

The xAI team is claiming good progress on their model. It will be called Grok 4 instead of Grok 3.5. It will release just after July 4th. Needs one more big run for a specialized coding model.

xAI has two AI data centers. The first data center had 100,000 Nvidia H100 for the Grok 3 training and it has 50,000 more H100 and 50,000 more H200 when Grok 3 was released. xAI added 30,000 B200 chips. This equals 400,000 Nvidia H100.

xAI is going to or has already activated the second AI data center with 110,000 Nvidia B200 which are equal to 550,000 H100. The second AI data center will expand to 1 million Nvidia B200 or B300 over the 5-8 months. This means adding 100,000 to 200,000 B200 chips every month.

Grinding on @Grok all night with the @xAI team. Good progress.

Will be called Grok 4. Release just after July 4th. Needs one more big run for a specialized coding model.

— Elon Musk (@elonmusk) June 27, 2025

xAI will likely dedicate the majority of the second data center’s resources to pre-training a next-generation model—potentially xAI’s most ambitious yet—while using the first data center for supplementary pre-training tasks or smaller-scale experiments.

Post-training (or fine-tuning) refines the pre-trained model by training it on specific tasks or datasets to improve performance in targeted areas. This phase requires fewer resources than pre-training but benefits from parallelization and flexibility.

First Data Center:
With 400,000 H100 equivalents, this data center can handle fine-tuning Grok 3.5 or its successors for specific applications (e.g., scientific reasoning, conversational accuracy, or domain-specific knowledge).

Second Data Center:
Initial Phase: Allocate a portion of the 550,000 H100 equivalents to fine-tune the pre-trained models on diverse tasks, running multiple fine-tuning jobs in parallel.

Expanded Phase: With 5,000,000 H100 equivalents, xAI can scale up fine-tuning efforts significantly, tailoring multiple model variants for different use cases (e.g., enterprise solutions, research tools, or real-time systems).

Reinforcement learning (RL) enhances the model’s decision-making and adaptability by rewarding desired outputs and penalizing undesired ones. This phase is computationally intensive for complex tasks and critical for achieving human-like performance.
First Data Center:
Apply RL to Grok 3.5 or smaller models to refine their behavior, such as improving response coherence or task-specific accuracy.

Second Data Center:
Initial Phase: Use part of the 550,000 H100 equivalents for RL on early models trained in this data center.

Expanded Phase: With 5,000,000 H100 equivalents, allocate significant resources to advanced RL techniques, potentially integrating the AI with real-world systems (e.g., robotics or simulations) to push its capabilities beyond language tasks.

Best Use: Reserve a portion of the computational power (e.g., 10-20% of the second data center) for RL, focusing on optimizing the model’s performance and exploring innovative applications.

They can prioritize pre-training for scale, fine-tuning for specialization, and reinforcement learning for refinement—while leveraging distributed training and a phased expansion—xAI can maximize the potential of these data centers

Faster Experimentation and Model Selection: With the ability to run many post-training experiments in parallel, xAI can test and validate multiple fine-tuning strategies, architectures, and hyperparameters at once. This dramatically reduces the time to identify optimal configurations, leading to higher-performing models in less time