Mark Zuckerberg boasts of owning the world's largest H100 chip cluster

Mark Zuckerberg said Meta is deploying a server cluster containing more than 100,000 Nvidia H100 GPUs, each chip costing about $40,000.

"The Llama 4 large language model is in development. We're training it on a cluster of servers using over 100,000 H100 chips, more than any system ever announced in the media," Meta CEO Mark Zuckerberg said on the Q3/2024 earnings call this week.

He did not disclose the exact number of H100s Meta owns. On the market, each Nvidia H100 chip costs $30,000-$40,000, meaning Meta's server cluster is worth more than $4 billion if only counting the cost of purchasing GPUs.

The 100,000 figure Zuckerberg mentioned is said to be a comparison to the Colossus supercomputer, deployed by billionaire Elon Musk's xAI startup since the summer. The Tesla boss then described it as the world's largest supercomputer, equipped with 100,000 H100 chips to train the Grok chatbot.

Nvidia H100 chip card. Photo: Serve The Home

Zuckerberg claims Llama 4 will be more powerful and capable of inferring, and much faster than previous models. “A small model of it could be released as early as 2025,” he said.

The number of H100 chips each company owns is becoming a factor when recruiting and attracting talent. Perplexity CEO Aravind Srinivas revealed this information when he was looking to hire a senior researcher from Meta.

“You know what the researcher said? He said, ‘Come back to me when you have 10,000 H100 chips,’” Srinivas said.

Elon Musk also announced earlier this week that xAi will soon double its server system, increasing the number of H100 and H200 chips to about 200,000 in the next few months.

The H100 GPU is the successor to the $10,000 A100, which has been dubbed the “workhorse” of the AI ​​industry. In AI training, GPUs have an advantage over CPUs thanks to their ability to perform massive parallel computations. Developers are using the A100 and H100 to build large language models (LLMs), the core of AI applications like ChatGPT.

Currently, Nvidia's GPUs are the most ordered, while other companies like AMD have also launched similar products.