OpenAI and Microsoft building "Stargate"

Microsoft and OpenAI have been discussing a project called “Stargate” that would see Microsoft spend $100 billion to build a massive supercomputing cluster to support OpenAI’s future advanced AI models, The Information reported Friday.
To put this in context, Microsoft is known to have spent more than “several hundred million dollars” to build the clusters used to train OpenAI’s current top-of-the-line model GPT-4, which OpenAI CEO Sam Altman also has said cost more than $100 million to train. It's also known that OpenAI is already training a successor model to GPT-4, likely called GPT-5, on one of Microsoft’s existing data centers. And we know that Microsoft last year broke ground on a new $1 billion data center in Wisconsin that analysts believe is intended to house the chips for training OpenAI’s next-generation models—probably due out in 2025 or 2026. The Information reported that this supercomputing cluster in Wisconsin may eventually cost as much as $10 billion, once the price of specialized Nvidia chips used for AI applications are factored in. So Stargate is anywhere from 10 to 100 times more expensive than any of the data centers Microsoft currently has on the books.
Of course, $100 billion is a lot of money, even for Microsoft. It's more than three times what the company spent on capital expenditures in 2023 and twice the amount it is on pace to spend this year. It is also more than almost any company, including those in far more capital-intensive sectors, spends on capex annually: Saudi Aramco, for instance, spent about $50 billion on capital projects last year. It's also two-thirds of the entire amount Amazon's AWS has said it plans to spend on all new data centers over the next 15 years.
Skeptics of today’s approaches to AI seized on the Stargate story as an “AI jumps the shark” moment. The only way a single OpenAI model could justify such an outlandish investment on a single data center, they said, was if that model were in fact AGI—or artificial general intelligence, or a single AI system that could perform most cognitive tasks as well or better a human. Achieving AGI is OpenAI’s founding mission. But since some of these skeptics doubt AGI is achievable within the next decade, they predicted this investment—if Microsoft does, in fact, make it—would prove foolish. Gary Marcus, the AI expert who has emerged as a perpetual critic of most neural network-based approaches to AI, called Stargate “the second worst AI investment in history”—comparing it to the more than $100 billion companies have plowed into self-driving cars, which today only operate in a few limited geographies. But remember, that $100 billion was spent by many different investors and companies. Stargate would be all Microsoft.
What’s more, Microsoft has to hope that whatever Stargate is being used for, it is not in fact to train AGI, since Microsoft’s partnership with OpenAI only entitles the tech giant to commercialize OpenAI’s technology that falls short of AGI. Once OpenAI’s board decides AGI has been achieved, the company doesn't have to share that technology with Microsoft. For Microsoft to invest so much in training an OpenAI model, it must be reasonably sure that the model will be very capable, but not yet so capable as to qualify as AGI.
Beyond this inherent contradiction at the heart of the Microsoft-OpenAI partnership, Project Stargate signals several other things. Jack Clark, who in addition to heading policy at OpenAI rival Anthropic writes a good newsletter on AI, Import AI, points out that the seemingly soaring capital intensity of advanced AI has important implications for AI policy. Capital-intensive industries, such as mining or oil and gas, tend to be highly concentrated, with a handful of large companies dominating the market. They also tend to be heavily regulated. Project Stargate may be an indication this is where AI is headed.

But there are other interesting implications for both policy and corporate strategy. In some ways, we may come to see Stargate as AI’s Star Wars moment. And no, I don’t mean the movie. I am referring to the nickname for the Strategic Defense Initiative, the space-based antiballistic missile system President Ronald Reagan announced the U.S. intended to build in 1983. The important thing about SDI was not the tech—most of which didn’t exist when Reagan declared the U.S. would build the system (and in fact, the whole project was later scrapped). What mattered was the audacious ambition and its eye-watering projected price tag—which at the time was estimated to be $30 billion ($93 billion in today’s dollars) over its first six years with estimates ranging from $100 billion to $1 trillion over the first decade. Many analysts have since claimed that those costs convinced the Soviet Union’s leadership that the country could not compete economically with the U.S. The Soviets simply didn't have the money to try to match the American investment while also modernizing their missiles to try to defeat the U.S. antimissile system. There are indications that this realization was among the factors that persuaded the Soviet leadership to begin economic reforms and political liberalization that ultimately hastened the collapse of the entire Soviet system.
Stargate may play a similar role between both companies—and countries. Microsoft currently operates with about a 35% profit margin. Meta’s is similar. Google’s is slightly less at 24%. Amazon, whose AWS service is another big cloud rival, operates at just 8% margins. If you are Amazon, are you really willing to spend $100 billion on a single data center to match Microsoft? What happens when the costs for the data center for the model beyond this next generation hits $200 billion? Stargate could well convince one of these companies that it can’t afford to be in the frontier model business.
The geopolitics of this are interesting too. Most of the large companies building frontier AI models are American. If you are China, which sees advanced AI, and certainly AGI, as a strategic asset, you are facing the prospect of the U.S. having not one but perhaps four or five $100 billion AI supercomputers being built in the next five years, all at no direct cost to the U.S. government. Meanwhile, China has been subsidizing some of the efforts by its internet companies, such as Baidu, to build AI supercomputing clusters, partly because these companies have been cut off from the most advanced AI chips produced by Nvidia and partly because of fears they may be falling behind their U.S. tech company rivals. At some point, Beijing may decide it can't afford to match the American effort like-for-like. This could have several effects, one of which might be to drive China to find other ways to race for AGI—maybe trying to find a path to AGI that does not depend on transformer-based neural networks and GPUs. It could put more money into different kinds of algorithms or different kinds of chips, perhaps achieving a surprise breakthrough. It could also redouble efforts to steal and then replicate U.S.-made AI models.
We’ll be watching to see how these dynamics play out in the coming months and years.


Source
https://fortune.com/2024/04/02/microsoft-openai-stargate-100-billion-ai-supercomputer-star-wars-sdi/