Microsoft’s $100 Billion Bet on OpenAI Is Stupid. Unless…

🌈 Abstract

The article discusses a mysterious $100 billion project called "Stargate" that Microsoft is allegedly investing in to build the most advanced data center in the world, in collaboration with OpenAI. It explores the potential reasons behind this massive investment, including the possibility of a shift towards "long-inference AI models" that can engage in more deliberate, iterative problem-solving, rather than just providing one-shot responses.

🙋 Q&A

[01] Scaling Laws and Efficiency Improvements in AI

1. What are the key developments that are making AI models more efficient and cheaper?

• Mixture-of-Experts (MoE) architectures, where only a subset of the model's experts are activated for each prediction, reducing computational costs
• Reducing parameter precision to 1-bit, eliminating the need for expensive matrix multiplications
• Hybrid architectures that combine attention mechanisms with more efficient, subquadratic operators
• Ring Attention, a new distributed computing architecture that reduces memory requirements

2. How are these developments painting a picture of AI becoming more efficient and cost-effective in the long run?

• Despite the continued growth in model size, these innovations suggest that running large AI models will become much cheaper over time
• This raises questions about why major tech companies like Microsoft are still investing heavily in massive data centers and compute power for AI

[02] Shift Towards "Long-Inference" AI Models

1. What is the evidence for a shift towards "long-inference" AI models?

• Research from OpenAI, MIT, Google DeepMind, and others has shown that allowing AI models to iterate and explore solutions over a longer period of time can dramatically improve their performance
• Examples include AlphaCode, an AI model that reaches the 85th percentile in competitive programming competitions by sampling up to 1 million possible solutions, and AlphaGeometry, which explores different paths to prove geometry theorems

2. How do these "long-inference" models differ from traditional AI models?

• Traditional AI models allocate the same compute effort per token prediction, regardless of the complexity of the task
• "Long-inference" models aim to replicate human "System 2 thinking", where more deliberate, conscious effort is applied to solve complex problems
• This requires allowing the models to explore and iterate, rather than rushing to a single answer, which is much more computationally expensive

3. How does this relate to Microsoft's "Stargate" project?

• The article suggests that Stargate may be focused on training the next generation of "long-inference" AI models, rather than just scaling up existing language models
• This would explain the massive $100 billion investment, as these new models are expected to be much more computationally intensive than current AI systems