Is the new mantra for AI: Do one thing well?

🌈 Abstract

The article discusses the evolution of language models, from GPT-3 to GPT-5, and how this progression can be analogized to the size and power of different marine creatures. It then explores the broader theme of how technology often moves from monolithic, centralized approaches to more federated, specialized solutions. The article suggests that this pattern may also apply to the development of AI models, where we may see the emergence of "appropriately sized" AI systems that are better suited for specific tasks and edge deployments.

🙋 Q&A

[01] The Analogy of Marine Creatures

1. What is the analogy used to describe the relative sizes of GPT-3, GPT-4, and GPT-5? The article uses an analogy of marine creatures to convey the idea of scale, with GPT-3 being portrayed as a Great White Shark, GPT-4 as an Orca, and GPT-5 as a Humpback Whale, indicating the massive increase in data in their training.

2. What do the different marine creatures represent in terms of technology evolution? The article suggests that the analogy is "doubly interesting" when considering what the different marine creatures represent. GPT-3 is described as the "apex predator in the fish world," while the Humpback Whale eats krill and has fallen victim to Orcas. This is likened to the battle between different technology architectures, where one architecture (like Unix) often wins out over others.

[02] The Battle Between Monolithic and Federated Approaches

1. What is the key theme regarding the evolution of technology approaches? The article discusses how the battle between monolithic and federated approaches has played out repeatedly in technology, with the winner often being the federated and specialized approach, the idea of "do one thing well" and then collaborate to drive outcomes.

2. Can you provide examples of this pattern in different technology domains? The article gives several examples of this pattern, including:

• The battle between VMS, Windows NT, and Unix, where Unix's "simple philosophy" of creating individual tools that can be chained together won out
• The rise of distributed systems, SOA, and REST as a reaction to unmaintainable monolithic systems
• The shift from the "mythical great big database in the sky" to distributed data meshes
• The move from large single-core processors to multi-core, multi-purpose solutions

3. How does this pattern relate to the size and capabilities of technology solutions? The article suggests that this pattern of moving from monolithic to federated approaches also applies to the size and capabilities of technology solutions. It notes that Unix has been great at fitting into smaller form factors, and questions whether we'll see a similar trend with AI models, where we get "appropriately sized" AI systems that are better suited for specific tasks and edge deployments.

[03] The Potential for "Appropriately Sized" AI Models

1. What are the potential benefits of having "appropriately sized" AI models? The article suggests that as AI models get deployed to edge devices like mobile phones, there will be a need for smaller, more purpose-specific models that can protect privacy and remove network latency, rather than relying on large, centralized models.

2. How does Apple's "Intents" approach in Apple Intelligence represent a new way of building collaborative AI applications? The article discusses how Apple's "Intents" approach allows apps to focus on specific tasks while still leveraging the capabilities of models, and enables different applications to collaborate through a shared interface. This represents a shift towards a more federated, collaborative approach to building AI applications.

3. What is the article's view on the potential for a single, monolithic AI model approach in the future? The article expresses doubt that a single model approach will be the future, stating that "it is possible that a single model approach could be the future, but I really doubt it." Instead, the article suggests that we will need "appropriately sized" AIs that can be clearly defined in their purpose and engaged in collaboration.