Does AI have a gross margin problem?

🌈 Abstract

The article discusses the challenges and opportunities around the profitability and gross margins of AI companies. It examines the cost structure of running large language models, the potential for optimizing AI models and hardware, and the long-term profitability outlook for AI businesses.

🙋 Q&A

[01] Financial Operations and Mercury

1. What are the key points about financial operations and Mercury?

• Financial operations are needlessly complex, requiring a patchwork of unintegrated tools that lead to time waste and errors
• Mercury simplifies financial workflows by providing banking and software that powers critical financial operations all within a single bank account
• Mercury offers new bill pay and accounting capabilities to help businesses pay bills faster, control company spend, and speed up reconciliation

[02] AI Revenue Potential and Profitability

1. What are the key points about the revenue potential and profitability of AI companies?

• There is no question about the revenue potential of AI, with companies like OpenAI and Anthropic seeing rapid growth
• However, the costs of running AI models, particularly the datacenter costs, are quite high and can put pressure on gross margins
• Estimates suggest Anthropic's gross margin was 50-55% in December, far lower than the 77% average for cloud software stocks
• The costs of training AI models, which can be up to $100 million per model, are not even reflected in the gross margin figures

2. What insights did the analyst Fred Havemeyer provide on the gross margin profile of generative AI?

• Larger AI models can be run with 60%+ gross margins
• Mid-sized AI models can achieve 80%+ gross margins
• Smaller, purpose-built generative AI models can reach 90%+ gross margins, similar to software-like gross margins
• This suggests AI companies can run profitably and not just be a gross margin drag on the business

3. What factors will separate the winners from the losers in achieving good gross margins for AI companies?

• Relationships with key suppliers, particularly cloud providers (AWS, GCP, Azure) and chip manufacturers (Nvidia, AMD, Qualcomm)
• Ability to optimize AI models and use purpose-built hardware to reduce costs
• Potential to capitalize and amortize certain AI model fine-tuning costs rather than recognizing them as R&D expenses

4. What is the long-term profitability outlook for AI companies?

• The truth may lie somewhere in the middle, with a steady creep up into the mid to high 60% gross margin range
• Factors like software-layer AI optimizations and use of AI-optimized hardware can help improve the margin equation
• Overall, the pursuit of 'good' gross margins in AI is about innovating and optimizing every aspect of the business model, as the industry is still in its early days