Microsoft releases powerful new Phi-3.5 models, beating Google, OpenAI and more

🌈 Abstract

The article discusses Microsoft's release of three new AI models in its Phi series, designed for different tasks such as basic/fast reasoning, more powerful reasoning, and vision (image and video analysis). The models, Phi-3.5-mini-instruct, Phi-3.5-MoE-instruct, and Phi-3.5-vision-instruct, are available for developers to download, use, and fine-tune under a Microsoft-branded MIT License, and they demonstrate near state-of-the-art performance across various benchmarks.

🙋 Q&A

[01] Microsoft's New Phi Series Models

1. What are the three new Phi 3.5 models released by Microsoft?

• Phi-3.5-mini-instruct: A 3.82 billion parameter model designed for basic/fast reasoning tasks
• Phi-3.5-MoE-instruct: A 41.9 billion parameter model with a mixture-of-experts architecture for more powerful reasoning
• Phi-3.5-vision-instruct: A 4.15 billion parameter model for vision (image and video analysis) tasks

2. What are the key features and capabilities of each model?

• Phi-3.5-mini-instruct: Compact size, strong reasoning capabilities, competitive performance in multilingual and multi-turn conversational tasks
• Phi-3.5-MoE-instruct: Scalable AI performance, strong performance in code, math, and multilingual language understanding, outperforms larger models in specific benchmarks
• Phi-3.5-vision-instruct: Integrates text and image processing capabilities, suitable for tasks like image understanding, optical character recognition, chart and table comprehension, and video summarization

3. How were the models trained?

• Phi-3.5-mini-instruct: Trained on 3.4 trillion tokens using 512 H100-80G GPUs over 10 days
• Phi-3.5-MoE-instruct: Trained on 4.9 trillion tokens with 512 H100-80G GPUs over 23 days
• Phi-3.5-vision-instruct: Trained on 500 billion tokens using 256 A100-80G GPUs over 6 days

4. What is the licensing for the Phi 3.5 models? All three Phi 3.5 models are available under the MIT license, which allows developers to freely use, modify, merge, publish, distribute, sublicense, or sell copies of the software.

[02] Impact and Significance

1. How does the release of the Phi 3.5 series represent a significant step forward for Microsoft in AI development?

• The Phi 3.5 series offers multilingual and multimodal AI capabilities, empowering developers to integrate cutting-edge AI into their applications.
• By offering the models under an open-source license, Microsoft is supporting the open-source community and fostering innovation across commercial and research domains.

2. What are the potential benefits of Microsoft's approach in releasing these models?

• The open-source licensing allows for greater accessibility and customization of the models, enabling developers to tailor them to their specific needs.
• The strong performance of the models across various benchmarks suggests they can be valuable tools for a wide range of AI-powered applications and research.