How to get from high school math to cutting-edge ML/AI: a detailed 4-stage roadmap with links to the best learning resources that I’m aware of.

🌈 Abstract

The article provides a 4-stage roadmap for learning the necessary math and machine learning skills to be able to read and understand cutting-edge ML/AI research papers. It covers:

1. Foundational math required for ML
2. Learning classical machine learning models
3. Understanding deep learning
4. Navigating the cutting-edge of ML/AI research

🙋 Q&A

[01] Foundational Math

1. What are the key math topics required for machine learning?

• The article states that the necessary math includes high school algebra, single-variable calculus, linear algebra, probability, statistics, and some multivariable calculus. It provides a detailed list of 242 individual math topics that are essential for machine learning.

2. How can one efficiently learn this foundational math?

• The author recommends using a structured course like the "Mathematics for Machine Learning" course, which can be completed in around 70 hours if the learner already knows single-variable calculus.
• Alternatively, free online resources can be used, but the author cautions that this approach requires piecing together content from various sources, which can be less efficient and more likely to lead to getting overwhelmed.

3. Why is this stage of foundational math learning so crucial?

• The author states that learning the foundational math is like achieving literacy - it opens up the world of further learning experiences in machine learning, whereas without this foundation, that world remains closed off.

[02] Classical Machine Learning

1. How does knowledge of classical machine learning help with understanding cutting-edge ML papers?

• The author explains that if you know the patterns and concepts from classical machine learning, such as loss functions and model training, you'll be able to intuitively grasp the high-level structure and meaning of equations and diagrams in cutting-edge ML papers, even if you don't understand all the details.
• This is analogous to how a musician with a strong understanding of musical theory and patterns can more easily improvise in a new song, or how an experienced software developer can quickly grasp the overall structure of unfamiliar code.

2. How can one effectively learn classical machine learning?

• The author recommends implementing streamlined versions of basic ML models from linear regression to small neural networks, rather than just passively consuming course material.
• Specifically, the author points to their own textbook "Introduction to Algorithms and Machine Learning" as a resource that focuses on this hands-on, implementation-oriented approach.
• After that, the author suggests following up with Andrew Ng's Machine Learning Specialization on Coursera to cover the full breadth of classical ML algorithms.

[03] Deep Learning

1. What makes the textbook "Understanding Deep Learning" a great resource?

• The author praises the textbook for being a "serious yet friendly" resource, with detailed content, visualizations, concrete examples/exercises, historical context, and references to current research.
• The book has received very positive reviews and the author believes it is an excellent way to learn deep learning for those with the necessary math and classical ML foundations.

2. What other resource does the author recommend for learning deep learning?

• After working through the "Understanding Deep Learning" textbook, the author suggests following up with Jeremy Howard's "Practical Deep Learning for Coders" course on Fast.AI, which provides hands-on, guided projects to solidify one's deep learning knowledge.

[04] Cutting-Edge ML/AI Research

1. What is the author's advice for navigating the "cutting edge" of ML/AI research?

• The author acknowledges that the cutting edge is more of a "zone" where guidance gradually fades, compared to a clear line between textbooks and the latest research.
• The recommendation is to start by collecting a set of foundational resources (papers, videos, blog posts) that can bring you from the beginning to the end of the cutting edge, rather than jumping straight to the latest research papers.
• Examples of such foundational resources include Andrej Karpathy's "Neural Networks: Zero to Hero" series and Jeremy Howard's "From Deep Learning Foundations to Stable Diffusion" on Fast.AI.

2. Does one also need software engineering skills for cutting-edge ML/AI research?

• The author confirms that software engineering skills are also important, in addition to the math and machine learning foundations covered in the roadmap.
• For those without prior software engineering experience, the author suggests starting by implementing the canonical data structures and algorithms covered in the "Introduction to Algorithms and Machine Learning" textbook.