No One Knows How AI Works

🌈 Abstract

The article discusses the lack of interpretability and understanding of how neural networks, a key component of modern AI systems, actually work. It highlights the disconnect between the rapid advancement and widespread deployment of these "black box" AI systems and the limited ability of researchers to explain their inner workings.

🙋 Q&A

[01] The Ubiquity of Neural Networks

1. What are the key points made about the ubiquity of neural networks?

• Neural networks are used in a wide range of applications, including language models like GPT-4, image recognition, medical diagnosis, weather forecasting, space exploration, and military purposes.
• Neural networks are not new or niche - they are present in hundreds of scientific areas and consumer phone apps.
• People use neural networks daily, and neural networks are also used on people daily.

2. What is the author's perspective on the common "AI = neural networks" equivalence?

• The author acknowledges that while this equivalence is technically incorrect, it has become an accepted approximation, except among scientists who would object to such a "gross equivalence".

[02] The Black Box Problem

1. What is the "black box" problem with neural networks?

• Despite the widespread use of neural networks, they remain "black boxes" - their inner workings and the reasons for their behavior are not well understood, even by the best tools and researchers.
• When examining the files and parameters of a neural network, it is not clear how they actually recognize or perform tasks like identifying cat breeds.

2. How do leading researchers in AI interpretability describe the current state of understanding?

• Experts like Dario Amodei (Anthropic CEO) and researchers from OpenAI and Google DeepMind admit that they only understand a small fraction (e.g. 3%) of how neural networks work.
• The consensus is that "we don't understand how neural networks work", as stated plainly by researchers.

3. What is the author's perspective on this lack of understanding?

• The author expresses a sense of both unease and fascination with the fact that the most important invention of our time remains largely unintelligible.
• The author's curiosity is driven by a desire for scientific understanding, rather than fear of the technology.

[03] The Reasons Behind the Black Box Problem

1. What factors have contributed to the black box problem?

• The focus has shifted from explanatory theories to predictive tools and statistical models, driven by the utility and profitability of AI systems.
• Researchers are underfunded compared to the vast sums being invested in making AI systems more complex and opaque.
• There is a tension between the desire to understand AI and the commercial interests in keeping it a black box.

2. How does the author characterize the current state of AI development?

• The author suggests that we are "engineering an intelligence our intelligence can't reverse-engineer", as AI systems grow exponentially in scale and complexity.
• There is a growing "intellectual debt" in the form of unintelligible technology that is being used without wisdom or restraint.