A National Security Insider Does the Math on the Dangers of AI

🌈 Abstract

The article discusses Jason Matheny, the CEO and president of the Rand Corporation, and his views on various technological and national security risks, including:

• Concerns about the potential misuse of AI and biological weapons, and the need to balance the risks and benefits of technologies that can create specialized knowledge
• The challenges of maintaining cybersecurity as AI-powered malware becomes more prevalent
• The moral and ethical implications of autonomous weapons, and the potential advantages they may provide to autocratic regimes
• The importance of understanding the U.S.-China relationship, including areas of competition and cooperation, and the effectiveness of export controls on advanced technologies

🙋 Q&A

[01] Matheny's Role at Rand

1. What are the three key areas Matheny wants to focus on during his tenure as CEO and president of Rand?

• Developing a framework for technological competition without a race to the bottom on safety and security, particularly in relation to China
• Mapping out a climate and energy strategy for the country that balances technological requirements, infrastructure, and economics
• Understanding the risks to democracy, both in the U.S. and globally, including the erosion of norms around facts and evidence in policy debates

[02] Biological Risks

1. What are Matheny's main concerns regarding biological risks?

• The potential for engineered pandemics or biological weapons, which could be more deadly than COVID-19
• The decreasing cost and increasing accessibility of the tools and materials needed to create biological weapons, while the time and cost to develop and distribute vaccines has remained relatively static
• The risk of intentional biological attacks by individuals or groups with mass-casualty intent, which could be enabled by the availability of AI-powered tools that could provide the necessary know-how

2. How does Matheny view the role of nature as a biological risk?

• Natural viral pandemics will continue to occur, and some may be worse than COVID-19, highlighting the need for societal resilience.

[03] AI Risks

1. What were Matheny's early concerns about the potential risks of AI?

• Matheny was concerned that as AI systems become able to generate highly specialized knowledge, including knowledge about weapons systems, this could enable the misuse of that knowledge, such as for creating malware or biological weapons.

2. Does Matheny foresee an "AI winter" or slowdown in AI development?

• Matheny does not believe an AI winter is likely, as there appears to be a positive feedback loop where increased investment leads to further investment and scaling up of AI capabilities.

[04] Autonomous Weapons

1. What are Matheny's views on the moral and ethical implications of autonomous weapons?

• Matheny notes that empirical studies have had mixed findings on the error rates and civilian casualty rates of autonomous weapons compared to human-operated weapons.
• He suggests that autonomous weapons may be more advantageous to autocratic regimes with weaker human capital and rule of law, rather than to democracies.

[05] China

1. Why is Rand increasing its investment in analysis on China?

• The U.S.-China relationship is one of the most important global competitions, and Rand wants to accurately assess the relative strengths and weaknesses of the two countries across various domains.
• Rand also wants to identify areas where the U.S. and China can collaborate, such as on non-proliferation, climate, and pandemic preparedness.

2. How effective are U.S. export controls on advanced technologies like Nvidia's GPUs in the long term?

• Matheny suggests it is difficult to determine the long-term effectiveness, as China may find alternative means to acquire the necessary technologies, or develop domestic substitutes that, while less advanced, may still be sufficient for certain capabilities.