Why We Urgently Need AI Control Systems

🌈 Abstract

The article discusses the need for "control systems" or tools that can help steer complex technologies, including AI, in a way that is useful, safe, and effective, without requiring the people leading and executing projects to have advanced degrees in AI or related fields. The author argues that as technology becomes increasingly complex, we need better tools to allow non-technical domain experts and leaders to participate in the design, development, and testing of these systems.

🙋 Q&A

[01] The Need for Control Systems

1. What is the author's perspective on approaching the AI control system question?

• The author argues that approaching the AI control system question from a perspective of fear is "clickbait" and that talking about machines being "smarter than us" without defining "smarter" is also "clickbait".
• The author suggests that we should instead approach the question of control from the perspective of how to make complex tech useful and who should be qualified to steer it.

2. What does the author mean by "control systems"?

• The author defines "control systems" as a way to effectively steer complex technologies (including AI and human-made) in a world that is getting increasingly complex.
• These would be "tools on the user-side" that give people the cognitive capacity to make sense of the complexity they face.

3. What is the author's view on the current state of solutions for controlling complex technologies?

• The author argues that too many of the current solutions are based on talk (e.g., policy, education, guidelines) rather than providing the necessary tools to allow non-technical domain experts and leaders to participate in the design, development, and testing of these systems.

[02] The Need for Non-Technical Leadership

1. What is the author's perspective on who should be leading the development of complex technologies?

• The author believes that the path to leadership in tomorrow's world should not require a STEM background, but rather a leadership background and the right tools for those leaders to build/buy the tech they need and control it so it works as they need it to.

2. What is the author's criticism of the current state of leadership in complex technology development?

• The author notes that the people in charge are never the "human-in-the-loop" and that there is a lack of control systems that allow non-technical, but domain-expert, senior executives to effectively participate in AI design, development, and testing.

3. Why does the author believe we need better tools for testing complex systems?

• The author argues that we need better tools for testing complex systems, such as AI, in order to build trust and be able to use them for anything important. The author notes that the lack of effective testing solutions is a reason why "attempts at 'enterprise scale GenAI without a human in the loop' keep dying not with a bang but with a whimper."