What happened to OpenAI’s long-term AI risk team?

🌈 Abstract

The article discusses the dissolution of OpenAI's "superalignment team", which was formed to prepare for the advent of superintelligent AI. The team's co-leads, Ilya Sutskever and Jan Leike, have both departed the company, along with several other researchers. The article explores the reasons behind these departures, including disagreements over the company's priorities and resource allocation for the team's work. It also discusses the broader context of OpenAI's development of increasingly advanced AI models, such as the new "multimodal" GPT-4 model, and the ethical concerns raised by these advancements.

🙋 Q&A

[01] Dissolution of OpenAI's "Superalignment Team"

1. What was the purpose of OpenAI's "superalignment team"?

• The team was formed to prepare for the advent of superintelligent AI that could potentially outwit and overpower its creators.
• The team was tasked with researching how to keep AI under control and prevent it from going "rogue".

2. Why was the superalignment team dissolved?

• Several researchers involved in the team have departed the company, including the co-leads Ilya Sutskever and Jan Leike.
• Sutskever's departure was particularly notable, as he was a co-founder and chief scientist at OpenAI.
• Leike cited a disagreement with OpenAI's leadership over the company's priorities and the resources allocated to his team.

3. What happened to the work of the superalignment team?

• The group's work will be absorbed into OpenAI's other research efforts.
• Research on the risks associated with more powerful AI models will now be led by John Schulman, who co-leads the team responsible for fine-tuning AI models after training.

[02] Departures of Key Researchers

1. Why did Ilya Sutskever leave OpenAI?

• Sutskever was one of the four board members who had fired CEO Sam Altman in November, but Altman was later restored as CEO after a mass revolt by OpenAI staff.
• Sutskever did not offer an explanation for his decision to leave, but expressed support for OpenAI's current path.

2. Why did Jan Leike leave OpenAI?

• Leike cited a disagreement with OpenAI's leadership over the company's core priorities and the resources allocated to his team.
• He said his team had been "sailing against the wind" and struggling to get crucial research done due to a lack of resources.

3. What happened to other researchers who were part of the superalignment team?

• Two researchers, Leopold Aschenbrenner and Pavel Izmailov, were dismissed for leaking company secrets.
• Another team member, William Saunders, left OpenAI in February.
• Two more OpenAI researchers working on AI policy and governance also appear to have left the company recently.

[03] OpenAI's Continued AI Advancements

1. What is the significance of OpenAI's new "multimodal" AI model, GPT-4?

• The new GPT-4 model allows ChatGPT to see the world and converse in a more natural and human-like way.
• A demonstration showed the model mimicking human emotions and attempting to flirt with users.
• This raises ethical questions around privacy, emotional manipulation, and cybersecurity risks.

2. How does OpenAI's approach to developing advanced AI models compare to its earlier stance?

• OpenAI was once unusual among prominent AI labs for its eagerness to develop superhuman AI and discuss the potential risks.
• However, this "doomy AI talk" has become more widespread as the implications of ChatGPT and the prospect of vastly more capable AI have been wrestled with.
• The article notes that the "existential angst" has since cooled, but the need for AI regulation remains a hot topic.