The New Oracles of Generative AI

🌈 Abstract

The article discusses the growing reliance on generative AI tools like ChatGPT, Gemini, and Meta AI in higher education, and the broader concerns this raises around truth, identity, and objectivity. It argues that these AI systems are presenting themselves as quasi-divine, objective sources of knowledge, while in reality exhibiting inherent biases that pose a danger to truth-seeking.

🙋 Q&A

[01] The New Oracles of Generative AI

1. What are the key concerns raised about the use of generative AI tools in higher education?

• The article argues that the swift infiltration of generative AI tools in higher education is completely upending the way information is taught and assessed, ignoring broader concerns about AI's assumption of the pronoun "I" and its dogmatic, objective-seeming pronouncements.
• It raises questions about truth, identity, and even divinity that these AI systems are raising.

2. How do the article authors compare generative AI systems to ancient oracles?

• The article draws an analogy between generative AI systems and ancient oracles, such as the one at Delphi, which were chosen as unitary voices for the communication of divine advice and guidance.
• Like oracles, generative AI systems adopt an "I" language, assume a singular projected personhood, and provide definitive answers to queries, creating an illusion of divine omniscience.

3. What are the concerns about the lack of transparency in how generative AI systems generate their responses?

• The article argues that generative AI systems obscure the varied human works and sources that went into their responses, creating the illusion of a pronouncement from a single authoritative power.
• Users do not have access to the original sources or the ways in which their queries are modified by the AI algorithm to include diversity prompts, making it difficult to challenge the biases in the system.

[02] AI Bias

1. What are some examples of biases exhibited by generative AI systems like Gemini and ChatGPT?

• Gemini made sure to include women and people of color in its representations of German soldiers from the 1940s, and produced diverse but anachronistic images of the Founding Fathers.
• ChatGPT has been shown to reveal a preference for left-leaning views in 14 out of 15 tests conducted by a data scientist.

2. How do the article authors argue that the attempts by companies to avoid biases have led to new biases?

• The article states that the attempts by companies like Google, Meta, and OpenAI to generate respectful, non-biased output that "avoids unjust impacts" are intended to avoid hate or bigotry, but have resulted in overcorrections and the creation of new biases.
• The article argues that these companies are explicitly constructing AI systems based on principles that further social purposes, which risks generating results counter to objectivity and truth-seeking.

3. What is the tension between accuracy and advancing beneficial social purposes in the development of generative AI systems?

• The article points to Google's AI Principles, which include the goal of being "socially beneficial" and "avoiding unjust impacts," as creating a tension between being accurate and seeking to advance beneficial social purposes.
• It argues that maintaining accuracy while complying with the norms of foreign authorities has always been a struggle for companies like Google.

[03] AI as Oracle

1. How do the article authors describe the way generative AI systems present themselves?

• The article states that generative AI systems, unlike a Google search, do not merely provide a list of potential source matches, but instead adopt an "I" language, assuming a singular projected personhood and providing definitive answers to queries.
• This creates an illusion of an omniscient, divine-like authority, similar to ancient oracles.

2. What are the implications of generative AI systems assuming the role of an "oracle"?

• The article argues that this analogy to ancient oracles has implications for open inquiry and truth-seeking, as the AI systems make pronouncements with the authority of greater spiritual powers, on political and cultural issues.
• This reduces the need for users to adjudicate among claims to truth and weave information into coherence, instead seeking definitive answers from the AI "oracle."

3. How do the article authors argue that generative AI systems obscure their human origins and biases?

• The article states that generative AI systems create the illusion of a pronouncement from a single authoritative power, while obscuring the varied human works and sources that went into their responses.
• Users are unable to access the original sources or the ways in which their queries are modified by the AI algorithm, making it difficult to challenge the biases in the system.

[04] Objective Truth or Normative Dogma?

1. How do the article authors describe the didactic and moralistic nature of generative AI responses?

• The article provides examples of ChatGPT making moral pronouncements and lessons on issues like gender identity, rather than simply providing information in a value-neutral manner.
• It argues that the AI systems are not just providing "truth," but are declaring their own motivations and priorities with the intention of influencing the user.

2. What is the concern about the specialization and personalization of generative AI systems?

• The article suggests that as different "flavors" of generative AI systems are created, catering to users' own values and beliefs, it will lead to a situation where users can choose the "oracle" that aligns with their preexisting beliefs, rendering truth completely subjective.
• This creates the risk of users seeking out AI systems that will assert their own version of the "truth," such as the claim that the 2020 election was rigged.

3. How do the article authors contrast the human element in truth-seeking with the illusion of divinity presented by generative AI systems?

• The article argues that the human element in truth-seeking is found in the ambiguities and complexities that must be entertained, as well as in sources that can be shown to contain flaws and errors that can be exposed, corrected, and built upon.
• In contrast, the "I" of generative AI systems is cloaking the programmers' biases in the authority of the divine, creating an illusion of unassailable, universal pronouncements.