A discussion of discussions on AI bias

🌈 Abstract

The article discusses the issue of bias in large language models (LLMs) and generative AI systems, and how the reactions to such bias differ from "classical" software bugs. It examines various perspectives and arguments made in online discussions around examples of bias in AI systems, and explores the underlying causes and challenges in addressing these biases.

🙋 Q&A

[01] Reactions to AI Bias vs. "Classical" Bugs

1. What are the key differences in how people react to bias in LLMs/generative AI compared to "classical" software bugs?

• People frequently deny that output that is the opposite of what the user asked for is even a bug in the case of LLMs and generative AI, unlike with "classical" bugs.
• Common arguments made include:
  • The high incidence of Asian women in images generated by Stable Diffusion models is evidence that there is no bias against Asian women.
  • The bias is due to the training data being skewed towards white people, so it's not a bug but just a reflection of the data.
  • AI is just converting the input to the "average" or most common output, so it's not a bug.

2. How do these reactions compare to how people respond to bugs in "classical" software?

• For "classical" software bugs, there is much less denial that the behavior is a bug that needs to be fixed.
• For example, people would not argue that a scheduling software converting a tire change request to an oil change is not a bug.

[02] Underlying Causes of Bias

1. What are some of the potential underlying causes of bias in AI systems discussed in the article?

• AI systems are often trained on data that reflects societal biases, such as stock photos being dominated by white people.
• There is an assumption that training data should be representative of the US population, which leads to biases against non-white groups.
• The prioritization of shipping features quickly over ensuring quality and lack of robust testing processes allows biases to persist in deployed systems.

2. How does the author compare the issue of bias in AI to "classical" software bugs?

• The author argues that bias in automation is not a new problem, and has existed as long as automation has existed, but is now more legible to the public due to the widespread use of ML.
• Examples are provided of biases in compression algorithms, search engine indexing, and name handling that have existed for a long time but don't generate the same level of attention as AI bias.

[03] Proposed Solutions and Challenges

1. What are some of the common proposed solutions to address AI bias discussed in the article?

• Increasing diversity of teams working on AI systems
• Instilling a culture and processes that prioritize quality and catching biases

2. Why does the author argue these solutions are unlikely to be effective?

• Increasing diversity alone has not worked in the past to address pervasive software bugs and biases.
• The fundamental issue is that there are strong market incentives to prioritize shipping features quickly over ensuring quality, which makes it difficult to implement robust solutions.
• The author argues that without addressing these underlying incentive structures, proposed solutions are unlikely to be effective.