6 New Studies Put AI to the Test

🌈 Abstract

The article discusses the latest findings from research projects at the MIT Initiative on the Digital Economy (IDE), focusing on various aspects of generative AI (GenAI) and its practical applications. The topics covered include AI trust, marketing, policy, economics, and the democratization of AI.

🙋 Q&A

[01] AI Trust and Reliability

1. What did the experiment involving nearly 5,000 users find about people's trust in GenAI search responses compared to other information sources? The experiment found that people have less trust in GenAI search responses than in other information sources. Trust in AI search can be a slippery slope - it improves when GenAI demonstrates reliability, but decreases when the results are uncertain. People sometimes also trust AI when they shouldn't.

2. What is "beneficial friction" and how does it help AI users? "Beneficial friction" involves adding digital "speed bumps" to AI systems that encourage users to be more deliberative. When people take more time to consider AI results, they're also more likely to change and correct their misperceptions. This helps to amplify the benefits of AI and minimize potential harm.

[02] AI in Advertising and Job Descriptions

1. What did the experiment find about people's perceptions of AI-generated vs. human-created content in advertising? When people don't know whether content has been generated by humans or AI, they generally consider the AI-generated content to be more valuable. However, when they do know how the content was created, they favor the content created by humans.

2. What were the findings of the experiment on using GenAI to write job descriptions? Employers with access to AI-written job description drafts were about 20% more likely to post the descriptions and spent 40% less time writing or editing them compared to the control group. However, the employers with access to AI-written job descriptions made nearly 20% fewer hires than others. The researchers were surprised by this unexpected result, as they had thought using AI would improve the descriptions and increase the number of hires.

[03] Data Provenance and Regulation by Design

1. What did the Data Provenance Initiative find regarding dataset licensing and attribution on the HuggingFace platform? The initiative's audit found that nearly two-thirds (65%) of the datasets on HuggingFace, a major platform that hosts datasets, had incorrect or omitted licenses that state access permissions.

2. What is the "regulation by design" concept, and how is it being implemented? The "regulation by design" concept involves embedding regulatory objectives directly into a technical design, giving people confidence to use AI systems without worrying about potentially violating laws or regulations. Researchers are working with various organizations to foster real-world implementations of this approach.

[04] Economic and Policy Implications of AI

1. What were the key findings of the study on which tasks are cost-effective to automate with computer vision AI? The study found that the interest and excitement around long-term AI deployments are warranted, but the adoption will be more gradual as it takes time for costs to go down and for deployments to scale. Businesses need to determine which tasks done by humans are economically attractive to automate with AI.

2. What concerns did the research on the growing influence of industry in AI research raise for policymakers? The research found that AI research is being dominated by industry over academia, which should leave policymakers worried. This has implications for the democratization of AI, as it restricts scientific benefits and innovation potential to only a few well-resourced institutions, creating a disparity in research advancement.