Considerations for AI Opt-Out

🌈 Abstract

The article discusses the considerations around creating an opt-out mechanism for AI models to use copyrighted content from the internet. It covers the following key points:

🙋 Q&A

[01] Considerations for AI Opt-Out

1. What are the key issues with using robots.txt as an opt-out mechanism for AI models?

• Robots.txt allows targeting directives to bots by path on the site or by User-Agent, but this may be inadequate for controlling AI model ingestion as it requires enumerating each individual AI model.
• This could lead sites to simply disallow all bots, which would negatively impact the web ecosystem by making it harder to introduce new crawler-based services.
• Alternatives like defining a special User-Agent for AI crawlers or creating a new well-known location (e.g. /.well-known/ai.txt) could address these issues.

2. How does the issue of previously crawled content differ between search engines and AI models?

• Search engines have an interest in keeping their index up-to-date, while AI models retain value from content ingested even if it is no longer available on the web.
• This means content owners may not have recourse if they weren't aware of the AI crawler at the time of ingestion.
• One potential solution is to state that the opt-out policy applies to any use of content obtained from a URL, regardless of when it was obtained.

3. What issue arises with robots.txt in terms of control of metadata?

• Robots.txt is controlled by the site administrator, meaning platform owners like Facebook or GitHub would control the opt-out policy for their users' content.
• To avoid this, users need to be able to express their preferences directly in the content itself, so the policy persists regardless of where the content ends up.

[02] What's Next?

1. What are the potential implications of the European legislation changing copyright to an opt-out model for AI?

• This shifts the balance of power between AI companies and content owners, making it important to offer content owners a genuine opportunity to opt-out.
• The technical details of the "appropriate manner" for opting out can significantly influence this power dynamic.

2. Where might the work on standardizing an opt-out mechanism happen?

• The article suggests the IETF mailing list on AI control as a potential forum for discussing these topics, as worldwide standards should be developed in open international standards bodies rather than regional fragmentation.