Guide to SB 1047

🌈 Abstract

The article discusses the final form of California's SB 1047 bill, which aims to regulate the development of large AI models. It provides a detailed overview of the bill's key provisions, recent changes, and the rationale behind them. The article also addresses various arguments and objections to the bill, and concludes that it is the best light-touch bill that can be achieved.

🙋 Q&A

[01] What SB 1047 Does

1. What are the key things SB 1047 requires of developers of large AI models?

• If a model requires $100 million or more in compute to train, the developer must:
  • Create a reasonable safety and security plan (SSP) to prevent the model from posing an unreasonable risk of causing or enabling critical harms (mass casualties or $500 million+ in damages)
  • Publish a redacted version of the SSP, an assessment of the model's risks, and get yearly audits
  • Adhere to the SSP and publish the results of safety tests
  • Be able to shut down all copies of the model under their control if necessary
• The quality of the SSP and whether the developer followed it will be considered in determining if they used reasonable care
• If a violation causes or enables critical harms, the developer can be fined up to 30% of the model's training costs

2. How does SB 1047 handle fine-tuned models?

• If a model is fine-tuned using less than $10 million in compute, the original developer is responsible for it
• If a model is fine-tuned using $10 million or more in compute, the fine-tuner is responsible for it

3. What other key provisions does SB 1047 include?

• Compute clusters must do KYC (know your customer) checks on large customers
• Whistleblowers get protections
• The bill aims to establish a public "CalCompute" cloud computing cluster

[02] Changes to the Bill

1. What were some of the key changes made to SB 1047 during the legislative process?

• The standard was changed from "reasonable assurance" to the more relaxed "reasonable care"
• Harms must be caused or materially enabled by the developer's failure to take reasonable care, not just by the model itself
• Civil penalties are now limited to cases where there is actual harm or imminent risk
• The Frontier Model Division was eliminated, and the Frontier Model Board's role was expanded
• A permanent $10 million fine-tuning threshold was added

2. How do these changes impact the bill?

• The changes make the bill less stringent and impactful, in order to reduce its downside costs and satisfy objections
• However, the transparency and whistleblower protections remain important features of the bill

[03] Arguments and Objections

1. What are some of the key arguments made against SB 1047?

• Claims that the bill will cripple innovation and drive startups out of California, despite the bill only applying to a handful of the largest AI models
• Concerns that the bill will destroy open source AI models, despite the bill effectively exempting open models from its requirements
• Assertions that the bill's thresholds and requirements are arbitrary and will be lowered over time, despite the bill's provisions to prevent this

2. How does the article respond to these arguments?

• The article argues that the claims about harming innovation and startups are unfounded, as the bill only applies to a tiny fraction of AI models
• It contends that the bill actually provides an exemption for open models, rather than banning them
• The article explains the rationale behind the bill's thresholds and the safeguards in place to prevent them from being lowered

3. What is the article's overall conclusion about SB 1047? The article concludes that SB 1047 is "by far the best light-touch bill we are ever going to get" and that it represents a reasonable compromise that balances the need for AI safety with the desire to avoid overly burdensome regulation.