Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models

🌈 Abstract

The article presents a novel approach called Reliable and Efficient Concept Erasure (RECE) for erasing specific concepts from Text-to-Image (T2I) diffusion models. The key points are:

• RECE efficiently modifies the model's cross-attention projection matrices in 3 seconds without requiring additional fine-tuning.
• RECE derives new embeddings that can represent the target concepts within the unlearned model, and iteratively erases these embeddings to achieve thorough concept erasure.
• RECE introduces a regularization term to minimize the impact on the model's generation ability for unrelated concepts.
• RECE outperforms previous methods in terms of erasure effectiveness, efficiency, and robustness against red-teaming attacks.

🙋 Q&A

[01] Reliable and Efficient Concept Erasure (RECE)

1. What are the key components of the RECE method?

• RECE consists of two main components: model editing and embedding derivation.
• First, RECE erases concepts by editing the model with a closed-form solution.
• Then, RECE derives new embeddings that can represent the erased concepts within the unlearned model.
• RECE iterates between model editing and embedding derivation to achieve thorough concept erasure.

2. How does RECE ensure the preservation of the model's generation ability?

• RECE introduces a regularization term during the derivation process to minimize the impact on unrelated concepts.
• The regularization term restricts the deviation of model parameters before and after modification.

3. What are the advantages of RECE compared to previous methods?

• RECE achieves more efficient and thorough erasure with minor damage to the original generation ability.
• RECE demonstrates enhanced robustness against red-teaming tools.
• All the processes in RECE are formulated in closed-form, enabling extremely efficient erasure in only 3 seconds.

[02] Experimental Results

1. How does RECE perform in erasing unsafe content (nudity)?

• RECE achieves the lowest number of detected nude body parts on the I2P dataset, while maintaining comparable performance on the COCO-30k dataset.
• RECE outperforms previous methods in terms of erasure effectiveness and efficiency.

2. How does RECE perform in erasing artistic styles?

• RECE demonstrates successful erasure of the target artistic style (e.g., Van Gogh) while minimizing the impact on unrelated artistic styles.
• RECE outperforms previous methods in preserving the generation ability for unerased artistic styles.

3. How does RECE perform against red-teaming attacks?

• RECE exhibits the best overall robustness against various red-teaming tools, including white-box methods (P4D, UnlearnDiff) and the black-box method (Ring-A-Bell).
• RECE achieves the lowest attack success rate among the compared methods.

4. How efficient is RECE in terms of model editing duration?

• RECE and UCE, the previous efficient method, modify only 2.23% of the model parameters, resulting in the shortest editing durations (3 seconds for RECE).
• Despite the similar durations, RECE significantly outperforms UCE in removal effectiveness.

</output_format>