![magic star](/img/share-magic.png?1753566981)
Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models
๐ Abstract
The article presents a novel approach called Reliable and Efficient Concept Erasure (RECE) for erasing specific concepts from Text-to-Image (T2I) diffusion models. The key points are:
- RECE efficiently modifies the model's cross-attention projection matrices in 3 seconds without requiring additional fine-tuning.
- RECE derives new embeddings that can represent the target concepts within the unlearned model, and iteratively erases these embeddings to achieve thorough concept erasure.
- RECE introduces a regularization term to minimize the impact on the model's generation ability for unrelated concepts.
- RECE outperforms previous methods in terms of erasure effectiveness, efficiency, and robustness against red-teaming attacks.
๐ Q&A
[01] Reliable and Efficient Concept Erasure (RECE)
1. What are the key components of the RECE method?
- RECE consists of two main components: model editing and embedding derivation.
- First, RECE erases concepts by editing the model with a closed-form solution.
- Then, RECE derives new embeddings that can represent the erased concepts within the unlearned model.
- RECE iterates between model editing and embedding derivation to achieve thorough concept erasure.
2. How does RECE ensure the preservation of the model's generation ability?
- RECE introduces a regularization term during the derivation process to minimize the impact on unrelated concepts.
- The regularization term restricts the deviation of model parameters before and after modification.
3. What are the advantages of RECE compared to previous methods?
- RECE achieves more efficient and thorough erasure with minor damage to the original generation ability.
- RECE demonstrates enhanced robustness against red-teaming tools.
- All the processes in RECE are formulated in closed-form, enabling extremely efficient erasure in only 3 seconds.
[02] Experimental Results
1. How does RECE perform in erasing unsafe content (nudity)?
- RECE achieves the lowest number of detected nude body parts on the I2P dataset, while maintaining comparable performance on the COCO-30k dataset.
- RECE outperforms previous methods in terms of erasure effectiveness and efficiency.
2. How does RECE perform in erasing artistic styles?
- RECE demonstrates successful erasure of the target artistic style (e.g., Van Gogh) while minimizing the impact on unrelated artistic styles.
- RECE outperforms previous methods in preserving the generation ability for unerased artistic styles.
3. How does RECE perform against red-teaming attacks?
- RECE exhibits the best overall robustness against various red-teaming tools, including white-box methods (P4D, UnlearnDiff) and the black-box method (Ring-A-Bell).
- RECE achieves the lowest attack success rate among the compared methods.
4. How efficient is RECE in terms of model editing duration?
- RECE and UCE, the previous efficient method, modify only 2.23% of the model parameters, resulting in the shortest editing durations (3 seconds for RECE).
- Despite the similar durations, RECE significantly outperforms UCE in removal effectiveness.
</output_format>
Shared by Daniel Chen ยท
ยฉ 2024 NewMotor Inc.