📖 DisDiff (Disrupting Diffusion)+Advdm
Liu et al. 2024.
ACM International Conference on Multimedia (MM ’24)
REF: Disrupting Diffusion: Token-Level Attention Erasure Attack against Diffusion-based Customization
Review about protect methods
GLAZE, AdvDM and Numwan’s thesis: employ invisible perturbations on personal images, protecting users from style theft and painting imitation.
-> Focuses on disrupting style extraction.
Anti-Dreambooth, SimAC, Dis-Diff: generate adversarial noises and protects personal identities from being used in fake image synthesis.
-> Focuses on disrupting personalized model training processes.
Comparison of DreamBooth, Anti-DreamBooth with DisDiff.
Analysis of varying timesteps
Framework
Key Technical Innovations:
Cross-Attention Erasure Module: DisDiff erases the model’s attention on “sks” token and dramatically distorts the model’s outputs.
Merit Sampling Scheduler: Adaptively modulates perturbation updating amplitude in a step-aware manner.
Cross-Attention Erasure loss:
Time-dependent function to adjust the perturbation updating steps in PGD adaptively.
Overall Projected Gradient Descent:
Performance
FDFR (Face Detection and Face Recognition): 評估模型防止人臉識別的能力
ISM (Identity Similarity Metric): 測量生成圖像與原始身份的相似度
FID (Fréchet Inception Distance): 評估生成圖像與真實圖像分佈之間的差異
BRISQUE (Blind/Referenceless Image Spatial Quality Evaluator): 評估圖像的無參考質量
