Towards Robust Privacy Protection via Anti-Personalized Diffusion Models

다국어 초록 (Multilingual Abstract)

Recent advances in diffusion models have enabled high-quality synthesis of specific subjects, such as identities or objects, across various domains. While this capability has opened up new possibilities in content creation, it also poses significant privacy risks, as malicious users could misuse personalization techniques to generate unauthorized images. While existing protection methods attempt to counter this by generating adversarially perturbed samples that disrupt personalization, they are vulnerable to common trans- formations like image compression and become ineffective if even a few clean images of the target subject are accessible. To address these challenges, we propose disturbing the diffusion model itself (rather than the images) with the targeted guidance that hinders the personalization of specific subjects, through our novel framework called Anti-Personalized Diffusion Models (APDM). To this end, we introduce a novel loss function, called Proxy- Target Guidance Loss, which effectively disrupts the personalization of the target subject from the model without loss of generative capability, along with text prompt augmenta- tion in the embedding space to strengthen protection. Moreover, we propose a new dual- path optimization strategy, coined Learning to Protect (L2P). By alternating between personalization and protection paths, L2P simulates future personalization trajectories and adaptively reinforces protection at each step. Experimental results demonstrate that our framework outperforms existing methods, achieving state-of-the-art performance in preventing unauthorized personalization.

목차 (Table of Contents)

• 1. Introduction 1
• 2. Related Work 4
• 2.1 Personalized Text-to-Image Diffusion Models 4
• 2.2 Protection against Unauthorized Personalization 4
• 3. Preliminaries 5
• 3.1 Text-to-Image Diffusion Models 5
• 3.2 Personalized Diffusion Models 5
• 4. Method 6
• 4.1 Problem Formulation 6
• 4.2 Analysis of Naïve Approach 6
• 4.3 Anti-Personalized Diffusion Models 7
• 4.3.1 Proxy-Target Guidance Loss 7
• 4.3.2 Learning to Protect 8
• 5. Experiments 11
• 5.1 Experimental Setup 11
• 5.2 Main Experiments 11
• 5.2.1 Protection Performance 12
• 5.2.2 Preservation Performance 13
• 5.3 Ablation Study 14
• 6. Conclusion 16
• Appendix 17
• A. Additional Experiments 17
• B. Generalizability of APDM 21
• C. Additional Qualitative Results 22
• D. Limitation and Future Work 22
• References 26