IdentityMask: A Robust Face-Centric Privacy Protection Against Unauthorized Personalization of Diffusion Models
IdentityMask: A Robust Face-Centric Privacy Protection Against Unauthorized Personalization of Diffusion Models
Weiwei Tan, Rui Wang, Lihua Jing, Yanjun Zhang, Runbo Li, Leo Yu Zhang
Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence
Main Track. Pages 1676-1684.
https://doi.org/10.24963/ijcai.2026/187
Unauthorized personalization based on diffusion models pose a severe and growing threat to digital privacy by enabling the unauthorized replication and exploitation of individual identities. Existing disrupting-based defenses primarily add invisible perturbations arbitrarily across the entire image space to disrupt the generation process. However, we reveal that these methods fundamentally overlook the spatio-temporal dynamics of the personalization process, resulting in inefficient optimization that fails to sufficiently disrupt the core identity encoding mechanism. To mitigate these limitations, we propose IdentityMask, a robust protection framework that shifts the paradigm from arbitrary confusion to precise, targeted feature corruption. By anchoring the perturbation on subject-specific semantics and prioritizing the most critical diffusion timesteps, our framework ensures the disruption is maximized precisely where the identity is encoded. Additionally, a novel manifold projection strategy is introduced to embed the adversarial signals into the intrinsic structure of the image, rendering the protection resilient against state-of-the-art purification. Extensive experiments across diverse datasets, personalization techniques, and defense settings demonstrate that IdentityMask consistently outperforms prior state-of-the-art approaches in both protection efficacy and robustness.
Keywords:
Computer Vision: Adversarial learning, adversarial attack and defense methods
