Model Conversion via Differentially Private Data-Free Distillation

1 Introduction

The success of deep neural networks in a wide array of applications [Jia et al., 2019a; Jia et al., 2019b; Ye et al., 2020] greatly owes to the open source of massive models. However, a major problem is that the training data of these models often contain a large amount of sensitive information that can be easily recovered with a few access to the models [Fredrikson et al., 2015; Yang et al., 2019]. How to protect such private information while maintaining the model performance has attracted a lot of attentions. Differential privacy (DP) [Dwork et al., 2006] is a common technique for protecting privacy. [Abadi et al., 2016] guaranteed that the model was differentially private regarding the training data by clipping gradients and adding Gaussian noise to gradients. However, the model accuracy decreases severely when the privacy requirements increase so that the model can't be applied directly to the case where only the pretrained models are given. [Papernot et al., 2017; Papernot et al., 2018] proposed a semi-supervised learning framework called PATE to reduce the impact of the DP noise by leveraging the noisy aggregation of multiple teacher models trained directly on the private data. It is possible to train a privacy-preserving student model using PATE framework given only sensitive teacher models, but it is difficult to find a suitable unlabeled public dataset for the distillation process.

∗ Shiming Ge is the corresponding author (geshiming@iie.ac.cn). Supplementary material is available at: https://arxiv.org/abs/2304.12528

In the meantime, an independent line of research concerning model compression shows that some data-free knowledge distillation approaches (DFKD) [Chen et al., 2019; Zhu et al., 2021; Choi et al., 2020] could achieve similar performance by vanilla training with only a teacher model. The data used for the distillation process is generated by a generator, which could potentially be a remedy for the above problem of the suitable public dataset. The generators of such methods mainly learn the data distribution rather than the image details, which intuitively also provides a degree of protection of privacy. [Ge et al., 2023] has showed that: It is possible to leverage the power of data-free knowledge distillation to train a privacy-preserving student model that is not necessary to access to the original dataset. But there is still a long way to take advantage of this intuition.

Inspired by the above observations, in this paper we propose a model conversion approach with Differentially Private Data-Free Distillation (DPDFD) to facilitate the model releasing by distilling a pretrained model as teacher into a differentially private student. Specifically, our DPDFD combines DFKD and DP, which applies DFKD to distill private knowledge and a DP mechanism AC,σ to guarantee privacy. The objective is to enable an effective conversion that achieves strong privacy protection with minimum accuracy loss when only private models (teachers) are given. As shown in Fig. 1, we first generate massive synthetic data with a generator. Then, we feed the synthetic data into the teacher model and student model to compute the loss. Differentially private gradients are calculated by applying DP mechanism AC,σ. Finally, we update the student with these gradients and update the generator by taking the student as a fixed discriminator. In particular, we achieve DP by performing normalization on the gradients of student outputs and adding Gaussian noise to gradients during student learning. The reason for performing normalization instead of clipping is that it will retain the relative size information of the gradients and achieve better performance with a smaller norm bound. The reason for adding Gaussian noise to the gradients of student outputs is that it has a lower dimension compared to other gradients. Both smaller norm bound and lower dimension gradients make it easier to balance the performance and privacy. In addition, the DP mechanism AC,σ also ensures differentially private training of the generator according to the postprocessing mechanism. We can use the generator to generate data for other down stream tasks if needed. We also provide privacy and convergence analysis for our DPDFD in theory. Furthermore, DPDFD can be extended to multi-model case, which aggregates multiple sensitive teachers into a privacypreserving student model.

In summary, our DPDFD can effectively convert a sensitive teacher model to a privacy-preserving student model through three key components. First, performing normalization instead of clipping which is usually used in other approaches retains information about the relative size of the gradients. Second, achieving DP by adding noise on the gradients of lower dimensional outputs makes it easier to balance performance and privacy. Third, synthetic data generated by the generator has a similar distribution with the training data of teacher model. In this way, we can convert the sensitive model to a privacy-preserving student with minimum accuracy loss.

Our major contributions are three folds: 1) we propose a model conversion approach to convert a pretrained sensitive model into a privacy-preserving model for secure releasing; 2) we provide privacy analysis and convergence analysis for our approach in theory; 3) we conduct extensive experiments and analysis to demonstrate the scalability and effectiveness of our approach.

2 RelatedWorks

The approach we proposed in this paper aims to train privacypreserving student models by distilling knowledge from given sensitive model(s) via a differentially private data-free distillation. Therefore, we briefly review the related works for two inspects, including differentially private learning and datafree knowledge distillation.

2.1 Differentially Private Learning

Differentially private learning aims to ensure the learning model is differentially private regarding the private data. [Abadi et al., 2016] proposed a differentially private stochastic gradient descent (DPSGD) algorithm which achieved DP by clipping and adding noise to the gradients of all parameters during the training process. However, the model performance degrades severely with strong privacy requirements. [Papernot et al., 2017] later proposed PATE which used semisupervised learning to transfer the knowledge of the teacher ensemble to the student by using a noisy aggregation. It can reduce the impact of noise on performance by increasing the number of teacher models. However, it is difficult to find an unlabeled public dataset that has similar distribution to the training data of teachers. Some works want to train differentially private generators to generate data that has similar distribution to private data while preserving privacy. [Xie et al., 2018] applied DPSGD to the training process of Generative Adversarial Networks (GAN) [Goodfellow et al., 2014] to get a differentially private generator. [Chen et al., 2020] suggested that it is not necessary to clip and add noise to all gradients, but only needs to achieve DP in the back propagation process from discriminator to generator. [Cao et al., 2021] applied differentially private optimal transmission theory to train generators. [Chen et al., 2022] proposed an energyguided network trained on sanitized data to indicate the direction of the true data distribution via Langevin Markov Chain Monte Carlo sampling method. In this paper, we make a better balance between performance and privacy by applying normalization instead of clipping and post-processing of DP.

2.2 Data-Free Knowledge Distillation

Data-free knowledge distillation is a class of approaches that aims to train a student model with a pretrained teacher model without access to original training data. It uses the information extracted from the teacher model to synthesize data used in the distillation process. [Srinivas and Babu, 2015] proposed to directly merge similar neurons in fully-connected layers, which cannot be applied to convolutional layers and networks because the detailed architectures and parameters information is unknown. [Lopes et al., 2017] first proposed data-free knowledge distillation, using the distribution information of the original data to reconstruct the synthetic data used in the distillation process. [Deng and Zhang, 2021] used this approach to generate synthetic graph data for data-free knowledge distillation for graph classification tasks. [Zhu et al., 2021] modified and applied it to federated settings. [Chen et al., 2019] proposed a novel framework named DAFL for training the student model by exploiting GAN. It uses the teacher model as the discriminator to train a generator. [Choi et al., 2020] proposed matching statistics from the batch normalization layers for generated data and the original data in the teacher. It can make the generated data closer to the original data. [Fang et al., 2022] proposed FastDFKD that applied the idea of meta-learning to the training process to accelerate the efficiency of data synthesis. Inspired by these approaches, it is possible to convert a sensitive model into a privacy-preserving model without access to the original data.

3 Preliminaries

Here we will first provide some background knowledge about differential privacy. We then draw connections between the definitions and theorems we introduced here and our DP analysis on DPDFD later. The following definition explains how DP provides rigorous privacy guarantees clearly.

Definition 1 (Differential Privacy). A randomized mechanism A with domain R is (ε, δ)-differential privacy, if for all O ⊆ R and any adjacent datasets D and D′ :

where adjacent datasets D and D′ differ from each other with only one training example. ε is the privacy budget and the smaller it is the better, and δ is the failure probability.

[Mironov, 2017] proposed a variant R´ enyi differential privacy (RDP), which computes privacy with R´ enyi divergence. We review the definition of RDP and its connection to DP.

Definition 2 (R´enyi Differential Privacy). A randomized mechanism A is (λ, ε)-RDP with λ > 1 if for any adjacent datasets D and D′ :

Different from DP, RDP has a more friendly composition theorem and can be applied to both data-independent and data-dependent settings. It supports a tighter composition of privacy budget which can be described as follows: For a sequence of mechanisms A1, ...,Ai, ...,Ak, where Ai is (λ, εi)-RDP, the composition of them A1 ◦ ...Ai... ◦ Ak is (λ, ∑i εi)-RDP. Ai represents one query to the teacher model in our case. Moreover, the connection between RDP and DP can be described as follows:

Theorem 1 (Convert RDP to DP). A (λ, ε)-RDP mechanism A also satisfies (ε + log λ-1 λ - log δ+logλ λ-1 , δ)-DP.

To provide DP guarantees, we exploit the postprocessing [Dwork and Roth, 2014] described as follows:

Theorem 2 (Post-processing). If mechanism A satisfies (ε, δ)-DP, the composition of a data-independent function F with A is also (ε, δ)-DP.

4 Proposed Approach

4.1 ProblemFormulation

Given a sensitive teacher model ϕt with parameters θt, the objective is converting it into a privacy-preserving student model ϕs with parameters θs that does not reveal data privacy and has the ability to perform similarly to the teacher model. To achieve that, we introduce a differentially private data-free distillation approach. We first sample a batch of noise vectors z = {zi}Bi=1 and feed them into the generator ϕg with parameters θg to generate massive synthetic data D = {di}Bi=1 . Enter them into the teacher model and student model to calculate the loss LT (ϕt(θt;D), ϕs(θs;D)) and then calculate updated outputs of student ys to achieve DP with a differentially private mechanism AC,σ. Finally update the student by the loss LS(ϕs(θs; D), ys) . Thus, the converting process can be formulated by minimizing an energy function E:

where g = ∂LT (ϕt(θt;D),ϕs(θs;D)) ∂ϕs(θs;D) and γ is the learning rate. We suppress the risk of privacy leakage with DP mechanism AC,σ which normalizes the gradients with norm bound C and adds Gaussian noise with variance σ2 . We will introduce how the differentially private mechanism AC,σ protects privacy in detail in the following.

4.2 Our Solution: DPDFD

Student model distilled from a sensitive teacher model directly may lead to privacy leakage and another main problem is that we don't have access to the original dataset. Thus, we aim to convert the sensitive teacher model to a privacypreserving student model while having similar performance to the sensitive model in a data-free manner.

Unlike [Abadi et al., 2016] which requires clipping and adding Gaussian noise to gradients of all parameters, due to the post-processing of DP, we only need to perform normalization and add noise to the gradients of student's outputs to calculate new differentially private outputs. As shown in Fig. 1 and Alg. 1, we first sample a batch of noise vectors z = {zi}Bi=1 and feed them into the generator ϕg with parameter θg to obtain massive synthetic data D = ϕg(z). Then enter the synthetic data D into the teacher and student to compute the loss LT (ϕt(θt;D), ϕs(θs;D)). To get better results,

Algorithm 1 DPDFD

Input: Training iterations T, loss function LT ,LS,LG, noise scale σ, sample size B, learning rate γ, γs, γg, gradient norm bound C, a positive stability constant e

• 1: for t ∈ [T] do
• 2: Sample B noise samples z = {zi}Bi=1
• 3: Generate B synthetic samples D = {ϕg(θg; zi)}Bi=1
• 4: for each synthetic data di = ϕg(θg; zi) do
• 5: Compute loss LT (ϕt(θt; di), ϕs(θs; di))
• 6: Compute the gradient gi = ∂LT ∂ϕs(θs;di)
• 7: Normalize the gradient ¯ gi = C·gi ||g||2+e
• 8: end for
• 9: Add noise ˜ g = 1B ( B∑ i=1 ¯ gi + N (0, σ2C 2 I))
• 10: Compute differentially private outputs of student ys = ϕs(θs;D) -γ · ˜ g
• 11: Compute loss LS(ϕs(θs;D), ys)
• 12: Update student θt+1 s = θts -γs · ∂LS ∂θts
• 13: Compute loss LG(ϕs(θs;D))
• 14: Update generator θt+1 g = θtg -γg · ∂(LS+LG) ∂θtg
• 15: end for
• 16: return θs and θg

we treat argmax(ϕt(θt;D)) as the target labels and then calculate the distillation loss LT same as [Zhao et al., 2022]. After that we achieve DP by the mechanism AC,σ which can be described as follows:

and generate new differentially private outputs ys. Compared with clipping, normalization can achieve higher accuracy at smaller norm bound C. This is because when C is small, clipping makes the gradients lose their variability but normalization retains the relative size relationship of gradients. Another important point is that the smaller the C, the smaller the privacy budget consumed by each query to the teacher model, which is exactly what we want. So we choose a smaller C and normalization operation to get a lower privacy budget and better performance. Finally, we compute the loss LS(ϕs(θs; D), ys) and update the student model with it. This loss function can take many forms, and we use cross-entropy loss in our experiments. In order to make the distribution of the synthetic data closer to the private data and balance the classes of the synthetic data, we add an additional loss LG when updating the generator. It can be formulated as:

where α and β are the tuning parameters to balance the effect of three terms, and we set both of them to 1. The first term ℓ(·) is a cross entropy function that measures the one-hot classification loss, which enforces the synthetic data having similar distribution as the private data. The second term is the information entropy loss to measure the class balance of generated data. The third term uses l2-norm ||∗||2 to measure the activation loss, since the features that are extracted by the student and correspond to the output before the fully-connected layer tend to receive higher activation value if input data are real rather than some random vectors, where θs ⊂ θs is the student's backbone parameters. We update the student and generator alternately in our experiments. In the nutshell, the trained student and generator satisfy DP because the training process can be seen as post-processing, given that ys is differentially private results. Overall, DPDFD can be formally proved DP in Theorem. 3.

4.3 Convergence Analysis

To analyze the convergence of our DPDFD, we follow the standard assumptions in the SGD literature [Allen-Zhu, 2017; Bottou et al., 2018; Ghadimi and Lan, 2013], with an additional assumption on the gradient noise. We assume that LT has a lower bound L∗ and LT is κ-smoothness, which can be described as follows: ∀x, y, there is an non-negative constant κ such that LT (x)-LT (y) ≤ ∇LT (x)⊤(x-y)+ κ2 ||x-y||2 . The additional assumption is that (gr-g) ∼ N(0, ζ 2 ), where gr is the ideal gradients of LT and g is the gradient we compute as an unbiased estimate of gr. Then according to [Bu et al., 2022], in our case we have:

where χ is min 0≤t≤T E(∥gtr ∥), d is a constant number, L0 is the initial loss and F(·) results only from the normalization operation same as [Bu et al., 2022] and it won't affect the monotonicity of input variables. We simply set the learning rate γ ∝ 1 √T and the gradients will gradually tend to 0 as T increases.

4.4 Discussion

Extension toMulti-Model Ensemble. Our DPDFD can be extended to the case of multiple sensitive models. Different from the case of a single teacher that reduces the impact of DP noise by averaging a batch of gradients, the case of multi-model achieves it by averaging the gradients from given sensitive models. In particular, given multiple teacher models {ϕjt }nj=1 , we first sample noise vectors and generate massive synthetic data same as the case of a single teacher. For each data di, we enter it into multiple teacher models {ϕjt (θj t ; di)}nj=1 and student model ϕs(θs; di) and compute losses {LT (ϕjt (θj t ; di), ϕs(θs; di))}nj=1 . Then, we compute gradients gij = { ∂LT (ϕjt (θj t ;di),ϕs(θs;di)) ∂ϕs(θs;di) }n j=1 with the losses. After that, we normalize them with norm bound C and add noise to them to get differentially private gradients

an updated output of student yis = ϕs(θs; di)-γ · ˜ gi. Finally, we update the student and generator same as Alg. 1. In this way, we can aggregate multiple sensitive teacher models into a privacy-preserving student model.

Direct Training on Private Data. The proposed DPmechanism AC,σ is a key building block of DPDFD, and can also be applied to situations where private data is available. Given a private dataset {x, y}, we input the data x into the model and calculate the cross-entroy loss L(ϕ(θ; x), y). After that, we run the same procedures to achieve DP and update the model as Alg. 1 except we don't need to update the generator. In this way, we can train a privacy-preserving model with direct access to private data.

5 Experiments

In this section, we present the experimental evaluation of DPDFD for converting the sensitive model to a differentially private model with high performance.

5.1 Experimental Setup

Datasets. We conduct our experiments on 7 image datasets, including MNIST [LeCun et al., 1998], FashionMNIST(FMNIST) [Xiao et al., 2017], CIFAR10 [Krizhevsky, 2009], CelebA [Liu et al., 2015], PathMNIST [Yang et al., 2021], COVIDx and ImageNet [Deng et al., 2009]. We created CelebA-H and CelebA-G based on CelebA. CelebA-H and CelebA-G are classification datasets with hair color (black/blonde/brown) and gender as the label, respectively. COVIDx is a classification dataset for COVID.

Baselines. We perform compartments with 14 state-of-theart benchmarks, including 9 explicit approaches that training classifiers with generative data (DP-GAN [Xie et al., 2018], GS-WGAN [Chen et al., 2020], PATE-GAN [Jordon et al., 2019], DP-MERF [Harder et al., 2021], P3GM [Takagi et al., 2021], DataLens [Wang et al., 2021], G-PATE [Long et al., 2021], DPSH [Cao et al., 2021], DPGEN [Chen et al., 2022]) and 5 implicit approaches that training classifiers with differentially private learning (DPSGD [Abadi et al., 2016], TSADP [Papernot et al., 2021], TOPAGG [Wang et al., 2021], GM-DP [McMahan et al., 2018], DGD [Ge et al.,

2023]). To make the comparisons fair, we take the results from original papers or run the official codes.

Networks. We adopt several popular network architectures as our teacher models, including AlexNet [Krizhevsky et al., 2012], VGGNet [Simonyan and Zisserman, 2015], ResNet [He et al., 2016], WideResnet [Zagoruyko and Komodakis, 2017], DenseNet [Huang et al., 2016], MobileNet [Howard et al., 2017], ShuffleNet [Zhang et al., 2018], GoogleNet [Szegedy et al., 2015] and ViT [Dosovitskiy et al., 2020]. For VGGNet, we use 19-layer net with BN. For ResNet, we use 50-layer networks for ImageNet and 34layer networks for others. For WideResnet, we use 50-layer networks as teacher. For DenseNet, we use 161-layer networks with growth rate equals 24. For ViT, we use the same architecture as [Dosovitskiy et al., 2020]. For student model, we use 34-layer ResNet for ImageNet and the same network as DataLens for others.

5.2 Experimental Results

In this section, we compare DPDFD with 14 baselines and evaluate it on different networks to verify its effectiveness. We first compare the model performance of DPDFD and other state-of-the-art approaches. Then, we conduct experiments on ImageNet with different networks under different privacy budget. We show that our DPDFD is scalable and outperforms all baselines.

Comparisons with 9 Explicit Baselines. To demonstrate the effectiveness of our DPDFD, we conduct comparisons with 9 baselines under different privacy budget and report the results in Tab. 1. All approaches are under a low failure probability δ = 10-5 . We can see that our DPDFD achieves the highest performance under the same condition of privacy budget. In particular, when ε = 1, our DPDFD achieves an accuracy of 0.9512 on MNIST and 0.8386 on FMNIST, which remarkably reduces the accuracy drop by 0.0409 and 0.0716 respectively, while most of the other approaches fail to achieve an accuracy of 0.8000. It shows that our approach has the best privacy-preserving ability and minimal accuracy drop. Even for high dimensional datasets like CelebA whose dimensionality is about 16 times larger than MNIST, all approaches suffer from accuracy drop with respect to their counterparts under high privacy budget while our DPDFD still delivers the highest test accuracy. This demonstrates that our approach is also effective for high-dimensional datasets.

Comparisons with 5 Implicit Baselines. In addition, we conduct experimental comparisons with 5 implicit baselines on MNIST and CIFAR10. The results are shown in Tab. 2, where our DPDFD achieves the highest accuracy of 0.9512 and 0.8601 under the lowest privacy budget of 1.0 and 2.0. The main reason comes from that we use normalization instead of traditional clipping. In this way, the differentially private gradients obtained after AC,σ retain more information (relative size between gradients).

Evaluation on Open Source Networks. To demonstrate the scalability of our DPDFD, we apply it to convert several popular networks pretrained on ImageNet under different privacy budget ε. These teacher models are taken directly from the PyTorch model zoo. The results are shown in Tab. 3. We find that the effect of network architectures is sometimes greater than the effect of public teacher model accuracy when ε is small. In particular, when ε = 1, the student model reaches the highest accuracy of 0.4964 when the teacher model is ShuffleNet, but the accuracy of teacher model is 0.1187 lower than the highest ViT. We guess this is because the simpler the teacher model is, the easier it is for the generator to learn the distribution of its training data, resulting in faster convergence of the student model. As ε increases, the student model can learn more and more fully from the teacher model, so the effect of teacher performance dominates. The more complex the teacher model, the more information about the training data it contains. The student model learns more accurate predictions from the teacher and the generator learns a more similar data distribution, both of which lead to higher accuracy of the student model. We can see that at ε = 5, the accuracy of the student is positively correlated with the accuracy of teacher model and it has approached that of the teachers, which proves that our DPDFD is not only scalable but also effective to a variety of popular network architectures.

Evalution of Training Directly with Private Data. In this section, we conduct experiments to evaluate the application of our DP mechanism AC,σ to the case where private data can be accessed. We compare 3 state-of-the-art approaches which trained directly with private data (based on DPSGD mechnism): DPSGD, TOPAGG and GM-DP on MNIST and CIFAR10 under the same settings except for the DP mechanism. The results are shown in Tab. 4. We can see that our approach achieves the best performance on both the low-dimensional MNIST dataset and the high-dimensional CIFAR10 dataset. These results depend on two aspects. On one hand, we only need to compute a new differentially private output in the first step of backpropagation instead of adding noise to each gradient like other approaches. On the other hand, we use normalization instead of clipping, which retains more gradient information in the case of small norm bound C. So our approach can achieve better results.

5.3 Ablation Studies

After the promising performance is achieved, we further analyze the impact of each component in our approach, including normalization vs. clipping, norm bound, noise scale, and composition of loss function.

Normalization vs. Clipping. To study the effect of different operations on the gradients, we conduct experiments with 34-layer ResNet pretrained on MNIST and FMNIST as teachers and report the results in Fig. 2. Performing normalization is significantly better than clipping when C is small, and even clipping makes the student not converge. The advantage of normalization gradually decreases as C increases, and when it increases to a certain level (about 10-3 as Fig. 2 shows), clipping will be superior to normalization. This is because normalization retains the relative size information of the gradients while clipping retains the absolute size information.

Norm Bound. Norm bound is an important hyperparameter of our DP mechanism AC,σ. We conduct experiments to investigate how norm bound C affects the performance of student model. The results are shown as the red and green lines in Fig. 2. As we see, the student performs better when C is small. This is because normalization preserves information about the relative size of the gradients. Although smaller C will be more affected by DP noise, it will allow more training epochs. So the student performs better when C is small.

Noise Scale. Like normbound, noise scale is also an important hyper-parameter of our DP mechanism AC,σ. To study the effect of it, we conduct experiments and the results are shown in Tab. 5. We observe that the model performance gets better as noise scale σ increases from 20 to 100. This is because a larger noise scale consumes a smaller privacy budget per epoch, which allows the student model to learn more fully with a limited privacy budget. However, we find that there is a slightly worse decrease in model performance as noise scale increases from 100 to 500. This is because the gradients will be more broken with a larger noise scale, thus leading to a slightly worse performance. In practical applications, a tradeoff should be made based on the actual situation.

Composition of Loss Functions. To check the effect of loss terms in the training generator, we investigate how each component of loss function contributes to the performance of student with 34-layer ResNet as the teacher model. We evaluate how they impact the performance by adding or removing each component and the results are shown in Tab. 6. We note that the information entropy loss term is the most important component based on the results. This is because removing it will result in an imbalance in the classes of the synthetic data. Cross entropy loss contributes differently for different datasets, with 1% improvement on the MNIST dataset and 13% improvement on the FMNIST dataset. l2- normalization is also a useful term though less critical, contributing to the 2%-3% of the performance improvement as shown in Tab. 6. In summary, all three compositions have a positive effect on the performance of the converted model.

6 Conclusion

Public pretrained models in model zoos may pose the risk of privacy leakage. To facilitate model deployment, we proposed a differentially private data-free distillation approach to convert sensitive teacher models into privacy-preserving student models. We train a generator for approximating the private dataset without the training data and student networks can be learned effectively through the knowledge distillation scheme. In addition, we perform normalization on the gradients of student outputs and add Gaussian noise to them to guarantee privacy. We also provide privacy analysis and convergence analysis for DPDFD. Extensive experiments are conducted to show the effectiveness of our approach. In the future, we will explore the approach in more real-world applications like federated learning on medical images.

Acknowledgements

This work was partially supported by grants from the National Key Research and Development Plan (2020AAA0140001), and the Beijing Natural Science Foundation (19L2040).