Unlocking the Potential of Lightweight Quantized Models for Deepfake Detection

1 Introduction

With the advancement of deep learning [Tao et al., 2021; Wei et al., 2020; Wu et al., 2024; Tao et al., 2022], recent progress in deepfake detection [Liu et al., 2023; Tan et al., 2025; Le and Woo, 2024; Tao et al., 2025a] has increasingly relied on large-scale models, including convolutional neural networks (CNNs) and transformer-based architectures, to achieve stateof-the-art performance. These models [Tao et al., 2025b; Le and Woo, 2024] excel at capturing intricate patterns within data, allowing them to detect even the most subtle manipulations. However, their reliance on computational resources for both training and inference often brings difficulties in deployment on resource-constrained environments, such as edge applications or real-time scenarios [Asnani et al., 2023;

∗Corresponding author.

Vice et al., 2024; Ding et al., 2023; Zeng et al., 2024]. The rise of AI-generated content on social platforms [Wu et al., 2023] necessitates the development of efficient detection systems to have low latency and computational requirements, no matter deploying on the cloud to process vast volumes of data, or carrying on the edge devices to safeguard the end users. Addressing the challenge requires minimizing the size and computational complexity of deepfake detection models without sacrificing performance. A promising approach is model quantization technique, which reduces the data precision and compacts the model while keeps the accuracy , paving the way for the deployment of deep learning models on resource-constraint scenarios.

The simplicity of the binary classification task suggests possible redundancy , leaving room for compression. As illustrated in Figure 1, we compare CLIP-ViT-L with 124M parameters, ResNet-50 with only 25.6M parameters and our model across three testing scenarios: seen domains generated by the same generator as the training data (light green) and unseen domains generated by other GAN generators (dark green) and Diffusion models (blue). The figure shows that smaller models can also do the same job. This raises the question: Do we really need to rely on large models for deepfake detection, especially on resource-constrained edge devices?

In this paper, we propose a novel quantization framework to compress deepfake detection models while maintaining high performance. We first identify and validate the effectiveness of the NPR (Neighboring Pixel Relationships) representation as a quantization-friendly alternative compared to original RGB images, which displays better compatibility to quantization errors, ensuring the feature consistency with low-bit representations. Based on this, we design a quantized model equipped with the Connected Quantized Block (CQB), which extracts forgery features by combining method-specific textures from full-precision shortcut connections with common features captured in the quantized main path. The auxiliary connections in CQB help preserve finegrained details, effectively addressing the distortion caused by quantization. Additionally, we introduce the Shifted Logarithmic Redistribution (SLR) quantizer to handle the unbalanced distribution of intermediate activations. By unfolding the negative ranges, SLR enhances the representation of forgery-related features and reduces information loss resulting from coarse quantization granularity. Together, these innovations form a robust framework for efficient and accurate deepfake detection.

Our carefully designed quantization framework achieves a substantial reduction in computational overhead by 10.8× and model storage by 12.4× without compromising performance. Notably, it outperforms state-of-the-art deepfake detection methods such as CNNDetection [Wang and others, 2020], LGrad [Tan et al., 2023], FrePGAN [Jeong and others, 2022c] using less than 5% of the FLOPs. The results validate the hypothesis of parameter redundancy in deepfake detection models and strongly demonstrates the potential of lightweight models. Beyond reducing memory footprint and inference latency, our framework enhances the ability to extract forgeryspecific features across various generators, paving the way for efficient and scalable deepfake detection systems, especially for resource-constrained environments. The main contributions are summarized as follows:

• We are the first to explore quantization in deepfake detection. Through extensive experiments, we demonstrate the potential of smaller models.
• We propose the Connected Quantized Block to complement method-specific fine textures into universal artifacts , compensating for the details.
• We introduce the Shifted Logarithmic Redistribution Quantizer, unfolding the clustered and unbalanced activations to mitigate the loss from coarse quantization.
• Our quantized deepfake detection model achieves a remarkable 10.8× and 12.4× reduction in computation and model storage, while maintaining or even surpassing the accuracy of full-precision counterparts.

2 RelatedWork

2.1 Deepfake Detection

Various strategies have been employed to enhance the generalization of detectors to unseen sources. These strategies include diversifying training data through augmentation methods [Wang and others, 2020; Wang and others, 2021; Yin et al., 2024], adversarial training [Chen and others, 2022], reconstruction techniques [Cao and others, 2022; He and others, 2021], fingerprint generators [Jeong and others, 2022b], and blending images [Shiohara and others, 2022]. Specific methodologies such as BiHPF [Jeong and others, 2022a] amplify artifacts' magnitudes through two high-pass filters. FreGAN [Jeong and others, 2022c] addresses the overfitting of training sources by mitigating the impact of frequency-level artifacts through frequency-level perturbation maps. Ju et al.[Ju and others, 2022] integrate global spatial information and local informative features in a two-branch model. AltFreezing by Wang et al.[Wang et al., 2023a] leverages both spatial and temporal artifacts for Face Deepfake Detection. Approaches by Ojha et al.[Ojha et al., 2023] and Tan et al.[Tan and others, 2023] utilize feature maps and gradients, respectively, as general representations. DIRE by Wang et al. [Wang and others, 2023] introduces a novel image representation by measuring the feature distance between an input image and reconstruction counterpart, aiming to alleviate generalization issues.

2.2 Model Quantization

Quantization has been applied on various downstream applications as a widely-used compression technique, due to its practicability. Particularly in platforms with limited computational resources and low task complexity, such as monitoring camera, mobile apps, and IoT devices [Ding et al., 2024; Hern´andez et al., 2024; Wang et al., 2023b]. However, few works focus on deepfake detection. Lanzino et al. [Lanzino et al., 2024] utilizes binarization on deepfake detection to uncover manipulation traces in frequency and texture domains but reduce the computational overhead by binarized operations. However, since binarization is an extreme quantization technique, the accuracy drop is always inevitable, and the computational complexity and parameter storage can be further reduced, which we have compared in our experiments.

3 Method

3.1 ProblemDefinition

A widely adopted approach for deepfake detection involves training a binary classifier using supervised deep learning methods to differentiate between real and fake images. This task is predominantly deployed on the cloud due to the limited computational capabilities of user-end devices. With the rise in popularity of video streaming platforms, deploying deepfake detection models directly on edge devices can greatly lower detection costs and latency while improving user security by protecting against deception from AI-generated content. To enable deployment on edge devices, minimizing the model's computational cost and storage size is crucial. However, the impact of backbone network size on this task remains unexplored. To address this, we evaluate CNNDetection [Wang and others, 2020] with backbones of varying sizes, assessing its performance on the training data domain as well as its generalization capabilities to other domains generated by different models. As shown in Table 1, increasing the backbone network size does not yield significant improvements in generalization performance. This phenomenon has led us to consider: Do we truly need larger models to perform binary classification for deepfake detection tasks, particularly in resource-limited scenarios?

Opportunities and Challenges of Quantization. We apply quantization techniques to deepfake detection models and achieve promising results. Specifically, we quantize a ResNet-23 model to 8-bit, 4-bit, and 2-bit using LSQ [Esser et al., 2019], and further binarize it using ReActNet [Liu et al., 2020]. Results in Figure 2 reveal that the quantized models maintain comparable accuracy, especially when tested on the seen data domain generated by ProGAN. It suggests that deepfake detection models have redundancy in parameter bitwidth, leaving room for further compression by quantization techniques.

3.2 Preliminary: Quantization-friendly NPR Representation

To harness the benefits of quantization while addressing the issue of detail loss caused by rounding, we propose a tailored quantization framework for deepfake detection.

Specifically, we adopt NPR [Tan et al., 2024b] (Neighboring Pixel Relationships) as the representation of artifacts due to its robustness against quantization. NPR is an effective approach for detecting forged images by capturing upsampling artifacts, i.e., patterns generated by interpolation or transposed convolutions, which are commonly used in generative models. The definition of NPR is

where x is the up-scaled feature map. Therefore, NPR focuses on the relationships between neighboring pixels, highlighting that these local interactions are more likely to contain potential artifacts. By removing semantic information while preserving edges, patterns, and pixel relationships, NPR reduces the reliance on large receptive fields, enabling the network to distinguish between real and forged images by focusing on small-scale details and artifact patterns.

Generally, NPR smooths out minor differences and thus enhances contrasts, which is an fundamentally similar effect to quantization. It suggests us to verify the compatibility between NPR and quantization, particularly in preserving forgery-related details.

First, quantization maps continuous real numbers to a discrete space with N values (N = 2k , k is the bitwidth). For example, in integer quantization, we have predefined threshold set T , and the input data x can be quantized as

where ti = 1+2(i-1) 2k-1 ∈ T N is the threshold of i-th value and I = {1, . . . , 1} ∈ Z N is an all-one vector. �·� is the Kronecker delta function, which takes the value of 1 if the condition is met; otherwise, it takes 0. Methods like LSQ [Esser et al., 2020], N2UQ [Liu et al., 2022] and QIL [Jung et al., 2019] have a trainable threshold set T , allowing the model to learn optimal quantization parameters during training.

Since the representation space is limited, it is inevitable for quantization to smooth out fine details, which can be formulated as

where δ represents a small variation in the image feature adjacent to the pixel x, as the quantization granularity around x is greater than δ. Fortunately, when applying NPR(·) representation as pre-processing, the quantization within the adjacent variation of x can be

It shows the consistency of NPR and quantization in terms of smoothing minor changes in adjacent pixels. Furthermore, NPR concentrates the image into a limited color space, only preserving the outline of the figure. It facilitates quantization by finer-grained threshold set T to minimize quantization error and retain essential details.

We perform a quantitative analysis of the quantization error by calculating the MSE loss for each layer using the images and NPR representations of Motorbike class in ProGAN dataset. The results are plotted in Figure 3. It suggests that using the original image as input leads to larger quantization errors, particularly in the earlier layers. Although the quantization errors diminish in the later layers, the fine-grained features may already be disrupted.

To give an intuitive conception, we visualize the RGB image and NPR representation under full-precision and 4-bit quantization. As shown in Figure 4, RGB images under 4bit quantization experience noticeable changes in lighting and pixel intensity relationships, resulting in sharper visual contrasts. This quantization process alters the balance of brightness and shadows. These effects are likely to be misinterpreted as artifacts generated by AI models during deepfake detection. However, quantization applied to NPR preserves most of the information.

3.3 Framework Overview

As shown in the left part of Figure 5, our framework introduces a lightweight CNN module named the Connected Quantized Block (CQB), built upon residual structures. It consists of a quantized main path with basic CNN blocks and a full-precision auxiliary path using only pooling and sampling. The quantized path, trained with learnable lowbit quantization, captures general features, albeit with potential detail loss due to rounding. To compensate, the auxiliary path preserves fine textures via shortcut connections with minimal overhead, effectively mitigating quantizationinduced information loss. Details are provided in Sec. 3.4. To further address activation imbalance, where values cluster densely in certain regions, we propose Shifted Logarithmic Redistribution (SLR) (see the right of Figure 5). SLR redistributes compact negative activations, reducing quantization errors caused by uniform binning and recovering critical texture details for forgery detection. Its formulation is discussed in Sec. 3.5.

3.4 Connected Quantized Block for Forgery Feature Compensation

In this section, we first outline the construction of the quantized CNN block and demonstrates the detailed implementation of the shortcut connections.

When quantized to extremely low bit-widths, the representation capacity of the parameters becomes significantly constrained. For instance, in a convolution layer Conv(·) with k-bit, the integer space is limited to 2k discrete values. As k decreases, this discrete space becomes smaller, leading to restricted representation capabilities. To mitigate the limit, we enhance the original block with a few full-precision paths, preserving and restoring the lost details with minimal computational cost. This approach is hardware-friendly and efficient, as it does not introduce additional intensive computations (such as multiplications) in the auxiliary path.

Specifically, we introduce the Connected Quantized Block (CQB) based on the residual block. The expand and reduction in the full-precision path introduced by CQB can be flexibly adjusted through the scaling ratio s. A typical residual block comprises three types of quantized convolution layers: QConvdn for downsampling, Convup for upsampling and Conveq for those do not change the channel number, respectively. In our CQB, these layers are all quantized, and we add weighted full-precision shortcut connections besides the three layers. For downsampling layer Convdn , we take the average for every s input:

where xi is the i-th input at location C i ,Hi ,W i , with [·, ·, ·] representing the indices along with the channel, height and width dimensions. The connection of Convup is constructed by concatenation function to interpolate values and align with the output dimension of the convolution layer:

where repeatC(·, ·) means copied along channel dimension for input x by s times. As for Conveq , the connection is

α1, α2, α3 in above equations are learnable scalars to balance the quantized and full-precision passes.

We showcase feature visualization heatmaps to illustrate the differences in features captured by the two paths and demonstrate the effectiveness of combining learnable quantized blocks with full-precision shortcuts for deepfake detection. And the results can be found in Sec. 4.2.

3.5 Shifted Logarithmic Redistribution for Texture Unfolding

When quantizing the activations below 4-bit, we observe a significant accuracy drop due to the feature distortion caused by quantization. And the results can be found in Sec. 4.2.

Therefore, we propose a Shifted Logarithmic Redistribution (SLR) Quantizer for the unbalanced activation distribution, which unfolds the clustered activations near zero, assigning fine-grained quantization intervals using a nonuniform quantizer to better quantize the majority of values.

Ideally, a well-trained quantizer prefers to have the values equally distributed within the quantized grids to maximize the information entropy. However, if we use uniform quantizer to unbalanced data in Figure 7(a), there will be nearly 80% integers dealing with less than 5% of activations, while the rest 95% activations are coarsely fallen into few intervals. This inspires us to adopt non-uniform quantizer to preserve more information at post-activation function quantization, but the initialization and training of non-uniform quantizers are nontrivial. Therefore, we first apply an offset off for the input activation X to shift the range into the domain of the logarithmic function:

where min(·),max(·), | · |, are the minimum, maximum and absolute functions, ϵ ≥ 1 is a minimal offset to avoid steeper slopes of the logarithmic function in the range of (0, 1] (refer to the illustration on the right part of Figure 5). And τ is a layer-wise parameter, which is trained together with the whole network, making the quantizer flexible between layers.

Next, the SLR quantizer can be written as

where ln(·) means logarithmic function. By applying the offset, we leverage the gentler region of the logarithmic function to redistribute unbalanced parameters. Notably, the proposed SLR quantizer can be seamlessly integrated into a quantizer based on a lookup strategy. For Eq. (3), the threshold set T transformed into T ′ = {et0 -off, . . . , etN-1 -off}. By merging the offset and logarithmic operations into the set T, SLR quantizer introduces no extra computational overhead.

4 Experiments

4.1 Experimental Settings

Baselines: CNNDetection [Wang and others, 2020], Frank [Frank and others, 2020], Durall[Durall and others, 2020], Patchfor [Chai and others, 2020], F3Net [Qian and others, 2020], SelfBland [Shiohara and others, 2022], GANDetection [Mandelli and others, 2022], BiHPF [Jeong and others, 2022a], FrePGAN [Jeong and others, 2022c], LGrad [Tan et al., 2023], Ojha [Ojha et al., 2023] and FreqNet [Tan et al., 2024a]. Evaluation Metrics: Following common practices [Jeong and others, 2022a; Jeong and others, 2022c; Ojha et al., 2023], we use average precision (AP) and accuracy (Acc) as the evaluation metrics.

4.2 Preliminary experiments

Feature Visualization. We visualize the features for each quantized convolution layer as the Main Path and their corresponding full-precision connections as the Auxiliary Path, as shown in Figure 6. Each row has three images: the output of the shortcut connection and the quantized convolution, as well as the weighted combination of the two features. It can be observed that quantization captures the primary features, especially edge information. However, due to the rounding operations , many important information and details are lost. For instance, the background and the main subject are shown as the same color. Fortunately, the auxiliary path preserves much of the original information, particularly textures and key elements in the image (such as the bird's beak). By combining the features from both paths, the final feature map retains the outline of subject, the distinction to the background, and critical details and texture, achieving more robust and comprehensive representations.

Activation Visualization. We visualize the intermediate activations at post-activation function in Figure 7(a). Each individual channel of activation are represents by x-axis, and the y-axis indicates the value distribution, with darker colors indicating higher densities. It is evident that after passing through the activation function, a large proportion of activations concentrate near zero, with only a small fraction of values much greater than zero, resulting in an imbalanced distribution. After quantization, there is always only very few effective bits represent the majority of values, leading to substantial information loss. While in Figure 7(b), after adding an offset to the parameters and mapping them to the logarithmic space, the activations are flattened and balanced, creating a more uniform distribution, friendly for quantization.

4.3 Detection Performance

In this section, we show the accuracy performance of our proposed lightweight designs compared to existing deepfake detection models and algorithms in Table 2 for Cross-GANSources Evaluation on ForenSynths [Wang and others, 2020] and Table 3 for Cross-Diffusion-Sources Evaluation on UniversalFakeDetect [Ojha et al., 2023]. Among these, only ProGAN is tested on seen data, while others are unseen.

When on the seen data domain, i.e., images generated by ProGAN, the detection accuracy of the quantized model with proposed methods is comparable to that of the models without quantization. For example, in Table 2, when we apply 2/3-bit weight and activations, our proposed quantization models have nearly no accuracy loss. And for unseen data domain, our method shows its effectiveness across different GAN and Diffusion. Under Cross-GAN-Source Evaluation, our method performs best in accuracy which is 91.8% on average, and the 95.1% AP also surpasses most of the previous methods. Especially, our model has 2.7% and improvement in Acc compared to CLIP model, while only utilizing 5% parameters and much fewer FLOPs. Meanwhile, under CrossDiffusion-Source Evaluation, our method also achieves the best performance, which are 2.4% and 3.1% higher than the previous SOTA in Acc and AP on average performance.

4.4 Generalization Performance

To show the generalization ability on different backbones of our methods, we conduct experiments on different sizes of ResNets, i.e., ResNet-23/50 and MobileNet, which has a different fundamental architecture compared to residual blocks. We show the accuracy performance of the quantized models

(a) On different backbones (b) On different quantizers

Table 4: Efficiency of methods, backbones and quantized bitwidth. For low-bit quantized methods, we calculate OPs as theoretical computation complexity, while for full-precision models, we use FLOPs.

using naive quantization techniques (light blue/yellow bars), or equipped with our proposed methods (dark blue/yellow bars) in Figure 8(a). All the cases are tested on ForenSynths, and we only show the average Acc and AP for simplicity. The experimental results demonstrate that our proposed method consistently and significantly improves detection performance across both ResNet and MobileNet architectures, achieving more than a 20% improvement in Acc and a 15% improvement in AP. Notably, due to the inherently lightweight structure of MobileNet, directly applying conventional quantization methods results in severe accuracy degradation. In contrast, our method achieves remarkable improvements, boosting Acc by 25.6% and AP by 26.6%. These gains are mainly attributed to our CQB module, which enhances the information density of the MobileNet architecture.

Regarding different quantizers, we evaluated our quantization method with MinMax,LSQ [Esser et al., 2020], and N2UQ [Liu et al., 2022] quantizers. As shown in Figure 8(b), our SLR consistently achieves stable improvements across all quantizers, which demonstrates that by leveraging shift and logarithmic redistribution, SLR effectively mitigates the issue of unbalanced parameter distribution, thereby significantly enhancing the performance.

4.5 Efficiency Performance

We analyze the efficiency of various methods and backbones in Table 4, reporting theoretical GFLOPs and storage (MB).

The methods can be roughly categorized into two kinds depending on the data preprocessing. Methods like [Durall and others, 2020; Frank and others, 2020; Tan et al., 2024a] are based on frequency analysis, which may have relatively fewer model storage but increased FLOPs due to DCT computation. Other approaches use diverse backbones such as CLIP-ViT, ResNet, or EfficientNet. CLIP-based models are the most resource-intensive, with up to 162 GFLOPs and 88.3MB storage. With quantization, the FLOPs can be reduced by higher throughput computations, and the storage can also be compact by packing the low-bit parameters together. For instance, BNext-T [Lanzino et al., 2024] uses binarization to achieve 0.89 GFLOPs and 13.3MB, though at a cost of significant accuracy loss. Therefore, we adopt 2/3-bit quantization in our method, allowing us to use smaller models while maintaining the accuracy. Based on ResNet-23, our model achieves strong compression, requiring only 0.25 GFLOPs and 5.0MB under 2-bit, while maintaining competitive accuracy.

4.6 Ablation Study

We evaluate each proposed component in Figure 9 on CrossGAN-Source Evaluations. The baseline (blue bar) uses 2-bit N2UQ on RGB images. Replacing RGB with NPR (pink bar) significantly boosts performance by 7.4% in Acc and 9.3% in AP, highlighting NPR is more quantization-friendly due to its compact value range and finer quantization intervals.

Adding shortcut connections in CQB (brown bar) yields a further 7.5% Acc gain by preserving fine-grained texture through the full-precision auxiliary path. Finally, our SLR quantizer (purple bar), designed for post-activation imbalance, brings a 2.1% improvement. These results confirm that combining NPR, CQB, and SLR effectively enhances detection performance under low-bit quantizationacross different generation models.

5 Conclusion

In this paper, we focus on the lightweight deepfake detection models and propose low-bit quantization method. It includes a Connected Quantized Block (CQB), which preserves fine details and textures to compensate for the forgery features in quantized convolutional layers, and the Shifted Logarithmic Redistribution (SLR) Quantizer, which unfolds unbalanced activations to make them more quantization-friendly and reduce the information loss. The proposed quantized deepfake detection model achieves a 10.8× reduction in computation and a 12.4× decrease in model storage, while maintaining accuracy comparable to full-precision or larger models.

Acknowledgments

This work was supported by Beijing NSF (No. L242021), National NSF of China (No. U24B20179), the Talent Fund of BJTU (No. 2024XKRC047) and the Open Project of Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, Anhui University (No. MMC202406).