HyQ: Hardware-Friendly Post-Training Quantization for CNN-Transformer Hybrid Networks

1 Introduction

Convolutional neural networks (CNNs), which are widely used in computer vision tasks, learn translation equivariance properties using small local receptive fields and thus have high computational and parameter efficiencies [He et al., 2016]. However, the local connectivity structure limits their ability to model global contexts and long-range dependencies in the data [Naseer et al., 2021]. Conversely, emerging vision transformers (ViTs) apply global self-attention mechanisms to capture non-local relationships between all spatial locations, achieving higher accuracy than CNNs owing to their large representational capacity [Dosovitskiy et al., 2020]. Although ViTs can acquire such a high selfattention capacity, they also increase computational complexity and parameter size [Liu et al., 2021a]. For example, the ViT-Base [Dosovitskiy et al., 2020] and Swin-Base [Liu et al., 2021a] models have 86M and 87M parameters, respectively. Accordingly, these models require massive DRAM access during inference, resulting in significant energy consumption[Nguyen et al., 2020b], making them unsuitable for deployment on resource-constrained mobile/edge platforms [Zhou et al., 2022; Nguyen et al., 2020a].

To address this limitation, hybrid models that combine CNNs and ViTs have recently achieved state-of-theart (SOTA) performance in various vision tasks [Mehta and Rastegari, 2021; Yang et al., 2022; Li et al., 2022]. To exploit the strengths of both CNNs and ViTs in CNN-transformer hybrid models, mobile convolution (MBConv) layers [Sandler et al., 2018] are used in the early stages to extract local features and then passed to transformer encoders to learn global representations [Mehta and Rastegari, 2021; Yang et al., 2022]. Although these hybrid models achieve SOTA performance in the trade-off between computation and accuracy, their high computational cost and memory footprint still impede the real-time inference of neural network (NN) models on resource-constrained mobile/edge devices. For example, MobileViT-s [Mehta and Rastegari, 2021] has 2G floating-point operations (FLOPs), which exceeds the typical mobile inference budget (<600MFLOPs) [Cai et al., 2019].

Quantization has become the most widely used approach for ensuring efficient deployment of NN models on resourceconstrained mobile/edge devices [Kim and Kim, 2021]. Among the various quantization techniques, quantizationaware training (QAT) compensates for accuracy drops by retraining a quantized model using a full training set [Jacob et al., 2018]. However, QAT complicates the training process and adds significant time costs. Specialized training techniques (such as straight-through estimators [Yin et al., 2019]) or hyper-parameter tuning may also be required. Conversely, post-training quantization (PTQ) compresses the high-precision (e.g., FP32) weights and activations of a pretrained NN into low-precision values (e.g., INT8) to accelerate hardware inference without requiring access to training data or retraining [Nagel et al., 2021]. This makes PTQ

highly useful when training data are unavailable or additional training is infeasible. However, applying PTQ to neural networks (NNs) without considering their characteristics significantly reduces accuracy, particularly for complex ViT models compared with CNNs [Liu et al., 2021b]. Based on in-depth analyses, new quantization techniques tailored to ViTs have recently been proposed, including FQ-ViT [Lin et al., 2021], PTQ4ViT [Yuan et al., 2022], I-ViT [Li and Gu, 2022], and NoisyQuant [Liu et al., 2023]. However, these methods are limited to standard transformer models and are difficult to apply to hybrid models containing convolutional blocks that have different characteristics than transformer models. The quantization solution for the hybrid model, Q-HyViT [Lee et al., 2023], determines the mixed quantization granularity and schemes for each layer using Hessian information. However, mixed granularity and schemes require complex implementation and dedicated kernels for inference acceleration. In addition, Q-HyViT is specific to the MobileViT architecture and consequently exhibits poor compatibility.

In this study, we propose HyQ, a novel quantization method for CNN-transformer hybrid models. First, we introduce quantization-aware distribution scaling (QADS) to eliminate large outliers caused by inter-channel variances in CNNs (i.e., MBConvs). We also propose approximating the computationally expensive softmax function in transformer blocks with an integer-only linear exponential function while considering hardware efficiency. As shown in Fig.1, we quantize various hybrid models, including MobileViT [Mehta and Rastegari, 2021], MobileViTv2 [Mehta and Rastegari, 2022], and EfficientFormer [Li et al., 2022], using the ImageNet-1k dataset [Deng et al., 2009] to evaluate HyQ. Using HyQ, the quantized MobileViT-s can achieve a 75% parameter size reduction with only a 0.39% accuracy drop decrease from the FP32 baseline, demonstrating SOTA results. Moreover, using hardware-friendly techniques, HyQ overcomes the limitations of previous research in terms of hardware usage. As a result of implementing the proposed linear softmax at the register-transfer level (RTL), the proposed linear softmax significantly reduces lookup table (LUT) and flip-flop (FF) usage without the utilization of digital signal processor (DSP), outperforming the existing second-order polynomial approximations. Our contributions can be summarized as follows:

• The QADS addresses the inter-channel variance challenges caused by depth-wise convolutions in MBConvs. Before quantization, QADS scales the original distributions with large outliers in the activations and weights in a quantization-aware manner.
• We propose a method to approximate the computationally expensive softmax, originally performed in FP32 precision, as a simple yet efficient linear function with integer-only computation.
• We demonstrate high hardware-friendliness and compatibility of QADS and linear softmax, emphasizing their practical and broad applicability.

2 RelatedWorks

2.1 CNN-Transformer HybridModels

ViTs using self-attention have significant interest in computer vision tasks owing to their remarkable performance improvements in various tasks, such as image classification, object detection, and segmentation [Dosovitskiy et al., 2020; Liu et al., 2021a; Liu et al., 2022]. Such advantages of ViTs over CNNs stem from the self-attention mechanism's ability to capture global contexts more accurately, whereas CNNs extract local features [Steiner et al., 2021]. However, ViTs are often computationally costly and require substantial training data to achieve such performance gains. Therefore, various CNN-transformer hybrid models have recently been proposed, achieving improved trade-offs between accuracy and efficiency by combining the high computational efficiency of CNNs and the global modeling capacity of transformers.

MobileViT [Mehta and Rastegari, 2021] was designed to combine the strengths of CNNs and ViTs in mobile vision tasks. MobileViT reduces model complexity while maintaining accuracy by selectively replacing certain convolutions in MobileNetV2 [Sandler et al., 2018]. Conversely, MOAT [Yang et al., 2022] uses a different approach, which combines mobile convolution and transformer blocks into a single block, simplifying the model architecture. Moreover, MOAT can handle tasks that require high-resolution inputs by converting global attention to window attention. EfficientFormer [Li et al., 2022] addresses the challenge of ViTs being slower than lightweight CNNs, which makes their deployment on mobile devices challenging, by using the dimensionconsistent design and latency-driven slimming method. Despite being well-designed, hybrid models still have considerable model sizes of up to 82M parameters [Li et al., 2022], requiring additional model compression techniques, such as quantization [Nagel et al., 2021] and pruning [Kim and Kim, 2023], for efficient deployment on resource-constrained mobile/edge devices.

2.2 Post-Training Quantization

Quantization is an approach to improve the memory and computational efficiency of NNs by reducing the high precision of parameters to low precision. Quantization techniques are divided into two categories: QAT and PTQ. To mitigate accuracy losses caused by quantization, the existing QAT research necessitates access to the full training dataset, followed by extensive retraining across multiple QAT epochs [Yin et al., 2019; Gysel et al., 2018]. However, retraining quantized networks becomes infeasible in scenarios where data access is restricted or hardware resources are limited. Given these challenges, there is a growing preference for PTQ methods that do not require additional retraining or data access. This shift is further motivated by the rising costs of fine-tuning increasingly complex ViTs and language models.

Several PTQ techniques for CNNs have been proposed. Data-free quantization [Nagel et al., 2019] proposed weight equalization to remove the large outliers of the weights and bias correction to compensate for the quantization error induced by the shifted output. OMSE [Choukroun et al., 2019] utilized the mean squared error (MSE) function to identify the scaling factor with the least quantization error. On the other hand, for transformer models, NoisyQuant [Liu et al., 2023] demonstrated that introducing minor random biases to activations can significantly reduce quantization errors. PTQ4ViT [Yuan et al., 2022] applied twin uniform quantization to postsoftmax and post-GeLU activations [Hendrycks and Gimpel, 2016a] and determined the optimal scaling factor using a hessian-guided metric. FQ-ViT [Lin et al., 2021] proposed a power-of-two factor that enables integer-only operations to address the significant inter-channel variance problem in LayerNorm. SmoothQuant [Xiao et al., 2023] relieves the difficulties of activation quantization by migrating channel-wise outliers in LLM activation to weights. OutlierSuppression [Wei et al., 2023] shifts the channel-wise center and scales channel-wise outliers. However, one significant limitation of these methods is their reliance on the specific structures of CNNs or transformers. In other words, the direct application of these techniques to hybrid models often results in significant accuracy drops owing to the distinct parameter distributions and layer characteristics of CNNs and transformers.

To address this, Q-HyViT [Lee et al., 2023] analyzed the activation distributions in hybrid models and then determined the optimal quantization granularity (per-tensor or per-channel factors) and scheme (asymmetric or symmetric) using Hessian-based analysis. However, the mixed granularity and schemes of Q-HyViT result in sub-optimal hardware efficiency and increased resource demands, making it less suitable for edge devices. Additionally, its applicability is confined to a specific MobileViT architecture, limiting its compatibility. In this study, we introduce HyQ, a hardwareoptimized quantization method specialized for the efficient deployment of various hybrid models on edge devices.

3 Hardware-Friendly PTQ for HybridModels

3.1 Overview of the Proposed Quantization

For the CNN-transformer hybrid model, the proposed quantization maps all layers with core operations represented by high-precision values (e.g., FP32) to low-precision values (e.g., INT8). We apply INT quantization to all operations, including stem, convolution, MatMul, FC, softmax, LayerNorm, and shortcut layers. In particular, for fully integer quantization, we fold the BatchNorm weights into the preceding convolution weights before quantization. In addition, we employ uniform quantization with fixed step sizes for efficient support of diverse hardware (i.e., GPU, FPGA, ASIC). For a b-bit unsigned uniform quantization, the quantized input Xq is defined as follows:

where Q(X) denotes the quantization function. The input X ∈ RC×H×W denotes a floating-point real-value input with channel C and H × W dimensions. In contrast, Xq ∈ RC×H×W denotes a b-bit integer value. The scaling factor s ∈ R1 denotes the step size of the quantizer, and the zero point z ∈ R1 denotes the real value zero as an integer.

First, we describe the overall flow of the proposed HyQ. Fig. 2(a) illustrates the QADS process. Initially, during the calibration step, an optimization process is conducted to determine the optimal QADS parameters using the MSE loss (as shown in the left side of Fig. 2(a)). Following the completion of calibration, the obtained QADS parameters are merged into the quantizer (as shown in the right side of Fig. 2(a)). Fig. 2(b) depicts the attention mechanism of the transformer. The left side depicts the conventional quantization method that processes softmax in FP32. In contrast, on the right side, the proposed linear softmax performs exponential operations on the INT32 input using only add operations.

3.2 Quantization-Aware Distribution Scaling

MBConv is a module used in various lightweight models (e.g., MobileNetV2 [Sandler et al., 2018], EfficientNet

[Tan and Le, 2019], and MobileViT [Mehta and Rastegari, 2021]) for efficient model usage on resource-constrained mobile/edge devices. It has recently been widely used in hybrid models that combine CNN and transformers to optimize the trade-off between accuracy and efficiency in both computations and parameters. However, the depth-wise convolution in MBConv causes significant inter-channel variance owing to its channel-wise operation [Kulkarni et al., 2021]. As a result, this leads to large outliers, making quantization considerably challenging. As shown in Fig. 3, the maximum values of input activation for the point-wise convolution layer of the first and second stage of MobileViT-xxs are 38.6 and 22.2, respectively. When min-max quantization is applied to such a distribution, most values are mapped to only a few quantization bins (indicated by yellow dotted lines), resulting in most values being quantized to zero. This implies that only a few bits were used. Moreover, incorrect quantization of the MBConv blocks in the initial layers of hybrid models results in significant quantization errors. These quantization errors accumulate gradually, leading to incorrect predictions in the final layers.

To address this issue, we propose QADS. The core idea of QADS is to scale down channels with large input values (i.e., outliers) to reduce the maximum value. This helps to map values more evenly to the quantization bin. First, we apply a per-channel scaling parameter α ∈ RC to the input activation X and weights W. The operation of the convolution layer using the scaling parameter α is expressed as follows:

where ̂X and ̂W denote the scaled input and weight, respectively. Convolution using the quantization function Q is defined as follows:

The new ̂W(=Wα) can be pre-computed offline, but X changes dynamically and cannot be pre-scaled like weights. Therefore, to avoid additional scaling operations on X, we absorb α for X into the quantization scale factor s as follows:

where s∗ = α · s denotes the new quantization scale factor. Hence, QADS uses channel-wise scaling factors while still achieving computationally efficient layer-wise quantization. These additional scaling parameters accounted for only 0.05% and 0.07% of the total number of parameters in the MobileViT-xxs and MobileViT-xs models, respectively, indicating a negligible overhead.

However, determining the optimal QADS factor α poses a significant challenge. If α corresponding to input channels is very large, quantization of the input X becomes easier while weight quantization becomes more difficult. Conversely, if α is very small, the large outliers in input Xmake input quantization more difficult. Therefore, the optimal α must be carefully determined by considering both input and weight values. To automatically explore α without requiring heuristic knowledge, we employ a training-based method using the following objective function to determine the optimal α∗ α∗ α∗ :

In other words, we search for α that minimizes the MSE between the quantized output of the scaled input and weights and the original FP32 output. As shown in Fig. 3(b) and (d), the resulting scaled activation distributions using the optimal α∗ have lower maximum values (i.e., 38.6 → 11.0, 22.2 → 9.98) and a more even mapping to quantization bins than the original distributions. In conclusion, the proposed QADS can significantly reduce quantization errors caused by activation. It is noteworthy that we have applied this approach to nonlinear functions, such as swish activation, addressing issues that SmoothQuant [Xiao et al., 2023] or OutlierSuppression [Wei et al., 2023] cannot.

3.3 Linear Softmax: LinMax

In the transformer, softmax is used to convert the input values into a probability distribution between 0 and 1, and is expressed as follows:

where i = 1, ..., k. The exponential (exp) function in softmax generally does not have linear properties like MatMul, making integer arithmetic operations infeasible (i.e., MatMul(s · Xq) = s · MatMul(Xq)); however, Softmax(s · Xq) ̸= s

Algorithm 1 Integer-only Linear Exponential

• 1: Input: q, s: quantized input and scale
• 2: Output: qout,sout: quantized output and scale
• 3: function Linear-EXP(q, s)
• 4: qln2 ← round(-ln2/s)
• 5: z ← round(q/qln2)
• 6: qp ← q - z · qln2
• 7: qb ← floor(b/s)
• 8: sL ← floor(a · s)
• 9: qL = q + qb
• 10: qout, sout ← qL << n- z, sL/2n
• 11: return qout, sout
• 12: end function

· Softmax(Xq)). Hence, enabling integer computations for softmax is challenging. To address this, I-BERT [Kim et al., 2021] approximated the exponential over a very small range (- ln 2, 0] using a second-order polynomial (i.e., ax2+bx+c) and processed it with INT32 precision to maintain network accuracy. However, INT32 multiplications consume substantial energy on hardware platforms, such as ASICs and FPGAs. For example, INT32 consumes energy ×15.5 and ×98 more than INT8 on ASIC and FPGA implementations, respectively [You et al., 2020]. Furthermore, I-BERT requires additional QAT training.

Considering the efficiency of implementing CNNtransformer hybrid models on hardware platforms, we approximated softmax using a lower-order polynomial (i.e., first-order) rather than the second-order polynomial used in I-BERT. Algorithm 1 presents the proposed integer-only linear exponential process. First, to ensure numerical stability, we restrict the range of the original exponential input by subtracting the maximum value as follows:

where xmax denotes max(xi). ˜xi = xi - xmax is a nonpositive value. We can then decompose ˜x as (-ln2)z + p, where z is the non-negative integer quotient of dividing ˜x by

-ln2 and p is the remainder in the range (-ln2, 0]. Then, the exponential for the input ˜xi can be expressed as follows:

where 2-z is efficiently converted to bit-shifting in hardware design. We seek approximation with a more hardwareefficient first-order polynomial (i.e., a linear function) than the second-order polynomial approximation in I-BERT. A naive approach would be to use a linear function that passes through the points (- ln 2, exp(- ln 2)) and (0, 1) within the range (-ln2, 0]. Alternatively, we can reduce the approximation error by narrowing the approximation range, such as using the range (-ln √2, 0]. However, these methods would incur significant approximation errors compared with secondorder polynomials. Therefore, rather than approximating the original exponential function, we use a smoothed exponential (e.g., exp(x)/8, exp(x)/16). For example, exp(x)/16

has a shape more akin to a linear function than exp(x) over (-ln2, 0]. We reformulate softmax by scaling the exponential by 1/16 as follows:

Then, the smoothed exponential for the input ˜xi can be expressed as follows:

where we approximate the smoothed exponential using a firstorder polynomial (i.e., ax+b) over (- ln 2, 0]. We determine the coefficients a and b by minimizing the L2 distance between the smoothed exponential and first-order polynomials. Finally, we obtain the following approximated linear function as follows:

where z = round(-˜x ln 2) and p = ˜x + z ln 2. Fig. 4 plots the results of the proposed linear softmax. The smoothed exponential and first-order functions are nearly identical. Particularly, the largest gap between these two functions over (-ln2, 0] is 1.89 × 10-3 . Compared with the second-order polynomial approximation in I-BERT, which has a maximum gap of 1.91 × 10-3 , our proposed method provides superior approximation while retaining the hardware implementation advantages. Furthermore, the first coefficient (i.e. 'α' in α(x+β)) is absorbed into the scaling factor s (in lines 8-9 of Algorithm 1). Therefore, the linear exponential needs only a simple and hardware-efficient operation of adding an integer value qb. It should be noted that the second-order polynomial exponential in I-BERT can also absorb the first coefficient, but it requires two addition operations and one multiplication operation in INT32 precision (i.e., α((x + β)2 + γ)).

4 Experiments

4.1 Experimental Environments

The proposed HyQ framework was validated using the ImageNet-1k benchmark dataset [Deng et al., 2009]. The hybrid models used in the experiments include MobileViT [Mehta and Rastegari, 2021], MobileViT-v2 [Mehta and Rastegari, 2022], and EfficientFormer [Li et al., 2022]. Additionally, ViT, DeiT, and Swin models are used as ViT-type models for the ablation study. We applied the INT8 uniform PTQ to all layers except the activation function (e.g., Swish [Ramachandran et al., 2017], GeLU [Hendrycks and Gimpel, 2016b]) and attention output (following FQ-ViT, we used a more challenging INT4 logarithmic quantization for the attention output). To optimize the QADS parameters, we used 100 unlabeled images from the ImageNet-1k training set. We performed 1,000 iterations with a batch size of 100 using the Adam optimizer [Kingma and Ba, 2014]. We used the PyTorch framework [Paszke et al., 2019] for all experiments and quantized the pre-trained models provided by the PyTorch Image Models library [Wightman, 2019]. Notably, applying QADS to MobileViT-xxs took only 2.5 minutes on a single NVIDIA GeForce RTX 3090 GPU.

4.2 Comparison with Other QuantizationMethods on MobileViT Models

In this subsection, we compare the performances of MobileViT on the ImageNet-1k dataset with other quantization methods. As shown in Table 1, the proposed method demonstrated SOTA performance for the quantized MobileViT. In particular, for MobileViT-xxs, our method mitigates a 0.95% accuracy drop compared with Q-HyViT [Lee et al., 2023] with the same quantization precision (i.e., W8A8). For larger MobileViT-s models, our proposed HyQ outperforms Q-HyViT, despite using fully integer quantization with hardware-friendly linear softmax. Furthermore, for MobileViT-xs, due to zero point overflow [Lee et al., 2023], existing quantization methods like OMSE [Choukroun et al., 2019] and Percentile [Li et al., 2019] exhibit severe performance degradation. In contrast, HyQ can substantially mitigate this issue by exploiting the fact that the minimum value of Swish used in MobileVit is fixed (-0.2785) when determining the scale and zero point for activation quantization.

4.3 Performance of HyQwith SOTAHybrid Models

In Table 2, we highlight the impressive performance of HyQ on SOTA hybrid models. For the MobileViTv2 model, we quantize all layers following the same approach as we employ for MobileViT. Notably, unlike Q-HyViT, we achieve a smaller accuracy drop even without resorting to reconstruction techniques based on the Hessian matrix. Moreover, for the EfficientFormer model, we consistently maintain the smallest accuracy drop across EfficientFormer-L1, L3, and L7 models even though we apply quantization to the convolution, softmax, pooling, LayerScale, and LayerNorm layers. In contrast, other methods exhibited variable performance depending on model size. Specifically, FQ-ViT suffered from unacceptable performance degradation in the smallest models, while Percentile and OMSE methods encountered significant accuracy drops in the largest models. These results show the robustness of the proposed HyQ approach across hybrid models of various sizes.

4.4 Performance Analysis of Linear Softmax

Table 3 presents the performance of various ViT models (i.e., DeiT-T/S, ViT-B/L, and Swin-T/S) as well as MobileViT series, applied with only the proposed linear softmax without QADS. We compare the accuracy of each model with I-BERT [Kim et al., 2021], which proposed a second-order polynomial approximation of the exponential function. As shown in Table 3, our well-approximated first-order polynomials achieve nearly identical accuracy to the second-order polynomial of I-BERT. Moreover, our proposed method outperforms I-BERT in terms of hardware implementation and computational complexity because it uses a first-order polynomial approximation and avoids expensive multiplications.

Based on these advantages, we validate the hardware efficiency of the proposed approach. To demonstrate that our proposed method is hardware-friendly, we compared the synthesis results of two approaches (i.e., the proposed linear exponential and I-BERT) designed in RTL using the XC7Z0101 chipset in the Zynq-7000 board at 125 MHz. In Table 4, unroll [Rahman et al., 2016] indicates how much the input of the exponential is pipelined, and LUT, FF, DSP, and Power are used as evaluation metrics. In detail, we used an input size of (4, 4, 64, 64) and the input data were unrolled for parallel processing. In addition, to reduce power, the data was loaded into BRAM and then processed. Experimental results show that our linear exponential uses fewer hardware resources (approximately 1.8 ∼ 2.1× LUT savings and 1.3 ∼ 1.9× FF savings) and consumes less power than the approximation method in I-BERT for all unroll factors. Importantly, our proposed method, which avoids INT32 multiplication, achieves this hardware efficiency without requiring DSP.

4.5 Ablation Study: QADS

We further analyzed the impact of QADS alone. The accuracies of MobileVit-xxs, xs, and s without QADS are 67.05%, 68.28%, and 77.67%, respectively. Conversely, with QADS, the accuracies become 68.15%, 73.99%, and 77.93%, respectively. These results show that MobileVit-xxs gains most from QADS, whereas MobileVit-s shows minimal improvement. This can be interpreted that the smaller the model, the more critical the impact of quantization of the backbone model, and it can be seen that QADS makes a significant contribution to minimizing this decrease in accuracy.

4.6 Ablation Study: Smoothed Exponentials

We demonstrate performance according to various smoothed exponential factors, specifically, exp(x) 1 , exp(x) 2 , exp(x) 4 , exp(x) 8 , and exp(x) 16 . This experiment was conducted using MobileViT-xxs on ImageNet without applying QADS to verify the effectiveness of linear softmax alone. The accuracies for smooth factors = 1, 2, 4, 8, 16 are 67.07%, 67.11%, 67.17%, 67.18%, 67.18%, respectively. We can observe that as the smooth factor decreases, the accuracy increases slightly and then saturates; thus, we set this factor empirically based on these findings.

4.7 Ablation Study: Robustness on Other Datasets

The robustness of the proposed HyQ model is further evaluated using the ImageNet-v2 and ImageNet-Sketch datasets to examine performance against minor distributional shifts and significant domain alterations, respectively. This evaluation demonstrates the capability of HyQ to preserve feature recognition and model efficiency in diverse conditions. We compared the Top-1 & Top-5 accuracy of HyQ with FQ-ViT[Lin et al., 2021] for MobileViT-xxs, xs, and s models. For QADS parameter optimization, we utilized 100 unlabeled images from the ImageNet-1k training set. As shown in Table 5, on the ImageNet-V2 dataset, we mitigated the Top-1 accuracy drop by 1.68%, 10.04%, and 0.21% for MobileViT-xxs, xs, and s, respectively, compared to FQ-ViT. On the ImageNetSketch dataset, HyQ exhibits slightly more accuracy degradation than ImageNet-V2 but still significantly outperforms FQ-ViT in reducing the accuracy loss.

5 Conclusion

In this study, we propose HyQ, a novel quantization technique for CNN-transformer hybrid models. HyQ adaptively employs QADS to handle outliers in a CNN and approximates softmax in the transformer as an integer-only linear function. Compared with existing methods, HyQ achieves significant performance gains in complex hybrid architectures. In particular, we demonstrate the superior hardware efficiency of HyQ by considering implementations on AI accelerators. Additionally, we verified the broad compatibility of HyQ by applying it to diverse hybrid models, and its high compatibility was confirmed by applying the proposed QADS and linear softmax to various CNN and ViT models, respectively.

Acknowledgments

This research was partly supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program (IITP-2024-RS2022-00156295) supervised by the IITP(Institute for Information & Communications Technology Planning & Evaluation and the Industrial Fundamental Technology Development Program (No. 20019367, Development of Low Power AI Architecture for AIoT) funded by the Ministry of Trade, Industry & Energy (MOTIE) of Korea.

Contribution Statement

Nam Joon Kim and Jongho Lee designed the algorithm and wrote the paper (equal contribution). Hyun Kim is the project leader who supervised this work.