Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantization

1 Introduction

With the explosive growth in advanced IoT applications, it is challenging to deploy deep neural networks (DNNs) on resource-constrained devices (e.g., MCUs and tiny NPUs) [Kwon et al., 2024; Lu et al., 2023; Aggarwal et al., 2024], which suffer from extremely limited memory (e.g., KB-level SRAM, and MB-level storage) and low computing speed [Wang et al., 2019; Zheng et al., 2022]. This results in a big gap between the computational demands and limited resources. Therefore, it is desired to compress DNNs with no or minor performance degradation for efficient deployment. To achieve this goal, one typical way is to utilize the lower bit-width to quantize the entire network for lightweight and acceleration, a.k.a, fixed-precision quantization (FPQ) [Banner et al., 2019; Bai et al., 2024], where single-precision floating point weights or activations are mapped to lower bit-width ones. The recent developments in inference hardware have enabled variable bit-width arithmetic operations for DNNs, and this leads to the emergence of mixed precision quantization (MPQ) [Cai and Vasconcelos, 2020; Sun et al., 2022; Ma et al., 2023; Dong et al., 2023; Liu et al., 2024; Sun et al., 2024b], which allows different bit-widths for different layers. Mixed precision of MPQ means a fine-grained bit-width allocation manner, where the quantization-insensitive layers can be quantized using much lower bit-widths than the quantization-sensitive layers. This way can naturally obtain more optimal accuracycomplexity trade-off than FPQ.

∗Corresponding author

Limitations of existing DMPQs. As a typical way of existing MPQ methods, differentiable MPQ (DMPQ) aims to learn the optimal bit-width assignment in an end-to-end manner [Cai and Vasconcelos, 2020; Zhang et al., 2021] (as shown in Fig. 1a). It applies continuous relaxation to transform the categorical choice of mixed precision quantization policy into continuous mixed precision quantization parameters [Yu et al., 2020]. In this way, the DMPQ is relaxed to a differentiable quantization search problem, and the bit-width associated with the largest magnitude of learnable bit-width parameters (quantization parameters, α) updated by gradient descent in each layer is selected to form the final mixed precision quantization policy. Such gradient-based bit-width selection process in DMPQ relies on an important assumption that the value of α updated by gradient descent represents the bit-width contribution to the accuracy improvement. However, little work focuses on the validity of the above assumption of DMPQ.

In this paper, we attempt to understand and overcome this problem. At first, we find that the largest magnitude of learnable bit-width parameters α updated by gradient descent does not truly reflect the bit-width contribution in many cases, which degrades the performance of the derived mixed precision quantization policy. In this work, this phenomenon is defined as the α's pitfall issue. Then, we observe that the bit-width operations in the quantization model are not independent of each other, which can be one potential reason of α's pitfall where the underlying relationships between bit-width operations are ignored.

If the learnable bit-width parameters updated by gradient descent are not an excellent indicator (e.g., for optimization learnable bit-width parameters (α)) of bit-width contribution, how to select optimal quantization policy for each layer during bit-width selection process? To tackle this issue, we propose a Shapley-based mixed precision quantization method, termed as SMPQ, which leverages Shapley value [Ancona et al., 2019; Castro et al., 2009; Sun et al., 2024a] to attribute real contributions to players in cooperative game theory. Fig. 1 shows the differences between our SMPQ and previous DMPQ methods. Shapley value directly measures the contributions of operations according to the validation accuracy difference. Meanwhile, it considers all possible combinations and quantifies the average marginal contribution to handle complex relationships between individual elements. Benefiting from these, Shapley value is effective for obtaining operation importance that is highly correlated with task performance. Instead of relying on the learnable bit-width parameters updated by gradient descent, the bit-width selection of SMPQ considers the actual contributions of bit-width operations to quantization performance, as shown in Fig. 1b, which can be described by "Where" and "How" to search optimal quantization policies. Contributions. The main novelties of our work include:

• We discover that DMPQ suffers from the issue that the learnable bit-width parameters updated by gradient descent fail to reflect the actual bit-width contribution in many cases, and then we conduct deep analysis on the potential reason for such issue.
• We propose a Shapley-based MPQ (SMPQ) method via capturing actual the contribution of bit-widths on the MPQ task of validation dataset. It is the first attempt in exploring enhanced bit-width selection of DMPQ.
• Extensive experimental results show that SMPQ consistently gets more optimal quantization policies than gradient-based counterparts. Furthermore, we find that SMPQ performs more efficiently than its comparators on

resource-constrained devices.

2 RelatedWork and Preliminaries

2.1 RelatedWork

Due to the page limit of the main text, the related work is provided in App. A.

2.2 Rethinking the DMPQMethod

Problem Setup. Suppose that a neural network F consists of n convolutional layers denoted as L1, ...,Ln. Each layer Ll (1 ≤ l ≤ n) has its corresponding set of weights Wl. The training process of the network F is conducted by solving an Empirical Risk Minimization problem. To quantize weights Wl and activations Al, we define a search space Sα and Sβ with nw and na bit-width candidates, respectively. To be specific, Sα represents the search space of weights, and Sβ denotes the search space of activations. To this end, the goal of MPQ is to find the optimal bit-width configuration for neural network F . Following the setting of method [Cai and Vasconcelos, 2020], for layer Fl, the quantization function is typically defined as follows:

where filter f is parameterized by weight tensor W T i , and z represents the filter input. Our goal is to find the optimal bitwidth configuration (o∗α and o∗β ) for the entire neural network F . Then, we can obtain:

where Y represents output of the quantized network. However, we find that the search space is huge. For instance, in the case of ResNet-101, the number of possible bit-width configures reaches 6101 = 3.9 × 1078 for each input when there are only 6 bit-width candidates. Therefore, a key challenge for the MPQ problem is how to reduce the search time.

The DMPQMethod. To reduce search cost, a representative DMPQ method, EdMIPS [Cai and Vasconcelos, 2020] is proposed and developed, and it can consume less than 10 GPU hours in ImageNet1K. The search space B of DMPQ is represented by a supernet, denoted as G = (V, E), where each node vi ∈ V represents a latent representation, and each edge (i, j) is associated with a bit-width o(i,j) . The core idea of DMPQ is to transform the selection of discrete bit-width operation into a continuous optimization problem via continuous relaxation, and then optimize the supernet with gradient descent. The intermediate node is computed as a softmax mixture of candidate bit-widths:

where α(i,j) o denotes the mixing weight of candidate bit-width o(i,j) . S represents the search space consist of Sα and Sβ . ¯ o is the input and mixed output of an edge. Given the optimal parameters α∗ , the mixed-precision network should be derived by discretizing the soft selector variables α(i,j) o into the binary selectors. With such relaxation, the DMPQ search can be performed by jointly optimizing the model weight W and learnable bit-width parameters α for their corresponding bitwidth candidate in a differentiable manner. In this work, we use a "winner-take-all" method to determine which bit-width is selected and others removed, computed as follows:

At the end of the search, the final mixed precision quantization policy is derived by selecting the maximum magnitude of learnable bit-width parameters α on every edge across all bitwidth choices. Notably, our analysis does not limit the scope of EdMIPS, it also can be generalized to other differentiable MPQ (i.e., FracBits-SAT, GMPQ).

3 The Magnitude-based Selection Pitfall Issue 3.1 Rethinking Actual Contribution of Bit-width in DMPQ

Despite the successful application of DMPQ in various model compression scenarios, it is demonstrated in the following that the gradient-based bit-width selection process of DMPQ does not accurately represent the actual contribution of the bit-width, i.e., the α's pitfall. As depicted in Eq. 3, the bitwidth selection process of the gradient-based in DMPQ relies on an important assumption that the largest magnitude of the learnable bit-width parameters updated by gradient descent signifies the contribution of the bit-width, i.e., the bit-width that corresponds to the highest probability is used as the optimal quantization strategy. However, due to the lack of comprehensive theoretical support for the rationality of the learnable bit-width parameters updated by gradient descent, we argue that selecting the bit-width based on the learnable bit-width parameters updated by gradient descent does not truly reflect its contribution, which leads to the α's pitfall issue (i.e., the magnitude-based selection pitfall). In the following section, we will substantiate our hypothesis with empirical evidence.

3.2 Experimentation on α's Pitfall of DMPQ

To verify the aforementioned issue, we first introduce search space S1 (consisting of weight bit {2, 3, 4, 5, 6, 7, 8} and activation bit {2}) to comprehensively explore relevant bit-width configurations and to facilitate the study of different aspects of quantization. Then, we conduct experimentation on learnable bit-width parameters updated by gradient descent does not represent actual bit-width contribution. This is achieved by calculating the discretization accuracy, which is obtained by using gradient descent to retrain quantized network 1 from the search space S1. Fig. 2 demonstrates that allocating small values of α updated by gradient descent to bit-widths can lead to high discretization accuracy at convergence.

These results indicate that the above α's pitfall issue results from the gradient descent process employed by DMPQ. To reveal the reason behind the α's pitfall issue, we argue that the α value from DMPQ is not always consistent with the corresponding contribution to the discretization accuracy in DMPQ. In fact, the contribution of bit-widths cannot solely be determined by the gradient's largest magnitude, and other factors such as the relationship of bit-width should be taken into account for accurate selection.

3.3 Further Analysis

Furthermore, we observe that the bit-widths in the quantization model are not independent but cooperative to each other during the learning of mixed precision quantization policy.

1 Such network is quantized via selecting a bit-width candidate for a layer while fixing involved bit-widths for other layers

Especially, as illustrated in Eq. 1, the differentiable learning strategy of DMPQ solely focuses on minimizing the loss function of learnable bit-width parameters (α), but ignores the effect of cooperation between different bit-widths on the quantization. In a sense, this can be one potential reason why DMPQ suffers from the issue of α's failure, where the underlying relationships between bit-widths are ignored.

To validate our observation, motivated by method [Abdolrashidi et al., 2021], we conduct further analysis to verify the issue, as shown in Fig. 3. To demonstrate the underlying relationships between bit-widths on different edges, we propose the search space S2, which consists of weight bit {1, 2, 3, 4} and activation bit {2, 3, 4}. To be specific, B0 uses ResNet18 (pre-trained in the ImageNet1K dataset) as the baseline model, and re-evaluates the discretization accuracy of the final mixed precision quantization policy based on EdMIPS [Cai and Vasconcelos, 2020] method. B1 changes the bit-width of the third edge from 4 bits to 3 bits of B0, and B2 changes the bit-width of the fourth edge from 2 bits to 3 bits of B0. Moreover, B3 alters the bit-widths of both edges of B0 by changing (4 bits, 2 bits) to (3 bits, 3 bits). Finally, we reevaluate B0, B1, B2, and B3 in the ImageNet1K dataset using the training settings shown in Table 1. As a result, we find the impacts of combinations of the two edges differ from the simple accumulation of their separate influence. Specifically, Fig. 3 shows that the accuracy of B3 at the convergence point exceeds the sum of B1 and B2, which indicates that there is a joint contribution between the 3th edge and the 4th edge for the whole model. That is, the impact of combinations of the two edges differs from the simple accumulation of their separate influence. This observation reveals the complex relationships between different bit-widths on different edges: some bit-widths can collaborate with each other, resulting in a significant joint contribution to the model's performance. Hence, we aim to resolve this issue in this paper by proposing a novel approach that leverages the Shapley value to estimate bit-width contribution from cooperative game theory, which can accurately reflect the contribution of the bit-width.

Conclusion. DMPQs indeed suffer from the issue that the learnable bit-width parameters updated by gradient descent fail to reflect the actual bit-width contribution.

4 Mixed Precision Search via Discovering Bit-width Contribution

4.1 "Where to Search" in SMPQ

"Where to search", i.e., where the optimal mixed precision quantization policy is derived, means defining the search space of MPQ. In this respect, we formulate the MPQ search space B as a layer-wise supernet G (consisting of bit-width sets B), similar to other DMPQ methods (as presented in Section 2.2). The only difference between them is that the updating of our supernet relies on the Shapley evaluation rather than the gradient in terms of α, and thus requires no relaxation functions (e.g., softmax) that are typically used by other DMPQs (as shown in Eq. 3). One merit of the supernet is to provide weight sharing, which means that the weights of all quantization models sampled from the supernet directly inherit from those of the supernet without training from scratch. In this way, the search efficiency is greatly improved.

Let Ω(Q) be the complexity of quantized network through the precision policy Q. Then, the bit-width contribution weights α and network weights W of supernet G can be optimized in a layer-wise manner with the following objective:

where Jv is validation loss, Dtrain is training datasets, Dval is validation datasets, and Ω0 is the resource constraint of the deployed device.

4.2 "How to Search" in SMPQ

"How to search" refers to how to optimize bit-width parameters α in supernet. Rather than the magnitude of bit-width parameters updated by gradient descent, we focus on their practical contribution to the target quantization performance. Moreover, we observe that the quantization operations in the supernet are often correlative with each other: the combinations of various bit-width quantizations may have different joint impacts on the quantization performance compared to their separate ones. To tackle such complex relationships, we employ the Shapley value to measure the average marginal contribution of the bit-width to the validation performance, where the MPQ search process is mapped into a cooperative game. The main procedures are as follows:

Assume that there are associated N players in the game, and each subset of players S ⊆ N (i.e., is coalition) is mapped to a real value V (S) (as expected payoff of the players for cooperation) via a value function V . For the supernet, each layer has identical cell structures, and each cell has |E| edges each with |O| bit-widths. Then, the bit-width set, N = O × E = o(i,j) o∈O,(i,j)∈E , can be regarded as players in the game, where all players cooperate towards the target performance V (N). For bit-width o(i,j) , its Shapley value ψ(i,j) o is calculated by:

In our scenario, we utilize the accuracy-complexity trade-off as the value function V to evaluate the quantization performance. However, the direct computation of Eq. 6 needs to enumerate all possible subsets, and thereby requires 2|O|×|E| time complexity, which is computationally prohibitive. For efficient evaluation, we adopt Monte-Carlo sampling method [Castro et al., 2009] to get the approximation of the Shapley value, of which a truncated sample technique is also used to clip current sampling if the bit-width results in a significant performance drop. Accordingly, the search objective Eq. 5 needs to be changed as:

where ψ is the Shapley value of edge (i, j). Since it is hard to directly solve the above objective, we need to update α(i,j) o using the Shapley value evaluated by the Monte-Carlo samplingbased approximate method:

where αt is the contribution weight at the kth iteration during the search, qk denotes the accumulated Shapley value in the kth iteration, || · ||2 is the L2 norm, and ξ is the term coefficient. In the bi-level optimization,Wt is trained by gradient descent ∇Lt(wt-1, αt-1) (as shown in Eq. 7) while αt is optimized by Shapley value until convergence on training datasets. To alleviate redundant fluctuations induced by the sampling, we incorporate the momentum into the optimization for stabilization in the Monte-Carlo sampling process:

where β + λ = 1, the coefficients β and λ are used to balance the accumulated Shapley value and the current sampling, respectively. Accv is the validation accuracy as value function, and wt-1 is the supernet weights at t - 1 iteration. At the end of the search, the final precision policy is derived by selecting the bit-width with the largest contribution on each edge.

4.3 Theoretical Analysis

In this section, we analyze the expected error of SMPQ based on the Shapley theory [Ancona et al., 2019; Castro et al., 2009] from a theoretical analysis perspective.

Theorem 1 (Upper-bounding of the risk on SMPQ). Given the previous setting for SMPQ, for a set N of n players (bit-widths), according to Eq.7, let ψ be the Shapley value of edge (i, j), SMPQ is able to converge to the optimal point when:

where ϵ is the expectation error loss on specific datasets, and abs(·) is absolute function. We say that ψ has converged to ∆ψ and stopped the iterative process.

Assume that after the kth iteration, the weight coefficients are α(i,j) o k, and after the (k +1)th iteration, the weight coefficients are α(i,j) o (k+1). To this end, we only need to show that ψ(Jv(W ∗ , α(k) )) will also converge. Therefore, we propose Lemma 1.

Lemma 1 (Bounding expectation (E) of its marginal contribution). According to Eq. 6, the Shapley value ψi(·) of the game ⟨N, V ⟩ for each subset of players (bit-widths) S ⊆ N is the expectation (E) of its Marginal Contribution to players (bit-widths) that can be formulated as:

The Shapley value can be explained as follows. Assume all the players are arranged in some order, with all orderings being equally likely. Afterward, φi(N, v) is the anticipated marginal contribution (overall orderings), of player i to the set of players who preceded him. According to the property of Shapley Value, for a mixed precision quantization game, one solution for N players in the quantization game must exist with the biggest contribution max(N) on validation datasets, which obtains best quantization weights for every player. In a mixed precision quantization game, each player (bit-width) only needs to maximize its own Shapley value (i.e., ψ(Jv (W ∗ , α(i,j) o ))) so that max(N) can be achieved as:

Proof of Lemma 1. According to Eq. 6, we can obtain:

(13)

Now, we define marginal contribution of o(i,j) to S as:

Based on Eq. 14, we can get:

According to the "Efficiency" of Shapley, we can obtain:

The convergence of α(i,j) o relies on Eq. 8, 9, namely, αk = αk-1+ξ · qk ∥qk∥2 , and qk = βqk-1+λ· ψk-1 ∥ψk-1∥2 are exponential moving average, where ψk-1 = ψ(Accval(wk-1, αk-1)). If W converge in Dtrain, αk and qk are stable in Dval. Therefore, we propose Lemma 2.

Lemma 2 (The convergence of quantized neural network). According to [Arora et al., 2018], we assume that quantized neural network is optimized by gradient descent with the learning rate η, depth L of neural network, and deficiency margin c > 0. For any ϱ > 0, the loss of quantized neural network at iteration T can be achieved as follows:

Proof of Lemma 2. Based on [Arora et al., 2018], we have:

Then, supernet weights W∗ = argminDtrain ℓ(W, α) converge via gradient descent with learning rate η, depth L, and deficiency margin c > 0, for any ϱ > 0. The loss L(W, α) update by:

starting from initial loss ℓ(0), the loss at iteration T satisfies:

Then, we can derive:

Finally, we can obtain:

As W→W∗ , Jv(W∗ , α(k) ) stabilizes, driving ψ to convergence.

Proof of Theorem 1. First, α is updated using Shapley values. By Lemma 2, for any ϱ > 0, we can derive ℓ(WT , α) ≤ ϱ, where T ≥ 1 η·e-2L · log ( ℓ(0) ϱ ) . Thus, WT →W ∗ . In bi-level optimization, W minimizes ℓ(W, α), while α minimizes Jv(W, α). By Lemma 1, the Shapley value is:

In addition, the validation loss is bounded as follows:

As αk → α∗ (optimal quantization policy), we can derive ψ(Jv(W ∗ , α(k) )) is stable. Then, we can derive:

Therefore, we have ∆ψ → 0. For any ϵ > 0, we can derive ∆ψ < ϵ. Theorem 1 is proven.

Remarks. We further interpret how the above proofs can verify the effectiveness of our proposed SMPQ. First, Lemma 1 proves that SMPQ can obtain bounding expectation of its marginal contribution for any subset of players/bit-widths, e.g., there exists an optimal contribution for any coalition, which proves there is a joint contribution between bit-widths on different edges (as presented in section 3.3). Such joint contribution of different bit-widths is a key to ensure high performance of our SMPQ. Second, Lemma 2 ensures the achieving of W∗ . Finally, theorem 1 shows that SMPQ can quickly converge towards a global minimum.

5 Experiments and Discussions

Experimental Settings. Following previous quantization methods [Cai and Vasconcelos, 2020; Chu et al., 2021], we train and evaluate our framework on single hardware platform, i.e., one NVIDIA Tesla A100 GPU with an Intel(R) Xeon(R) Gold 6133 CPU. In this paper, we use the training settings shown in Table 1. Moreover, the training settings of ResNet-50, and Inception-V3 are the same as ResNet-18. Hardware. All experiments are conducted on one NVIDIA Tesla A100 GPU with an Intel(R) Xeon(R) Gold 6133 CPU. Each experiment is executed on a single GPU at a time.

5.1 Comparison with State-of-the-Art Methods

The experimental results (all models with the same parameter setting are averaged over 10 random repetitions) on ImageNet1K are presented in Tables 2, and 3, respectively. From the reported results, we can have the following observations:

❶ As shown in Table 2, our SMPQ obtains 6.7, 2.6, 2.0, and 1.9 percent improvements over EdMIPS, DNAS, FracBitsSAT, and MetaMix with 2.9±0.07 GPU hours for ResNet-18

on ImageNet1K, respectively. As shown in Table 3, SMPQ achieves 72.0±0.02 accuracy, higher than peer competitors with 9.7±0.01 GPU hours for MobileNetV2. These results highlight the effectiveness of SMPQ. This is because that SMPQ methods can utilize the relationships between different bit-widths on different edges by our proposed SMPQ method, which can directly measure bit-width contribution to the performance of the quantized model on the validation sets.

❷When comparing the accuracy-complexity trade-off of SMPQ with baseline methods across various settings, it is evident that SMPQ provides a competitive accuracy-complexity trade-off under a variety of resource constraints. Importantly, this is achieved with significantly reduced BOPs (G). As shown in Table 2, compared with the chosen baselines for ResNet-18, the compression ratio of SMPQ is 1.7× more than DNAS, 1.8× more than EdMIPS, and 7.8× more than HAQ. This shows the effectiveness of our SMPQ.

❸ To the best of our knowledge, this work is the first to compare the performance among different search spaces. As shown in Tables 2, and 3, with bit-width increasing, the performance of models further increases, implying that search space is a key factor to MPQ. By comparing with "EdMIPS" under same search space, our SMPQ achieves competitive results with 68.7 top-1 accuracy and faster search speed, i.e., 2.4 GPU hours. v.s. 9.5 GPU hours. The main reason for better performance of SMPQ lies in that SMPQ can fully leverage relationship between different bit-widths through Shapley value. Shapley value is an excellent indicator of bitwidth contribution, enabling accurately selects optimal mixed precision quantization policy for neural networks.

5.2 Effectiveness for Larger Networks

To demonstrate the generalizability of SMPQ, we conduct experiments that scale to larger networks (i.e., ResNet-50, Inception-V3, etc.). The results are listed in Table 4. As shown in Table 4, compared with the chosen baselines for ResNet-50, the Top-1 accuracy of SMPQ is 4.1 more than EdMIPS, 0.7 more than HMQ, and 10.6 more than EdMIPS-C. By comparing with the chosen baselines for Inception-V3, the Top-1 accuracy of SMPQ is 3.6 more than HWGQ, 0.9 more than "Integer Only", and 2.2 more than EdMIPS. We can figure out that our approach is far better than peer competitors, which implies good generalizability on larger networks.

5.3 Correlation Analysis

Then, we perform a correlation analysis using Kendall's Tau (τ) coefficient. After the search phase, we select 10 discrete quantization policies from the supernet and calculate their corresponding bit-width strength by averaging the magnitude of the learnable bit-width parameters. We then plot the test accuracy vs. average bit-width magnitude obtained by DMPQ and our SMPQ. To measure the correlation between test accuracy and average bit-width magnitude, the results of Kendall's Tau (KTau) coefficient on ImageNet1K are depicted in Fig. 5. The larger the KTau value, the more predicted contribution matches the real contribution. From Fig. 5, we can see that our SMPQ achieves a higher correlation with the test accuracy (τ

= 0.494), while DMPQ (the learnable bit-width parameters are updated by gradient descent) shows a worse correlation with the final test accuracy. In summary, the correlation analysis demonstrates that our proposed SMPQ can predict the real contribution to bit-widths more accurately than DMPQ.

5.4 Ablation Study

Influence of sampling M and truncated sample. In fact, the two factors are crucial for model accuracy and search cost, as shown in Fig. 6 and 7. With the increase of M, the test error degrades significantly, while the search cost increases dramatically. As a result, when M = 10, we obtain accuracy-complexity trade-offs for SMPQ. For truncated sample, medium truncated sample (threshold=0.5) achieves the best accuracy-complexity trade-off.

Influence of β and ξ. Then, by varying β and ξ, we evaluate their influence with respect to the model accuracy. As shown in Table 5, we can find that our SMPQ is insensitive to β and ξ, and when β = 0.75 and ξ = 0.05, we get the optimal results.

6 Conclusion

In this paper, we attempt to understand and enhance MPQ methods from the bit-width selection perspective. We examine the magnitude-based quantization selection process of DMPQ and provide empirical evidence to show the α's pitfall issue, i.e., the value of α cannot well reflect the actual bit-width contribution, and then conduct deep analysis on the potential reason of such issue. To overcome this issue, we propose a Shapley-based MPQ method that directly estimates the bitwidth contribution via its contribution to the performance of the quantized model on the validation sets. Specifically, the Shapley value of bit-widths can be efficiently approximated by a Monte-Carlo sampling algorithm with early truncation. The proposed method is able to consistently obtain more optimal quantization policies than magnitude-based MPQs on a set of datasets and several search spaces.

Acknowledgments

This work is supported by National Natural Science Foundation of China under Grant 62472079, Fundamental Research Funds for the Central Universities (N2417008, N2417003), the Key Technologies R&D Program of Liaoning Province (2023JH1/10400082), and Guangdong Basic and Applied Basic Research Foundation (2024A1515012016).