Device-Cloud Collaborative Correction for On-Device Recommendation

1 Introduction

With the rapid development of recommendation models and device computing power, deploying recommendation models on devices has become feasible. Recommendation on devices, being closer to the data source compared to cloudbased recommendation, demonstrates better real-time performance and privacy. Additionally, device-based recommendation helps alleviate the computational pressure on the cloud, making it an important area of research [Gong et al., 2020; Lv et al., 2024]. Device-based recommendation is generally responsible for reranking the candidate list provided by the cloud model based on the user's real-time behavior sequence [Yao et al., 2021; Lv et al., 2025]. Therefore, device-based recommendation models are mainly temporal recommendation models. The mainstream devicebased recommendation models are divided into Transformerbased temporal recommendation models [Lai et al., 2023; Petrov and Macdonald, 2022; Kang and McAuley, 2018] and Recurrent Neural Network (RNN)-based temporal recommendation models [Hidasi et al., 2016; Lin et al., 2018; Niu and Zhang, 2017].

∗Corresponding authors.

Previously, Transformer-based models [Kang and McAuley, 2018; Sun et al., 2019; Liu et al., 2024] were widely applied in this field due to their superior performance compared to RNN-based recommendation models [Hidasi et al., 2016]. The strong performance of Transformer-based models primarily stems from their ability to better retain early user behavior information when handling long sequences. In contrast, RNN-based models tend to gradually forget early user behavior information as the sequence length increases. However, the memory advantages of Transformer-based models come at a significant time and space cost. Specifically, assuming the length of the user behavior sequence is N and the tensor dimension is d, ignoring the number of network layers, the time complexity and space complexity of Transformer-based models are O(N 2 · d) and O(N · d), while for RNN-based models, they are O(N · d) and O(d), respectively. Therefore, existing device recommendation models struggle to balance both real-time performance and high performance when handling long user behavior sequences.

To address the aforementioned challenges, we designed Device-Cloud Collaborative Correction Framework for OnDevice Recommendation (CoCorrRec), which consists of two modules: the Self-Correction Network (SCN) and the Global Correction Network (GCN). After pretraining based on historical user behavior data, SCN and GCN are deployed on the device and the cloud, respectively. The goal of SCN is to retain the linear computational cost and fixed memory overhead of RNN-based models while improving their performance. To address the long-sequence forgetting issue in RNN-based models, we draw inspiration from test-time training and large language models, using self-supervised learning to dynamically update model parameters and create a hidden state model, enhancing the model's ability to remember long-sequence information [Sun et al., 2024]. Considering real-time performance, SCN employs a mini-batch acceleration strategy to achieve parallel computation, enabling rapid and efficient updates to the device model. Furthermore, we analyze the potential shortcomings of SCN: (1) SCN updates model parameters during the testing phase, which is key to reviving RNN-based models for device recommendations. However, the data distribution on the device is only a biased subset of the global distribution, which negatively impacts the model's generalization ability while enhancing its personalization. (2) The mini-batch acceleration strategy used by SCN processes multiple user behaviors simultaneously with shared hidden states. While this improves efficiency, it also reduces the effective search space (as shown in Figure 1), limiting the model's ability to capture subtle patterns in user behavior. To address these limitations, we designed GCN, which receives the hidden state from the device's SCN, processed from the real-time user click sequences. GCN then performs device model parameter correction from a global perspective based on this hidden state. Since GCN uncovers similar user behaviors and trains using historical behavior data from numerous users, it exhibits significantly stronger generalization capabilities than the device model. As a result, CoCorrRec combines device-level personalization with cloud-level generalization, balancing real-time performance and high efficiency in device-based recommendations. Note that although GCN is designed to enhance SCN and can effectively improve performance, CoCorrRec does not rely on GCN. The device-based recommendation component of CoCorrRec, even when used on its own, can still achieve excellent performance. Extensive experiments on multiple datasets demonstrate that CoCorrRec outperforms existing device recommendation models, such as Transformer-based and RNNbased models, achieving performance improvements ranging from 0.06% to 4.56% across various metrics compared to the best-performing baseline models. Notably, CoCorrRec accomplishes these gains with fewer parameters and lower FLOPs, highlighting its ability to deliver superior recommendation accuracy while maintaining efficiency in terms of model size and computational cost.

The main contributions are summarized as follows:

• We introduce CoCorrRec, a novel recommendation system that integrates device-level personalization and cloud-level generalization to balance real-time performance and efficiency in device-based recommendations.
• We design SCN, which utilizes self-supervised learning and mini-batch acceleration to improve the ability of RNN-based models to handle long user behavior sequences while keeping computational cost and memory overhead low.
• We develop GCN, which corrects device model parameters using global user behavior patterns from the cloud, enhancing the model's generalization ability and overall performance.
• We conduct extensive experiments on multiple datasets, showing that CoCorrRec outperforms existing device recommendation models, such as Transformer-based and RNN-based models, with fewer parameters and lower computational overhead.

2 RelatedWork

Sequential Recommendation. Sequential recommendation models have traditionally relied on the Markov Chain assumption, where current interactions depend solely on the most recent interaction or a few preceding ones [Rendle et al., 2010]. While these models are straightforward and effective, they fail to capture long-term sequential dependencies. To address this limitation, various neural networks, such as Recurrent Neural Networks (RNNs) [Hidasi et al., 2016; Lin et al., 2018; Niu and Zhang, 2017], Convolutional Neural Networks (CNNs) [Tang and Wang, 2018], Attention networks [Lai et al., 2023; Petrov and Macdonald, 2022; Li et al., 2023; Su et al., 2023a; Su et al., 2023b] , Graph Neural Networks (GNNs) [Peng et al., 2023; Liu et al., 2023], and Multi-Layer Perceptrons (MLPs) [Long et al., 2024a; Zhou et al., 2022], have been employed to better utilize sequential information. More recently, advanced techniques such as contrastive learning and diffusion models [Wang et al., 2023b] have also been applied to sequential recommendation tasks.

Device-Cloud Recommendation. Device-cloud collaboration is an emerging approach in recommendation systems that combines the personalization capabilities of device-side models with the generalization power of cloud-based models [Yan et al., 2022; Long et al., 2024b; Qian et al., 2022; Fu et al., 2025; Fu et al., 2024]. This approach addresses challenges inherent in traditional recommendation methods, such as user-oriented bias/fairness issues [Li et al., 2021], as well as privacy concerns [Muhammad et al., 2020]. Early efforts, including DCCL [Yao et al., 2021] and MetaController [Yao et al., 2022], developed collaborative frameworks where device-side models focused on personalization[Zhang et al., 2020] while cloud-side models provided global knowledge [Lv et al., 2023; Lv et al., 2024]. When the TTT model is deployed on a device, updating parameters during recommendations for a specific user resembles single-user finetuning, which may compromise the model's general knowledge. To mitigate this, the cloud-based network can provides similar user information to the specific model and perform parameter fusion[Li et al., 2024], compensating for the loss of generalization.

Test-Time Training. Test-Time Training (TTT) refers to a strategy where each test instance defines a unique learning problem, targeting its own generalization [Sun et al., 2020]. As test instances lack labels, the learning problem must be framed using a self-supervised task. An early implementation of this concept, known as local learning, involves training on the neighbors of each test input before making predictions. This approach has been successfully applied across models [Hardt and Sun, 2023; Zhan et al., 2025]. Previous research demonstrates that TTT with reconstruction notably enhances performance, particularly for outliers [Gandelsman et al., 2022]. These improvements are even more pronounced when applied to video frames in a streaming context [Wang et al., 2023a], where TTT operates autoregressively.

3 Preliminary

3.1 Overview of Test-Time Training

With the observation that self-supervised learning can compress a massive training set into the weights of a model such as LLMs, TTT layer represents a novel category of sequence modeling layers, where the hidden state is a model, and the update rule is defined as a step in self-supervised learning [Sun et al., 2024]. Two primary instantiations of this class are TTT-Linear and TTT-MLP, in which the hidden state corresponds to a linear model and a two-layer MLP, respectively. In this work, we select TTT-Linear as the core component of the TTT Block.

For self-supervised learning in TTT layer, three lowrank projection learnable matrix θK, θV , θQ are introduced. Specifically, when processing a sequence x1,...,xt, θK applies corruption to xt, θV extracts critical information for reconstruction and θQ ensures that xt remains aligned with the input dimensionality of f. The update rule for W is a gradient descent step:

where the self-supervised loss function l is:

The output token, zt, is predicted for xt using f with the updated weights Wt:

3.2 Mini-batch Strategy

In naive TTT, each update of Wt depends on the previous step Wt-1, which inhibits efficient parallel computation. The general gradient descent method can be expressed as:

To address this limitation, mini-batch gradient descent is employed. Let the TTT mini-batch size be b. W can be computed in parallel as follows:

where t′ = t- mod (t, b) is the last timestep of the previous mini-batch (or 0 for the first mini-batch). This approach enables parallel computation of b gradient updates at a time.

3.3 Dual Form

To fully harness the matrix computing capabilities of the GPU, the calculation process for TTT has been optimized. Consider the simplest case of l, where θK = θV = θQ = I, and focus on the first TTT mini-batch of size b. Additionally, assume that f is a linear model. In this context, just compute the output Z = [z1, ..., zb] and initializing Wb for the next mini-batch:

where

4 Methodology

To enable recommendation models on devices to balance real-time performance and high performance, we designed a self-correction network based on TTT to improve RNNbased recommendationmodels. It extract features fromusers' real-time click sequences, enabling high-quality recommendations with minimal resource usage. To further overcome two specific performance limitations of corrected adaptor, we developed a cloud-based computationally intensive Correction Network (GCN). This network expands TTT's parameter search space and extracts knowledge from similar users, thereby integrating collaborative filtering information into the recommendation process.

4.1 On-Device Self-Correction Network

To ensure optimal efficiency and simplicity, the on-device SCN model M(·; θM) is designed with a streamlined architecture, comprising only an embedding layer, TTT Block, and a prediction layer (as illustrated in a figure 3).

The embedding layer maps item IDs to a high-dimensional space using a learnable embedding matrix, E ∈ R|V |×D , where D denotes the embedding dimension. When applied to the input item sequence Su, we obtain the initial item embeddings Eu. To enhance robustness and mitigate overfitting, RMS normalization is applied after the embeddings are retrieved:

While the TTT block demonstrates potential as a sequenceto-sequence model, we focus on leveraging its strengths for sequential recommendation tasks. Based on practical application, the TTT block has been streamlined to further reduce computational cost and enhance the efficiency of the ondevice model. Starting with the hidden states H, a low-order projection is performed using θQ, θK, and θV :

Rotary positional encodings (RoPE) [Su et al., 2024] are then applied to XQ and XK as follows, in which rotary embedding is calculated by XV :

TTT Block employs self-supervised learning to process the input X. When using a mini-batch and dual form, the calculation of Wb and the output Z detailed in Algorithm 1 assuming that η = 1 and the first mini-batch. Finally, it undergoes Layer Normalization (LN) and a final linear projection:

In the prediction layer, the final item embedding of O is used to compute the output prediction scores:

Algorithm 1 TTT Block of SCN

Input: H

Output: Wb, Z

Process: Wb, Z

XQ, XK, XV ← θQH, θKH, θV H

XQ, XK ← RoPE(XQ, XV ), RoPE(XK , XV )

∇XK l ← W0XK - XV

∇Z l ← σ′ (∇XK l)

▷ σ′ is the derivative of sigmod function

∇W0 l ← ∇Z l(XK )T

Wb ← W0 - ∇W0 l

b ← b0 - mask(∇Z l)

attn ← mask(XTK XQ)

Z ←W0XQ -∇Zl · attn+ b

return Wb, Z

4.2 On-Cloud Global Correction Network

The On-Cloud Global Correction Network (GCN) generates dynamic parameter corrections for the corrected adaptor based on real-time samples from specific devices. Its purpose is to expand the search space and provide collaborative filtering information to enhance the recommendation process. Specifically, the Correction Network is implemented as a three-layer MLP, denoted as C(·; θc) with parameters θc. It takes H from the device as input and predicts the correction values K. Due to its high computational cost (measured in FLOPs), the GCN is more suitable for cloud deployment rather than on-device implementation. Since it operates independently of the corrected adaptor, both can run in parallel. Based on Algorithm 1, we instantiate GCN as Cattn and Cb to compute corrections for the key values attn and b:

After the cloud-generated results are produced, they are sent directly to the on-device model for correction. To ensure robust results, Kattn,Kb are then normalized to match the mean and variance of the corresponding values attn and b:

To further refine the corrections, learnable fusion parameters P and Q ∈ R1×1 are introduced for adaptive fusion:

4.3 Training and Inference

The training of CoCorrRec is performed entirely in the cloud, utilizing abundant computational resources. All layers are optimized simultaneously using global historical data, SH = {x(i) H ; y(i) H }NH i=1 , where NH represents the total number of training data samples. The loss function, Ltotal, is defined as follows:

where Dce(; ) denotes the cross-entropy between two probability distributions, and Hx represents the hidden state of input x as described in Section 4.1.

Since the input Hx to C is computed before the TTT Block, the entire framework can correct the TTT Block during training and inference in parallel which is detailed in Section 5.3.

This parallelization significantly improves the training and inference efficiency of CoCorrRec, making it more suitable for deployment in real-world environments.

After the CoCorrRec training is completed, the SCN component is deployed to specific devices for real-time reranking. Meanwhile, the computationally intensive GCN is retained in the cloud to enhance the recommendation performance of on-device SCN instance. For real-time samples, SR = {x(i) R }NR i=1 , where NR represents the total number of real-time data samples, the recommendation process is formulated as:

5 Experiments

In this section, we conduct extensive experiments on three realworld datasets to answer the following research questions:

• RQ1: What is the resource consumption required for various basic recommendation models (FLOPs and maximum memory usage)?
• RQ2: What is the Time delay caused by device-cloud communication of the CoCorrRec?
• RQ3: How does our CoCorrRec perform compared to the baselines under various experimental settings and how do the designs of it affect the performance?

5.1 Experimental Settings

Datasets

We conduct experiments on three publicly available datasets in different scenarios. Amazon-beauty and Amazonelectronic are from an e-commerce platform Amazon 1 , which covers various product categories such as books, electronics, home goods, and more. We also evaluated on Yelp 2 which is a representative business dataset containing user reviews for different restaurants. For all datasets, we first sort all interactions chronologically according to the timestamps. And then discard users and items with interactions ≤ 10. All useritem pairs in the dataset are treated as positive samples. In the training and test sets, the user-item pairs that do not exist in the dataset are sampled at 1:4 and 1:100, respectively, as negative samples. The detailed statistics for each dataset after preprocessing are summarized in Table ??.

1 https://jmcauley.ucsd.edu/data/amazon/

Baselines

We compare CoCorrRec with several competitive baselines, including both RNN-based and Transformer-based methods. GRU4Rec [Hidasi et al., 2016] employs gated recurrent units (GRUs) to model behavior sequences, achieving excellent performance. DIN [Zhou et al., 2018] introduces attention mechanisms to capture users' interests related to the current product within their historical behavior. SASRec [Kang and McAuley, 2018] leverages self-attention to efficiently capture long-term dependencies in user behavior sequences. BERT4Rec [Sun et al., 2019] utilizes a bidirectional Transformer to extract contextual information from user behavior sequences, delivering more accurate recommendations. Lastly, Mamba4Rec [Liu et al., 2024] highlights recent advances in state-space models (SSMs) for efficient sequential recommendation tasks.

Evaluation

For performance comparison, we adopt three widely used metrics UAUC, HitRate@K and NDCG@K over the top-K items, where K is set as 5, 10 or 20.

Hyper-parameters Settings

We tune the learning rates of all models in {1e-3 , 2e-3 , 5e-3 , 1e-2 } and select the best performance. As to model-specific hyper-parameters, we select default parameters from open-source code. Additionally, unless otherwise specified, the length of the user sequence is

2 https://www.yelp.com/dataset/

10. All models are trained using a single RTX-4090 GPU. Other hyperparameters of training are shown in table ?? and the number of head is for transformer-based models.

5.2 Analysis of Resource Consumption(RQ1)

On resource-constrained devices, the increasing FLOPs (Floating Point Operations Per Second) and memory consumption of models as the user interaction sequence grows during the recommendation process can gradually degrade the user experience. Table ?? details the parameters and resource consumption of each model. Notably, most of these models have a comparable number of parameters. The FLOPs metric quantifies the computational cost of a single inference, while memory consumption indicates the peak memory usage during a complete inference epoch on Amazon-Beauty. Figure 4 illustrates the FLOPs and maximum memory usage of the models when the user sequence lengths are 10, 15, 20, and 25, respectively.

It is expected that RNN-based models grow linearly with sequence length, but it is surprising that Transformer-based models like SASRec exhibit the same behavior. In fact, for Transformers, the total computation of self-attention (SA) is 4nh2d2 +2n2hd, where d is the head size and h is the number of heads, while the Feed-Forward Network (FFN) computation is 8nh2d2 . Therefore, for a Transformer-base (d = 64, h = 12), when the sequence length n ≤ 1536, the linear complexity of the FFN dominates, which is the usual circumstance for recommendation.

It can be observed that our designed CoCorrRec (w/o. GCN) has certain advantages in terms of computational cost and memory usage, with minimal additional overhead as the user sequence grows. Under the same resource constraints, it can handle longer user interaction sequences. Table ?? also shows the additional cost introduced by the GCN. Although the number of additional parameters is small, the FLOPs and memory consumption of the computationally intensive GCN are significant. Therefore, it is more suitable for deployment on cloud with abundant computational resources.

5.3 Analysis of Time Delay(RQ2)

Table ?? summarizes the size and time delay involved in uploading hidden state H and downloading correction parameters to the device. It also details the time required for the cloud-based GCN to generate corrections and the device's tolerance time (i.e., the duration from hidden state H generation to the application of corrections on the device) as shown in figure 5. The results demonstrate that the time needed for on-device model to receive corrections consistently remains within the tolerance limit, confirming its suitability for realtime applications. Furthermore, uploading embeddings for real-time samples enhances user privacy. These findings validate that the cloud-based GCN does not introduce additional delays to the on-device model, as both processes can run concurrently. This result highlights the effectiveness of CoCorrRec in achieving real-time application requirements.

5.4 Analysis of CoCorrRec (RQ3)

Overall Performance

Table 1 compares the performance of CoCorrRec with baseline models. CoCorrRec (w/o. GCN) represents the pure on-device model, while CoCorrRec integrates the on-device model with the cloud-based GCN. As this study focuses on sequential recommendation tasks, all models were exclusively trained and tested on sequential recommendation datasets. From Table ??, the following conclusions can be drawn:

• The SCN demonstrates superior robustness in sequential recommendation tasks. While the performance of various baselines varies across datasets due to their respec-
• tive strengths, CoCorrRec (w/o. GCN) maintains relatively stable performance across all datasets by updating model parameters during inference.
• In all cases, our revised CoCorrRec consistently outperforms CoCorrRec (w/o. GCN). This underscores the effectiveness of the cloud-based GCN in enhancing the representation capabilities of the on-device model. Furthermore, the cloud-based GCN improves model performance in parallel without increasing the on-device model's time overhead.

Effect of Mini-Batch Size

The mini-batch size determines the number of tokens processed simultaneously using the mini-batch parallelization strategy. Although processing all tokens at once minimizes resource consumption, it may not achieve optimal performance. To address this, we assessed the performance of minibatch sizes of 1, 2, 5, 10, and 20 on the Beauty dataset, with a sequence length of 20. Figure 6 illustrates the variations in model performance and inference time as the mini-batch size b increases. While performance shows some fluctuation, the results indicate that the performance at b = 20 is nearly identical to that at b = 1. However, the inference time at b = 20 demonstrates a significant improvement. Therefore, parallel computation of all tokens is generally a more efficient choice.

6 Conclusion

In this paper, we propose the CoCorrRec, which deploys an RNN-based Self-Correcting Network (SCN) on the device and a Global Correcting Network (GCN) on the cloud. Through the collaboration of both, the model addresses the current limitations of recommendation models on devices in terms of balancing real-time performance and high efficiency. CoCorrRec repositions RNN-based models as highly competitive for sequential recommendation, especially on devices, and demonstrates the potential of device-cloud collaborative parameter correction for device-based recommendation.

Acknowledgements

This work was in part supported by 2030 National Science and Technology Major Project (2022ZD0119100), National Natural Science Foundation of China (No. 62402429, U24A20326, 62441605), the Key Research and Development Program of Zhejiang Province (No. 2024C03270), ZJU Kunpeng&Ascend Center of Excellence, Ningbo Yongjiang Talent Introduction Programme (2023A-397-G), Young Elite Scientists Sponsorship Program by CAST (2024QNRC001).