Dual Personalization on Federated Recommendation

1 Introduction

Federated recommendation is a new service architecture for Internet applications, and it aims to provide personalized recommendation service while preserving user privacy in the federated settings. Existing federated recommendation systems [Muhammad et al., 2020; Yi et al., 2021; Perifanis and Efraimidis, 2022; Wu et al., 2022b] are usually to be an adaptation of distributed recommendation algorithms by embodying the data locality in federated setting and adding privacypreserving algorithms with guaranteed protection. However, these implementations of federated recommendations still inherit the traditional service architecture, which is to deploy large-scale models at servers. Thus it is impractical and inconsistent with the newly raised on-device service architecture, which is to deploy a lightweight model on the device to provide service independently without frequently communicating with the server. Given the challenge of implementing data locality on devices in federated settings, the personalization mechanism needs to be reconsidered to better capture fine-grained personalization for end-users.

∗Corresponding author.

Personalization is the core component of implementing federated recommendation systems. Inherited from conventional recommendation algorithms, existing federated recommendation frameworks are usually composed of three modules: user embedding to preserve the user's profile, item embedding to maintain proximity relationships among items, and the score function to predict the user's preference or rating for a given item. They usually preserve user-specific personalization in the user embedding module while sharing consensus on item embeddings and score functions.

This paper proposes a new dual personalization mechanism designed to capture fine-grained two-fold personal preferences for users in the federated recommendation system. Inspired by human beings' decision logic, we believe all modules in the recommendation framework should be used to preserve part of personalization rather than use user embedding only. For example, the score function is to mimic the user's personal decision logic that is natural to be diverse across clients. Furthermore, given an item set, different people may have different views on measuring their proximity relationships. Therefore, personalized item embedding could be essential to capture people's personal preferences further.

To implement the aforementioned ideas in federated settings, we propose a new federated recommendation framework to implement fine-grained personalization on multiple modules which are illustrated in Figure 1 (c). First, we use a personalized score function to capture user's preferences, and it could be implemented with a multi-layer neural networks. Second, we remove the user embedding from the federated recommendation framework because the current neural-based personalized score function has enough representation capability to preserve the information of user embeddings. Third, we implement light finetuning to learn personalized item embeddings in federated settings. This proposed decentralized intelligence architecture is a natural simulation of human beings' decision-making that each person has a relatively independent mind to make decisions.

The learning procedure is also carefully tailored in a federated setting. A personalized score function will be learned using the user's own data on the device, and then it won't be sent to the server for global aggregation that usually generates a general view for all devices. Moreover, the personalized item embedding will be implemented through light finetuning in a federated learning framework, thus it can leverage both the general view from server and the personalized view from user's own data.

In summary, we propose a novel federated recommendation framework that integrates both the personalized score function and personalized item embedding via light finetuning from the shared item embedding. Our key contributions are summarized as follows.

• We propose a novel federated recommendation framework which is more naturally consistent with layer-wise neural architecture and can better fit federated learning.
• We design a novel dual personalization mechanism to capture user preferences using a personalized score function and fine-grained personalization on item embeddings. It can be integrated with other baselines to improve their performances.
• We formulate the proposed federated recommendation learning problem into a unified federated optimization framework with a bi-level objective.
• Our method can significantly outperform existing federated recommendation baselines.

2 RelatedWork

2.1 Personalized Federated Learning

Federated learning is a new machine learning paradigm that a server orchestrates a large number of clients to train a model without accessing their data [Kairouz et al., 2021; Li et al.,

2020; Long et al., 2021; Long et al., 2020; Tan et al., 2022c; Tan et al., 2022b; Chen et al., 2023]. The vanilla federated learning method, FedAvg [McMahan et al., 2017], is to learn a robust model at the server while embodying data locality for each device with non-IID data. Personalized federated learning is to learn a personalized model for each device to tackle the non-IID challenge. Per-FedAvg [Fallah et al., 2020] exploits Model-Agnostic Meta-Learning (MAML) [Finn et al., 2017] to find a shared initial model for all clients and each client can learn a personalized model with its own data. Ditto [Li et al., 2021] proposes a bi-level optimization framework for PFL which introduces a regularization term by constraining the distance between the local and the global model. [Shamsian et al., 2021] propose to replace the global model with a global hyper-network orchestrate clients' local training. FedRecon [Singhal et al., 2021] is a meta-learningbased method that preserves a local model for each client and trains a global model collaboratively with FedAvg. There are also some attempts about graph-based [Tan et al., 2022a; Chen et al., 2022] and cluster-based [Ma et al., 2022; Long et al., 2023] methods. In this paper, we focus on developing personalization in the federated recommendation scenario where device data distributions are heavily non-IID, and it has not been well explored.

2.2 Federated Recommendation Systems

Federated recommendation has attracted much attention recently due to the rising privacy concern. Some recent works focus on only using the interaction matrix which is the most fundamental recommendation scenario. FCF [Ammad-UdDin et al., 2019] is the first FL-based collaborative filtering method, which employs the stochastic gradient approach to update the local model, and FedAvg is adopted to update the global model. Improving user privacy protection, FedMF [Chai et al., 2020] adapts distributed matrix factorization to the FL setting and introduces the homomorphic encryption technique on gradients before uploading to the server. MetaMF [Lin et al., 2020b] is a distributed matrix factorization framework where a meta-network is adopted to generate the rating prediction model and private item embedding. [Wu et al., 2022b] presents FedPerGNN where each user maintains a GNN model to incorporate high-order useritem information. However, the server in both MetaMF and FedPerGNN preserves all the recommendation model parameters which can be used to infer the user's interaction information, resulting in the risk of user privacy leakage. FedNCF [Perifanis and Efraimidis, 2022] adapts Neural Collaborative Filtering (NCF) [He et al., 2017] to the federated setting which introduces neural network to learn user-item interaction function to enhance model learning ability.

Besides, there are federated recommendation methods using rich information that consider multiple data sources in modeling. FedFast [Muhammad et al., 2020] extends FedAvg [McMahan et al., 2017] with an active aggregation method to facilitate the convergence. Efficient-FedRec [Yi et al., 2021] decomposes the model into a news model on the server and a user model on the client, and reduces the computation and communication cost for users. Both works rely on more data sources, such as user features or news attributes rather than an interaction matrix. [Lin et al., 2020a; Du et al., 2021; Minto et al., 2021; Lin et al., 2021] are endeavors that focus on enhancing privacy of FedRec. There are also attempts for other applications in FedRec, such as federated attack [Wu et al., 2022c; Zhang et al., 2022], social recommendation [Liu et al., 2022b], Click-Through Rate (CTR) prediction [Wu et al., 2022a] and fair recommendation [Liu et al., 2022a]. Existing federated recommendation methods usually combine the distributed recommendation algorithms and privacy-preserving techniques directly. They inherits the heavyweight models at the server and pay little attention on lightweight models deployed at client. In this paper, we present a novel federated recommendation framework aiming to learn many user-specific lightweight models rather than a heavyweight model on the server.

3 ProblemFormulation

Federated learning is to learn a global model parameterized by θ to serve all clients whose data are private. The optimal solution should minimize the below accumulated loss of all clients,

where Li(θ) is the supervised loss on the i-th client with dataset Di, and all clients share the global parameter θ. The αi is a weight for the loss of the i-th client. For example, the conventional FL algorithm, FedAvg [McMahan et al., 2017], defines αi as the fraction of the size of the client's training data, i.e., αi := |Di|/ ∑Nj=1 |Dj|. Once the global model is trained, it can be used for prediction tasks on all clients.

Personalized federated learning simultaneously leverages common knowledge among clients and learns a personalized model for each client, with learning objective as,

where each client has a unique personalized parameter θi, and θ is the global parameter as mentioned in Eq. (1). For example, [Fallah et al., 2020] leverage θ as initialization of θi, i.e., θi := θ -∇li(θ), where li(θ) is the loss of a vanilla model on the i-th client. The Li(θ, θi) is then formulated as

Recommendation with neural networks. This work focuses on the fundamental scenario where recommendation only relies on the user-item interaction matrix without extra user/item attributes. The recommendation framework can be divided into three components: a user embedding module E, an item embedding module E and a score function S. We denote these modules' parameters with θ := (θu , θm, θs ) and formulate the learning objective as,

where eu and em are one-hot encodings representing users and items. r is a user's rating to the given item and ˆ r is a prediction from the score function S(E(eu ),E(em)). L is the loss evaluation metric, which could be a point-wise loss as used in [Wang et al., 2016; He et al., 2017], or a pair-wise loss as in [Rendle et al., 2012; Wang et al., 2019]. It is worth noting that conventional Matrix Factorization (MF) methods could be viewed as a special case of the framework in Eq. (4), i.e., the conventional MF is a model where the score function S is simplified as the inner product operator without learnable parameters, and the embedding of user/item is obtained by the decomposition of the user-item interaction matrix.

4 Methodology

In this section, we propose a novel Personalized Federated Recommendation (PFedRec) framework, which aims to simultaneously learn many user-specific recommendation models deployed on end devices.

4.1 Objective Function

Federated learning objective. We regard each user as a client under FL settings. The on-device recommendation task is then depicted as a PFL problem. Particularly, the item embedding module Ei is assigned to be a global component which learns common item information and the score function Si is maintained locally to learn personalized decision logic. To further capture the difference between users and achieve a preference-preserving item embedding, we devise a bi-level optimization objective,

where θi := (θm i , θs i ) is the personalized parameter for Ei and Si, and Li will be evaluated on the i-th client local data Di. Under this framework, PFedRec first tunes E into a personalized item embedding module Ei, and then learns a lightweight local score function Si to make personalized predictions. Different from the conventional recommendation algorithms, the user embedding module E is depreciated since the personalization procedure on a client will automatically capture the client's preference. There is no use to learn extra embeddings to describe clients.

Loss for recommendation. Equipped with the item embedding module and score function, we formulate the prediction of j-th item by i-th user's recommendation model as,

Particularly, we discuss the typical recommendation task with implicit feedback, that is, rij = 1 if i-th user interacted with j-th item; otherwise rij = 0. With the binary-value nature of implicit feedback, we define the loss function of i-th user as the binary cross-entropy loss,

where Di is the negative instances set of user i. Notably, other loss functions can also be used, and here we choose the binary cross-entropy loss to simplify the description. Particularly, to construct Di efficiently, we first count all the uninteracted items set as,I -

where I denotes the full item list and Ii is the interacted item set of i-th user. Then, we uniformly sample negative instances from I - i by setting the sampling ratio according to the number of observed interactions and obtain Di .

4.2 Dual Personalization

We present a dual personalization mechanism to enable the proposed framework can preserve fine-grained personalization for both user and item.

Using partial-based federated model aggregation to learn personalized user score function on each device. Our proposed model is composed of a neural-based score function parameterized by θs and an item embedding module parameterized by θm. The coordinator/server of federated system will iteratively aggregate model parameters or gradients collected from each participant/device. Due to the concern of personalization and privacy, we implement a partial model aggregation strategy by keeping the score function as a private module on devices while sharing the item embedding to the server. Therefore, the server only aggregates the gradients or parameters θm from the item embedding module. The user's personalized score function module θs won't be sent to the server and thus won't be aggregated. Generally, the simple and swift multi-layer neural network is capable of tackling most scenarios, which is convenient for client deployment.

Finetuning the item embedding module to generate personalized representations for items on each device. According to Eq. (5), the learning objective of θm could be viewed as searching for a "good initialization" that could be fast adaptive to the learning task on different devices. It shares similar ideas with meta-learning-based methods [Fallah et al., 2020] which have a local loss in Eq. (3). However, our proposed method takes a different optimization strategy we call post-tuning. Specifically, rather than directly tuning a global model on clients' local data, it first learns the local score function with the global item embedding, and then replaces the global item embedding with personalized item embedding obtained by finetuning θm.

4.3 Algorithm

Optimization. To solve the optimization problem as described in Sec. 4.1 - objective function, we conduct an alternative optimization algorithm to train the model. As illustrated in Algorithm 1, when the client receives the item embedding from server, it first replaces its item embedding with the global one, and then updates the score function while keeping the item embedding module fixed. Then the client updates the item embedding based on the updated personalized score function. Finally, the updated item embedding would be uploaded to the server for global aggregation.

Workflow. The overall algorithm workflow could be summarized into several steps as follows. The server is responsible for updating shared parameters and organizing all clients

Algorithm 1 Optimization Workflow of PFedRec

ServerExecute:

• 1: Initialize item embedding θm and score function θs
• 2: for t = 1, 2, ... do
• 3: St ← (select a client set of size n randomly from all N clients)
• 4: for client i ∈ St in parallel do
• 5: θm i ←ClientUpdate(i, θm)
• 6: θm ← 1n ∑ni=1 θm i

ClientUpdate(i, θm):

• 1: Initialize θm i with θm
• 2: Initialize θs i with the latest update
• 3: Count all uninteracted items set I - i with Eq. (8)
• 4: Sample negative instances set Di from I - i
• 5: B ← (split Di ∪Di into batches of size B)
• 6: for e from 1 to E do
• 7: for batch b ∈ B do
• 8: Compute Li(θi; r, ˆ r) with Eq. (7)
• 9: θs i ← θs i - η∇θs i Li
• 10: Compute Li(θi; r, ˆ r) with Eq. (7)
• 11: θm i ← θm i - η′∇θm i Li
• 12: Return θm i to server

to complete collaborative training. At the beginning of federated optimization, the server initializes the model parameters, which would be used as initial parameters for all client models. In each round, the server selects a random set of clients and distributes the global item embedding θm to them. When local training is over, the server collects the updated item embedding from each client to perform global aggregation. We build on the simplified version of FedAvg, a direct average of locally uploaded item embeddings. The overall procedure is summarized in Algorithm 1.

Efficient on-device update. Focusing on the fundamental recommendation scenario, i.e., with only user-item interaction matrix, the item embedding module Ei dominates the parameter volume in the recommendation model due to large item set size, which brings challenges to end devices with limited computing resources. Generally, the items set that each user interacts with is much smaller than the complete item collection. Based on this observation, we propose that each device only needs to maintain the interacted positive items and sampled negative samples instead of the complete item embedding module, resulting in an efficient on-device update. For clarity, we continue to use θm in the Algorithm formulation. In practice, each device only needs to maintain a subset of the complete item embeddings.

5 Discussions

5.1 Privacy on Federated Recommendation

Privacy-preserving is an essential motivation to advance existing cloud-centric recommendation to client-centric recommendation service architecture. In general, the federated learning's decentralized framework can embody data locality and information minimization rules (GDPR) that could greatly mitigate the risk of privacy leakage [Kairouz et al., 2019]. To provide service with privacy guarantee, the FL framework should be integrated with other privacypreserving methods, such as Differential Privacy and secure communication. Our proposed framework derives the same decentralized framework from vanilla FL to preserve data locality. For example, to tackle the privacy leakage risk caused by sending item embedding to the server, we could simply apply differential privacy to inject noise into the embeddings so that the server cannot simply infer the updated items by watching the changes of embeddings. More analysis and experimental verification can be found in Sec. 6.6.

5.2 AGeneral Framework for Federated Recommendation

The proposed framework in Figure 1 (c) could be a general form of federated recommendation because our framework could be easily transformed into an equivalent form of other frameworks. For example, if we assign the score function as a one-layer linear neural network, PFedRec is equal to FedMF in Figure 1 (a). Moreover, if we change the personalized score function from full personalization to partial layer personalization, our method could be equivalent to FedNCF in Figure 1 (b) which has a shared score function across clients. Furthermore, our proposed framework's architecture could be naturally aligned with the classic neural network architecture, thus it has a bigger potential to achieve better learning efficiency and is more flexible to extend.

6 Experiments

6.1 Experimental Setup

We evaluate the proposed PFedRec on four real-world datasets: MovieLens-100K, MovieLens-1M [Harper and Konstan, 2015], Lastfm-2K [Cantador et al., 2011] and Amazon-Video [Ni et al., 2019]. They are all widely used datasets in assessing recommendation models. Specifically, two MovieLens datasets were collected through the MovieLens website, containing movie ratings and each user has at least 20 ratings. Lastfm-2K is a music recommendation dataset, and each user maintains a list of her favorite artists and corresponding tags. Amazon-Video was collected from the Amazon site, containing product reviews and metadata information. We excluded users with less than 5 interactions in Lastfm-2K and Amazon-Video. The characteristics of datasets are shown in Table 1. For dataset split, We follow the prevalent leave-one-out evaluation [He et al., 2017]. We evaluate the model performance (%) with Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG) metrics.

6.2 Baselines and Implementation Details

Baselines. Our method is compared with baselines in both centralized and federated settings. Focusing on the performance improvement of the infrastructure of recommendation models that all others derive from, we select the general and fundamental baselines that conduct recommendations only based on the interaction matrix.

• Matrix Factorization (MF) [Koren et al., 2009]: This method is a typical recommendation algorithm. Particularly, it decomposes the rating matrix into two embeddings located in the same latent space to characterize users and items, respectively.
• Neural Collaborative Filtering (NCF) [He et al., 2017]: This method models user-item interaction function with an MLP, and is one of the most representative neural recommendation models. Specifically, we apply the interaction function with a three-layer MLP for comparison, which is adopted in the original paper.
• FedMF [Chai et al., 2020]: It is a federated version of MF which is a typical FedRec method. It updates user embedding locally and aggregates item gradients globally.
• FedNCF [Perifanis and Efraimidis, 2022]: It is a federated version of NCF. Specifically, each user updates user embedding locally and uploads item embedding and score function to the server for global update.
• Federated Reconstruction (FedRecon) [Singhal et al., 2021]: It is a state-of-the-art PFL framework, and we test it under the matrix factorization scenario. Between every two rounds, this method does not inherit user embedding from the previous round but trains it from scratch.
• Meta Matrix Factorization (MetaMF) [Lin et al., 2020b]: It is a distributed matrix factorization framework where a meta-network is adopted to generate the rating prediction module and private item embedding.
• Federated Graph Neural Network (FedPerGNN) [Wu et al., 2022b]: It deploys a GNN in each client and the user can incorporate high-order user-item information by a graph expansion protocol.

Implementation details. We sample 4 negative instances for each positive instance following [He et al., 2017]. For a fair comparison, we set the user (item) embedding size as 32 and the batch size is fixed as 256 for all methods, and set other model details of baselines according to their original papers. The total number of communication rounds is set to 100, and this value enables all methods to be trained to converge through experiments. One exception is FedPerGNN, we follow the experimental setting in the original paper with the official code, whose rounds number is set to 3. For our method, we assign the score function with a one-layer MLP for simplification, which can be regarded as an enhanced FedMF with our dual personalization mechanism. We implement the methods based on the Pytorch framework 1 and run all the experiments for 5 repetitions and report the average results.

1 Code: https://github.com/Zhangcx19/IJCAI-23-PFedRec

6.3 Comparison Analysis

We conduct experiments on four datasets for performance comparison and the resuls are shown in Table 2.

Results & discussion. From the results, we have several observations: (1) PFedRec obtains better performance than centralized methods in some cases. In the centralized scenario, only user embedding is regarded as the personalized component to learn user characteristics, and other components are totally shared among users. In comparison, our dual personalization mechanism considers two forms of personalization, which can further exploit user preferences. (2) PFedRec realizes outstanding advances on the two MovieLens datasets. In these two datasets, each user has more interaction samples which can be used to train device recommendation models, hence promoting user personalization learning and fitting our method better. (3) PFedRec consistently achieves the best performance against all federated methods. In FedRec, the common item embeddings help transfer the shared information among users, which facilitates collaborative training of individual user models. However, different users present rather distinct preferences for items and existing federated methods deploy the global item embeddings indiscriminately for all clients ignoring user-specific preferences. In comparison, our dual personalization mechanism learns fine-grained personalization which fits user preferences.

6.4 Enhance Federated RecommendationMethods with Our Dual Personalization Mechanism

This paper proposes a lightweight dual personalization mechanism to enhance personalization handling, which can be easily integrated into federated learning methods. Particularly, we take FedMF, FedNCF and FedRecon as examples to verify the efficacy of the dual personalization mechanism.

Results & discussion. According to Table 3, all three federated recommendation methods are significantly improved by integrating our dual personalization mechanism. Among them, FedNCF attains the most remarkable boost. The highest HR@10 and NDCG@10 increase exist on MovieLens100K, i.e., 13.53% and 18.29%. Compared with Lastfm-2K and Amazon-Video, the improvement of the dual personalization mechanism is more evident on the two MovieLens datasets, almost around 10%, where each user has more samples locally and facilitates user preference capture. In summary, our proposed dual personalization mechanism can help the local model to learn user-specific item embedding, which benefits the recommendation system prominently.

6.5 AClose Look of Personalization in PFedRec

To further verify and analyze the role of personalized item embedding in our method, we conduct empirical experiments to answer the following questions:

• Q1: Why personalized item embeddings benefit recommendation more than the global one?
• Q2: How specific are the personalized item embeddings among users?

To answer Q1, we first discuss its straightforward insight, then we present visualization to demonstrate our claim. The recommendation system aims to provide user-specific recommendations by exploiting historical interactions. In the FedRec setting, item embedding is consistently considered to maintain the common characteristics among users, and its role in depicting user-specific preferences has been neglected. On the other hand, describing users with common item embedding introduces noisy information, which may incur unsuitable recommendations. Through personalizing item embedding, we enhance personalization modeling in federated learning methods, which depicts the user-specific preference.

We compare the item embedding learned by baselines (top3 FedRec methods for performance due to limited space) with our method. Particularly, we select a user randomly from the MovieLens-100K dataset and visualize the embeddings by mapping them into a 2-D space through t-SNE [Maaten and Hinton, 2008]. In this paper, we mainly focus on the implicit feedback recommendation, so each item is either a positive or negative sample of the user. As shown in Figure 2, the item embeddings of positive (blue) and negative (purple) samples are mixed in baselines, where all users share the global item embeddings. However, they can be divided into two clusters by PFedRec. We can easily conclude that our model learns which items the user prefers.

To answer Q2, we compare three usages of item embedding during inference as follows:

• Random: Each client is assigned with item embedding from a random user, i.e., every client runs with its score function and item embedding from a random user.
• Global: We assign each client with globally shared item embedding, i.e., every client runs with its score function and global item embedding.
• Own: It follows our setting that every client runs with its score function and personalized item embedding.

Specifically, we first train PFedRec, then assign the learned item embeddings as the above three ways for inference. As shown in Figure 3, clients with their item embedding achieve the best performance, and clients with item embedding from others degrade significantly. Item embedding from a random user contains little helpful information for inference, even less than the common characteristics in global item embedding. The personalized item embedding learned by PFedRec has been adapted to client preference, and different clients achieve rather distinct item embeddings, depicting the userspecific preference.

6.6 Protection with Local Differential Privacy

To enhance the preservation of user privacy, we integrate the Local Differential Privacy (LDP) technique [Choi et al., 2018] into our framework. Particularly, we add the zero-mean Laplacian noise to the client's item embedding before uploading to the server, i.e., θm = θm +Laplace(0, λ), and λ is the noise strength. We set λ = [0, 0.1, 0.2, 0.3, 0.4, 0.5] to test our method's performance.

As shown in Table 4, performance declines slightly as the noise strength λ grows, while the performance drop is still acceptable. For example, when we set λ = 0.4, the performance is also better than baselines in most cases. Hence, a moderate noise strength is desirable to achieve a good balance between recommendation accuracy and privacy protection.

7 Conclusion

This paper proposes a novel personalized federated recommendation framework to learn many on-device models simultaneously. We are the first to design the dual personalization mechanism that can learn fine-grained personalization on both users and items. This work could be fundamental work to pave the way for implementing a new service architecture with better privacy preservation, fine-grained personalization, and on-device intelligence. Given the complex nature of modern recommendation applications, such as coldstart problems, dynamics, using auxiliary information, and processing multi-modality contents, our proposed framework is simple and flexible enough to be extended to handle many new challenges. Moreover, the proposed dual personalization is a simple-yet-effective mechanism to be easily integrated with existing federated recommendation systems.

Acknowledgements

Chunxu Zhang and Bo Yang are supported by the National Key R&D Program of China under Grant Nos. 2021ZD0112501 and 2021ZD0112502; the National Natural Science Foundation of China under Grant Nos. U22A2098, 62172185, 62206105 and 62202200; Jilin Province Capital Construction Fund Industry Technology Research and Development Project No. 2022C047-1; Changchun Key Scientific and Technological Research and Development Project under Grant No. 21ZGN30.