Boosting Knowledge Transfer and Retention with Brain-inspired Multi-View Incremental Learning

1 Introduction

Across many real-world applications, such as medical imaging [Weng et al., 2024], social network analysis [Su et al., 2026; Wu et al., 2023b], and recommendation systems [Li et al., 2024b], data are often observed as incremental views, where sensors or modalities become available over time rather than simultaneously. However, most existing multi-view learning methods [Zhao et al., 2017; Xu et al., 2024] are designed under a static setting and encounter two fundamental challenges in this incremental scenario. On one hand, they typically require retraining from scratch whenever a new view is introduced, leading to prohibitive computational overhead. On the other hand, when trained incrementally, they suffer from catastrophic forgetting, failing to retain the knowledge acquired from earlier views.

∗Corresponding Authors.

As illustrated in Figure 1(a), static multi-view models degrade significantly under view-incremental settings (blue curve). This degradation arises because knowledge learned from previous views cannot be effectively transferred to facilitate learning on newly arriving views. Consequently, the model becomes overly dependent on the quality of the current view, limiting its ability to integrate information across views and to generalize. This phenomenon can be further understood from an optimization perspective, as shown in Figure 1(b). Each ellipse corresponds to an optimal parameter region associated with a particular view space. After learning view A, the model converges to an optimal region θ∗A. When a new view B arrives, the optimization process is driven solely by gradients from view B, causing the parameter updates to drift toward θ∗B and overwrite the knowledge in θ∗A. Ideally, the optimization trajectory should be constrained by previously learned views, enabling convergence to a parameter region that jointly accommodates multiple view spaces (red curve). The inability to achieve this coordinated update highlights the intrinsic rigidity of existing incremental multi-view learning paradigms. Moreover, as shown in Figure 1(c), effectively modeling the consistency and complementarity between old and new views requires strong knowledge integration rather than isolated view-wise adaptation.

Interestingly, the human brain exhibits a markedly different learning behavior in analogous settings. It can continuously incorporate new sensory information while preserving previously acquired knowledge, a capability supported by dynamic neural interactions and memory consolidation mechanisms [Wang et al., 2025; Chiel and Beer, 1997; Ito et al., 2022]. Neuroscientific studies suggest that this flexibility is enabled by the brain's ability to reorganize synaptic connections and functional networks in response to new stimuli, while maintaining stable representations for existing memories [Deco et al., 2011; Hassabis et al., 2017]. In particular, the hippocampus facilitates the coexistence of plasticity and stability by forming partially decoupled yet interacting memory pathways, allowing new information to be integrated without destructive interference [Kumaran et al., 2016; Scoville and Milner, 1957; Squire, 1992]. Furthermore, multisensory integration has been shown to enhance memory retention, indicating that coordinated processing across modalities plays a key role in robust learning [Stein and Stanford, 2008].

Motivated by these observations, we propose a braininspired framework for incremental multi-view learning, termed Hebbian View Orthogonal Projection (HVOP), as illustrated in Figure 2. HVOP draws inspiration from neural mechanisms to explicitly address knowledge retention and transfer under evolving view spaces. By introducing lateral connections and Hebbian learning dynamics, the framework approximates an orthogonal projection process that constrains parameter updates, thereby mitigating information loss when new views are introduced. This design enables previously learned view knowledge to actively guide optimization in subsequent learning stages, effectively mimicking the human brain's capacity to integrate and preserve information across multiple sensory channels. The effectiveness of this method has been verified through a large number of experiments in various multi-view benchmark tests. In these experiments, the performance of HVOP was mostly superior to that of traditional and state-of-the-art multi-view learning methods. We demonstrate that biologically inspired dynamic subspace tracking enables orthogonal learning in view-incremental settings. Specifically, the framework employs mechanisms analogous to:

• Lateral connections in neural circuits: These are simulated in our network architecture to facilitate the integration of information across different views, enhancing the model's ability to maintain a coherent representation of the data [Harris and Mrsic-Flogel, 2013].
• Synaptic plasticity via Hebbian learning: This is used to adapt the synaptic weights, ensuring that the connections are strengthened or weakened according to their utility in task performance [Feldman, 2012;

Gerstner et al., 2018]. This adjustment supports the mechanism of orthogonal projection.

• Memory retention and transfer in the hippocampus: We mimic this aspect by introducing a specific Knowledge Transfer Space within our model. This is akin to regions in the hippocampus that are responsible for encoding new memories and consolidating long-term ones, respectively [Cohen, 1993].

2 RelatedWork

Multi-view learning [Wang et al., 2021; Yao et al., 2024; Zhuang et al., 2025a; Xu et al., 2025] integrates and encodes information from multiple sources to derive low-dimensional representations that capture both consistency and complementarity across views [Wu et al., 2023a; Zhuang et al., 2024; Wu et al., 2026]. In contrast, multi-view incremental learning, or multi-view continual learning [Yan et al., 2024; Li et al., 2024a] primarily focuses on mitigating catastrophic forgetting by regularizing weight updates. Research addressing catastrophic forgetting typically centers on two key strategies: knowledge retention and knowledge transfer. Knowledge retention methods aim to preserve information from old tasks during the training of new ones, often through memory mechanisms or regularization techniques [Babakniya et al., 2024; Chen et al., 2021]. Conversely, knowledge transfer methods seek to leverage knowledge from prior tasks to facilitate learning new tasks; common approaches include using outputs from old tasks as guidance via knowledge distillation [Kang et al., 2023; Kumari et al., 2022; Zhou and Cao, 2021]. Overall, achieving an optimal balance between retaining old knowledge and acquiring new knowledge remains challenging. While recent work explores multi-view incremental learning in scenarios with increasing views [Chen et al., 2025b], its emphasis on restricting weight updates may hinder effective learning of new views.

3 Methodology

For multi-view incremental learning, two objectives are essential: (i) preserving knowledge acquired from previous views, and (ii) effectively integrating information from newly arriving views. Learning from new views should therefore avoid interfering with representations formed from earlier ones. Inspired by biological processes in the human brain, the proposed Hebbian View Orthogonal Projection (HVOP) aims to replicate the adaptability to dynamic data observed in natural neural systems. Based on the principles of synaptic plasticity and memory consolidation in biological networks, HVOP integrates and preserves knowledge across dynamic view flows.

3.1 Preliminaries

Notations. Denote given multi-view feature matrices as X = {Xv}Vv=1 , where Xv = [x1, · · · , xN] ∈ RN×dv is the data from the v-th view for any v ∈ {1, · · · , V }. Here, N is the number of samples, V is the number of views, and dv denotes the distinct dimensionality of the feature matrices for each view. Additionally, c represents the number of classes. To quantify forgetting, we use Fi,j to denote the test accuracy on view j after finishing training on view i.

Definition 3.1 (View Forgetting). Given a sequence of view data X = {Xv}Vv=1 , after learning the v view features Xv, the performance of the learned view v is tested, represented as Fv,v. When the learning of the last viewXV is completed, the performance of the model significantly decreases in the previously tested views, shown as Fv,V < Fv,v.

3.2 Model Initialization for Incremental Views

For the existing view data, we use graph convolutional neural networks [Kipf and Welling, 2016; Chen et al., 2023a; Chen et al., 2025a; Zhuang et al., 2025b] and fully connected layers to capture view-specific knowledge.

Upon the arrival of view Xv, we build a kNN graph and obtain the corresponding adjacency Av ∈ RN×N . We adopt a single-layer GCN with parameters shared across views:

Here σ(·) denotes the activation function (ReLU in our implementation), ˜Av = Av + I adds self-loops, and Dev is the corresponding diagonal degree matrix used for normalization. And all views are aligned to a unified feature dimension before applying the shared transformation. After that, we designed a fully connected layer for implementing the classification, which can be represented as

where Hv ∈ RN×c defaults to the final predicted output.

Previous work has shown that the above models perform well in static multi-view learning scenarios. However, dynamic view-level learning has been neglected, where the model should be reused when posing the newly captured views. Therefore, the question is how can incremental view learning be achieved without affecting the performance of models trained on historical views? In order to preserve the knowledge of the old views, we need to impose a restriction on the direction of the gradient update for the next training.

3.3 Orthogonal Projection for Knowledge Transfer

This section defines the learning objective for preventing view forgetting and presents how it can be satisfied through orthogonal constraints. We first state a desideratum that prevents view forgetting (Eq. 3), then translate it into an orthogonal projection operator (Eq. 4), and finally realize this projection online via biologically inspired subspace tracking (Eqs. 6-9).

Assume the input vectors xold ∈ Rn from prior views, span the subspace X. we explicitly formulate the desired condition that new-view updates must preserve the response to old-view inputs. Considering that changes in view information affect the updating of weights, which leads to view forgetting, in order to preserve the knowledge of the old view, we need to impose restrictions on the direction of the gradient ∆WP update for the next view, which makes it orthogonal to the subspace X, satisfactory that ∆WP xold = 0, which xold denotes the past input vectors. This ensures that the gradient update of the weights does not affect the integration of knowledge of past views, demonstrated as

Eq. 3 therefore serves as the fundamental learning objective that subsequent projection techniques are designed to satisfy. Intuitively, Eq. 3 enforces that the model's response on historical inputs remains unchanged after learning a new view, i.e., the update must not alter previously formed representations.

To address this, we specifically designate the principal subspace of X as the Knowledge Transfer Space (KTS). The KTS is pivotal in assimilating crucial information which summarizes dominant historical representations for constraining future updates. Unlike task-based methods such as GPM or OWM, which compute static principal directions from accumulated gradients, KTS serves as a reference subspace that captures dominant representations from previous views, against which new updates are constrained. In other words, KTS acts as a running summary of dominant historical representations, providing the subspace to be protected when learning subsequent views.

If we calculate a projection matrix P to the subspace orthogonal to the KTS, then gradients can be projected as ∆WP = ∆WPT . Inspired by [Saha et al., 2021], for the gradient update of the fully connected layer, it can be adjust by subtracting its projection onto the space spanned by the top k principal component. If K ∈ Rd×k contains the top-k principal directions of historical representations, then the gradient for the classifier can be projected onto the orthogonal complement:

Here, ∇WL is the gradient of loss with respect to weight W, KKT is the projection matrix to the principal subspace of the input feature. Thus, this effectively filters out components of the gradient that align with the principal directions, potentially reducing interference from predominant features that are already well-represented in the model. However, explicitly estimating K by SVD is costly and does not naturally adapt to streaming views.

3.4 Recursive Lateral Connections for Online Projection

This mechanism serves as an online approximation of the orthogonal projection defined in Sec. 3.3. Since explicitly estimating K by SVD is impractical under streaming views, we replace the static principal basis with an online subspace tracker, denoted by R. Inspired by recursive lateral connections in biological neural systems, we introduce a recurrent projection mechanism to integrate new knowledge while suppressing interference with previously acquired representations.

Given an input representation Q, we first project it into a neural subspace via O = RQ, where R encodes the interaction between prior and incoming knowledge. A recurrent feedback term is then computed as

which acts to remove components aligned with previously learned directions. Combining the original and recurrent representations yields

Notably, this mechanism does not introduce a new objective; it provides an efficient realization of the orthogonal projection required by Eq. 3 in an online manner. This operation can be interpreted as a soft orthogonal projection onto the complement of the subspace spanned by prior knowledge.

In this way, the new view knowledge in the KTS can be further integrated and enhanced to provide richer information for global cognition. Here, the Q will change when each view is added, and will spontaneously integrate the knowledge left over from the previous step of the model, represented as

Among them, Qv refers to the representation we fuse after new view v arrives, which is composed of the representation of the current view and the knowledge of the previous view.

To alleviate the view forgetting phenomenon, we introduce an orthogonal projection mechanism inspired by regularization-based methods to impose restrictions on weight updates. In this way, the knowledge update brought by the new view will not conflict or interfere with the already solidly mastered knowledge system of the old view, thus facilitating a more efficient and robust learning process. From Eq. (6), we can conclude that the projection matrix P′ as

We find a striking similarity between Eq. (8) and the orthogonal projection mechanism in Eq. (4). Thus, as long as the recursive lateral connections R are equal or similar to the principal component matrix K, the model can realize orthogonal projections, which inspires us to employ Hebbian learning to dynamically extract the principal component matrix. Oja rule, as an improvement of Hebbian learning, gradually approximates the principal direction of the data by adjusting the weight matrix. Its update formula is:

where Rt denotes the dynamic principal matrix, η refers to learning rate, xt is the input vector and yt indicates the output vector computed from the current weight matrix and inputs.

With this rule, the data is projected and normalized to gradually approach its principal direction, ultimately causing the weight matrix to approximate the principal components of the input data. This adaptive mechanism enables online subspace regularization and mitigates view forgetting without resorting to explicit SVD-based decomposition. In this way, we leverage the biological mechanisms of recursive lateral connections and Hebbian learning to achieve orthogonal projection and facilitate knowledge transfer within a view incremental framework. This makes HVOP fundamentally different from PCA/SVDbased orthogonal methods that require storing past data or recomputing subspaces offline.

To further ensure the structural consistency of the model when dealing with different views, we design the following loss function to ensure that the graph structural information can be preserved in the incremental learning of views: LRE = 12 ∑|T | t=1 ∥At - σ (Q|T | · QT|T |) ∥2F , where At denotes the adjacency in the past view information and |T | is the current training number of views. Despite the fact that the view data is in a constant state of flux, the intrinsic topology of the graph data shows surprising stability. With such constraints, we can capture the consistent representation between view information very well. Besides, for semi-supervised node classification, we calculate CrossEntropy and update parameters in HVOP, as follows LCE = - ∑i∈Ω ∑cj=1 ˆ yij lnyij, where Ω refers to the set of labeled samples, ˆ yij usually denotes to onehot encoding format and c is the amount of classes. Our approach constantly transfers and retains knowledge in the view incremental setting, while approaching a more comprehensive overall perception. We analyze the stability-plasticity tradeoff (retention vs. new-view adaptation) and hyper-parameter sensitivity in Sec. 4.

4 Experiments

To validate the effectiveness of the proposed method, we have designed the experimental section intended to answer the following key evaluation questions (EQs):

• EQ1 Does HVOP achieve desired performance for multiview classification task?
• EQ2 Does HVOP alleviate view forgetting in view incremental learning through orthogonal projection?
• EQ3 Does HVOP capture the association between old and new views when new ones add up, and does it achieve global knowledge integration across views?
• EQ4 How does HVOP behave in terms of training convergence, robustness to different view orders, and sensitivity to key hyper-parameters?

4.1 Experimental Setup

Datasets. We adopt six multi-view graph datasets widely used in different domains to evaluate the performance of HVOP compared to state-of-the-art baselines. Table 1 illustrates the details of all six datasets.

Compared Methods. Besides, we compare the proposed method with classical and state-of-the-art baseline models, including GAT [Veliˇckovi´c et al., 2018], SI [Zenke et al., 2017], MAS [et al., 2018], LGCNFF [Chen et al., 2023b], DUANet [Geng et al., 2021], RCML [Xu et al., 2024] and MVCIL [Li et al., 2024a], MVIL [Chen et al., 2025b].

Evaluation Metric. Four well-known classification evaluation metrics are applied to our experiments, including Accuracy (ACC), Precision (P), Recall (R), and Macro-averaged F1 Score (MAF1). Experimental rounds of the proposed method are 800 per view under 10% randomly labeled data and the average value is taken by repeating the run 3 times.

4.2 Excellent Perception: Comparison to SOTA (EQ1)

We evaluate the effectiveness of our method on the multiview classification task, the results are summarized in Table 2. We first compare it with the conventional static multi-view methods, including DUANet, LGCNFF and RCML, which are designed to simultaneously access all complete views, effectively leveraging inter-view correlations. DUANet excels at integrating reliable evidence by assessing the confidence level of each view. LGCNFF and RCML achieve desired performance by capturing both consistent and complementary information from multi-view data. These methods excel in scenarios where datasets consist of a fixed set of high-quality views, enabling effective exploitation of the available information. However, as the number and variety of views increment, static multi-view learning may fail to perform optimally in such more intricate scenarios.

In dynamic multi-view incremental environments, we incorporate classical baseline GAT into the view incremental framework. And observe that GAT struggles to retain information from previous views, with its performance largely dependent on the quality of the most recent view data rather than the integration of all views. We further introduce continual learning methods that leverage synaptic plasticity, SI and MAS, which aim to capture overall information through plasticity mechanisms. While these methods effectively leverage biological properties to maintain consistency across views, they encounter challenges when handling large category sets. Additionally, we compare our approach to multi-view classincremental learning (MVCIL) methods. MVCIL demonstrates strong performance in handling datasets with multiple views, largely due to its effective retention of previously learned knowledge. However, it still faces limitations when dealing with low-quality views. Overall, the proposed HVOP reinforces the ability to extract maximal information from each view while seamlessly and efficiently integrating the continuously expanding view data.

4.3 Stability in Knowledge Retention: Demonstration of View Forgetting Relief (EQ2)

To verify that HVOP excels at memorizing and integrating knowledge from previous views, we tested it on inductive reasoning in Figure 3. We assessed the performance of both GCN and HVOP under conditions where views were incrementally introduced, focusing specifically on their ability to retain and utilize knowledge from earlier views. The results revealed that GCN exhibited a more significant deterioration in view performance as new views were added. In contrast, HVOP demonstrated a more gradual decline, with instances of stabilizing performance, indicating its resilience to the introduction of new information. Even as new views are continuously incorporated, HVOP is able to preserve and integrate knowledge from previous views. This implies that the HVOP model possesses robust memory retention capabilities, enabling it to maintain a stable understanding of previous views while nurturing the development of a more holistic cognition.

4.4 Mitigating Forgetting: Verification of Knowledge Transfer Ability (EQ3)

We plot the change in performance as the views continue to increase, and add the results of training with only one view as a comparison, on the Figure 4. We observed that the performance of single-view learning is inferior to that of HVOP, indicating that HVOP effectively facilitates knowledge transfer and integration between views during the learning process, thereby enhancing overall cognitive ability. By integrating both coherent and complementary information from multiple views, model gains a more comprehensive understanding of data, resulting in more accurate and robust classifications. Furthermore, we designed the ablation experiment by removing the orthogonal projection module to visualize its importance. The experimental results show that the performance is significantly affected by the fluctuations in the quality of the new view data, showing an unstable behavior. While our HVOP not only underscores the model's exceptional capacity for knowledge accumulation, but also demonstrates its ability to continuously absorb and utilize new knowledge from additional views while preserving previously information.

4.5 Convergence Insights (EQ4)

In Figure 5(a), we show the loss convergence of the NGs dataset under both GCN and HVOP methods. The GCN's loss exhibits sharp fluctuations when new view data is introduced, suggesting that it struggles with effective knowledge transfer and integration across different views. This variability in loss indicates that the GCN model may not be robust to changes in input data, potentially affecting its performance and reliability in real-world applications where data variability is common. In contrast, the HVOP model demonstrates a smoother loss trajectory, indicating that changes in view data have a less significant impact on its performance. This suggests that the HVOP model is better equipped to integrate associations between old and new data, promoting a more stable and continuous progression in overall cognition.

The comparative analysis of loss reduction between the HVOP and GCN models across two different datasets highlights HVOP's superior ability to handle data variability and integrate new information smoothly. This stability is crucial for applications requiring reliable performance in dynamic settings, suggesting that the HVOP framework could be a preferable choice in scenarios where continuous learning and adaptation to new data are required.

4.6 ViewOrder Analysis (EQ4)

In Figure 5(b), we present the impact of different sequential combinations of views on HVOP in Dataset Flower17. Unsurprisingly, all variations of view permutations lead to improvements in the final overall cognition, indicating that HVOP is not overly sensitive to the specific order in which views are introduced. Although different view orders induce distinct learning trajectories and intermediate representations, they consistently converge to a similar performance level, revealing a shared trend of steadily enhanced cognitive ability. This behavior suggests that the proposed orthogonal projection and knowledge transfer mechanisms enable robust integration of multi-view information under diverse streaming orders. In future work, we will further investigate how view sequencing interacts with view quality and complementarity, and how adaptive ordering strategies may further improve overall cognition.

4.7 Parameter Sensitivity Analysis (EQ4)

In this subsection, we performed the sensitivity analysis on the parameters k and hidden, where k denotes the number of neighbors chosen by kNN and hidden denotes the number of hidden layer dimensions in Figure 6.

Impact of k It can be observed that the performance of Animals, NGs and NoisyMNIST increases with the increase of k value. This could suggest that a larger k-value might facilitate better performance. However, the performance of others showed an upward trend with the decrease of k value. For instance, the Flower17's high number of views or YaleB_Extended's relatively small sample size might necessitate a focus on fewer, more relevant neighbors to optimize performance. A lower k value, in this context, could reduce noise and prioritize the most pertinent data points, thereby leading to improved performance outcomes.

Impact of hidden d Three datasets, Iaprtc12, NoisyMNIST, and YaleB_ Extended, showed an upward trend in performance as d increased, while the other datasets showed the opposite trend. Datasets characterized by complex, high-dimensional features, and a substantial number of multi-view settings, generally exhibit enhanced performance with larger hidden values. However, beyond a certain threshold, the performance improvements may plateau. Consequently, the optimal hidden value can differ significantly depending on the intrinsic properties of various datasets, including feature size, class count, and the number of views.

5 Discussions and Conclusions

The traditional multi-view learning frameworks are often limited by their ability to handle static multi-view data, making it difficult to adapt to the dynamic growth and changes in realworld view data. In response, this study proposes a Hebbianbased View Orthogonal Projection framework, called HVOP to overcome this challenge. HVOP emphasizes view incremental learning, aiming to maintain the stability of the overall cognitive structure while focusing on the in-depth understanding of a single view in each learning task. We provide an empirical analysis of view forgetting and formulate view transfer learning as a problem setting. This concept hypothesizes that by effectively mitigating view forgetting, it is possible to achieve smooth transfer and integration of knowledge across views. HVOP method cleverly integrates recursive lateral connection mechanisms with Hebbian learning principles. This combination not only facilitates efficient knowledge extraction across views but ensures deep integration and continual optimization of both new and old view information within the neural network. Experiments exhibited that HVOP significantly enhances the model's adaptability and generalization ability in handling dynamic multi-view data.

Acknowledgments

This work was supported in part by the Brain Science and Brain-like Intelligence Technology-National Science and Technology Major Project under Grant No.2025ZD0215500 and National Natural Science Foundation of China (NSFC) under Grant No.62506084.