Towards Generalized Action Recognition on Low-Resolutions with Domain-Invariant Representation

1 Introduction

Action recognition under low-resolution (LR) conditions [Oguz and Ikizler-Cinbis, 2024] remains a fundamental yet underexplored problem. The resolution mismatch between high-resolution (HR) training data and LR test inputs causes severe performance degradation, as discriminative spatial cues collapse and temporal representations become unstable. Beyond resolution mismatch, low-resolution action recognition suffers from severe cross-domain shifts, including both human-tohuman transfer across environments and human-to-nonhuman transfer across species [Fuchs et al., 2025; Maekawa et al., 2021]. These shifts, driven by differences in anatomy and motion dynamics, are substantially amplified at low resolution. From Fig. 1, low-resolution action recognition is hindered by coupled challenges of spatial detail loss, non-stationary degradation, and severe cross-domain shifts across environments and species. These factors jointly destabilize feature representations and limit the robustness of existing CNN- and Transformer-based models.

Existing solutions address these challenges only partially. Super-resolution-based approaches [Xie et al., 2023; Ledig et al., 2017] reconstruct HR frames prior to recognition, but incur high computational cost and often introduce hallucinated artifacts that disrupt temporal consistency. Domain adaptation methods [Ganin et al., 2017; Ghosh et al., 2025] align source and target distributions, yet commonly rely on fixed target domains and struggle under dynamically varying lowresolution conditions. Multi-resolution and ensemble-based strategies [Meng et al., 2020; Li et al., 2025] improve robustness across scales, but remain largely human-centric and do not readily generalize to cross-species scenarios. These limitations motivate a unified framework that directly addresses resolution-induced domain shifts while enabling robust generalization across both domains and species.

To this end, we propose a generalized Action recognition framework for Low-resolution conditions with Domaininvariant Representation learning, termed ActLDR. It is built upon a three-fold idea: 1) use dual-branch disentanglement and spatial-temporal asymmetry to extract spatial and temporal features from motion dynamics with resolutionsensitive attributes, 2) utilize bidirectional cross-attention to exchange spatial and temporal features, allowing each branch to suppress resolution- and domain-specific noise by attending to complementary cues, 3) exploit invariance induced consistency-driven learning and Gaussian distribution perturbations to enforce stable decision boundaries and distinguish action domain-invariant and bias representations.

The main contributions of this work are as follows: 1) We first construct a generalized action recognition architecture under low-resolutions to address resolution-induced domain shifts and cross-species discrepancies; 2) A unified dualbranch architecture with domain-invariant representations is designed to disentangle invariant features, spatial appearance, and temporal dynamics, yielding high generalization and robustness in low-resolutions; 3) We use feature-level interference and consistency regularization, which generate richer virtual domains instead of target domains, to get stable decision boundaries for mining domain-invariant representations.

2 RelatedWork

We review research lines: CNN-based architectures, Transformer-based video models, low-resolution action recognition, and domain generalization.

CNN-Based Action Recognition. Two-stream networks [Simonyan and Zisserman, 2014; Zhu et al., 2019] separate appearance and motion cues using RGB frames and optical flow, while 3D CNNs such as C3D, I3D, R(2+1)D, SlowFast, X3D, and MF-Net learn joint spatio-temporal representations through volumetric convolutions [Tran et al., 2018; Carreira and Zisserman, 2017; Feichtenhofer et al., 2019; Feichtenhofer, 2020]. These architectures remain strong baselines on standard RGB benchmarks such as UCF101, HMDB51, and NTU RGB+D 120 [Soomro et al., 2012; Kuehne et al., 2012; Liu et al., 2019]. CNN-based models strongly depend on local spatial details and fixed receptive fields. Under low-resolution conditions, discriminative appearance cues collapse, resulting in significant performance degradation and a struggle to generalize across domains, which limits the applicability of cross-resolution and crossspecies scenarios.

Transformer for Video Understanding. Transformer architectures, such as ViViT, TimeSformer, and VideoMAE [Arnab et al., 2021; Bertasius et al., 2021; Tong et al., 2022] demonstrate strong representation learning through self-attention and masked video modeling. Largescale frameworks VideoMAE v2, OmniVec/OmniVec2, and BIKE-achieve near-saturated performance on canonical benchmarks through combinations of self-supervised learning, cross-modal pretraining, and image-to-video knowledge transfer [Wang et al., 2023; Srivastava and Sharma, 2024; Wu et al., 2023]. To improve structural efficiency on RGBonly, pose-guided or motion-aware Transformers [Zhang et al., 2024; Long et al., 2022] introduce auxiliary inductive biases to enhance fine-grained temporal modeling, while efficient attention mechanisms and hybrid architectures (e.g., Mamba-style sequence models [Li et al., 2024]) aim to reduce computational overhead. Although some of these works are inspired by additional modalities (e.g., pose or language), the final inference often remains RGB-based, with auxiliary signals used only during training. Meanwhile, their attention mechanisms rely heavily on rich spatial tokens, making them fragile under severe resolution degradation.

Low-Resolution and Domain Generalization. Superresolution-based methods [Xie et al., 2023; Ledig et al., 2017; Zhang et al., 2019; Isobe et al., 2022] reconstruct highresolution frames with hallucinated textures and temporal inconsistencies before recognition, resulting harm to recognition accuracy and computational cost. Resolution-invariant or multi-resolution models [Meng et al., 2020] mostly assume fixed degradation patterns and struggle with non-stationary resolution changes commonly observed in practice. Motioncentric or skeleton-inspired approaches [Yan et al., 2018; Zhang et al., 2023] mainly rely on reliable spatial cues during feature extraction and degrade sharply under ultra-low resolutions, resulting in difficulty extending to non-human species without specialized annotations. Domain generalization (DG) [Ganin et al., 2017; Ghosh et al., 2025] aims to improve robustness to unseen environments without access to target data. However, transferring RGB action models from humans to animals presents unique challenges, which exhibit distinct kinematic structures, motion periodicities, and anatomical constraints, even when semantic labels overlap (e.g., running or jumping).

To overcome above, this motivates our proposed dualbranch disentanglement architecture with feature interference and consistency-driven learning, which explicitly models resolution-induced perturbations and constructs domaininvariant representations without relying on additional input modalities for generalized action recognition.

3 Methodology

3.1 Preliminaries

We establish the theoretical foundations of domain-invariant representation learning for generalized low-resolution action recognition. We first introduce the core concepts of domains, domain generalization, and domain-invariant representations.

Definition 1 (Domain / Environment [Ben-David et al., 2009]). A domain (or environment) is defined as a joint probability distribution Pm(X, Y ), where X ∈ X denotes the input space and Y ∈ Y denotes the label space.

Definition 2 (Domain Generalization [Muandet et al., 2013]). Domain Generalization (DG) aims to learn a predictive model using only a set of source domains D = {Dm}Mm=1 that generalizes well to an unseen target domain Dt, where Pt(X, Y ) ̸= Pm(X, Y ) for all m.

Definition 3 (Domain-Invariant Representation [Muandet et al., 2013]). Given a feature extractor ϕ : X → Z, the induced representation Z = ϕ(X) is said to be domaininvariant if

Definition 4 (Virtual Domain via Feature Intervention [Shankar et al., 2018]). A virtual domain ˜D is induced by feature-level intervention:

Definition 5 (Prediction Consistency [Xie et al., 2020]). Prediction consistency requires the predictive distribution to remain invariant under feature-level intervention:

where p = h(Z) and ˜ p = h( ˜ Z) denote predictions before and after intervention.

These definitions characterize different notions of invariance, frommarginal distribution alignment to label-sufficient representations and consistency under perturbations. However, they do not specify which components of the learned representation should be invariant, nor how such invariance can be effectively enforced under severe spatial degradation.

Following subsections, we exploit the distinct domain sensitivities of spatial and temporal information to formulate principled assumptions that guide the design of our domaininvariant representation framework. Additional formal definitions related to feature representations and domain-invariant notions (Definitions 6-8) are provided in the Appendix B.

3.2 Problem Setup

We study action recognition on low-resolutions with a domain-invariant representation. The model is trained on source domain, then generalize to an unseen target domain without accessing any target-domain data during representation learning.

Let D = {D1, D2, . . . , DM} denote a set of M source domains. Each domain Dm = {(xmi , ym i )}Nm i=1 consists of labeled video clips sampled from an unknown joint distribution Pm(X, Y ), where x ∈ X denotes a low-resolution video clip with T frames and y ∈ Y is the corresponding action label.

Domain shifts naturally arise from variations in spatial resolution degradation, acquisition conditions, background statistics, motion blur, and even anatomical or behavioral differences across datasets or species. The goal is to learn a representation that captures task-relevant motion patterns while being robust to such domain-specific variations, so that it generalizes to an unseen target domain Dt with distribution Pt(X, Y ).

Following standard representation-level domain generalization protocols, the predictive model is decomposed as

where g(·) denotes a feature extractor and h(·) is a lightweight classification head.

During evaluation on unseen domains, the feature extractor g is completely frozen, and only the classification head h is trained. Under this protocol, the performance on target domains reflects the degree of domain invariance of the learned representation, as defined in the preceding section.

3.3 Domain-Invariant Representation

Domain-Invariant Feature Extraction. In low-resolution action recognition, domain shifts primarily arise from variations in spatial appearance, while temporal motion patterns remain relatively stable across domains. This motivates learning representations that emphasize domain-invariant temporal cues and suppress domain-dependent spatial variations.

Assumption 1 (Spatial-Temporal Asymmetry). In lowresolution action recognition, temporal motion patterns are more stable across domains than spatial appearance features. Let the domain discrepancy between representations Z from domains Di and Dj be defined as

where Di and Dj denote different domains (Definition 1).

Then, for spatial and temporal features Zs and Zt, we assume

This assumption reflects an intrinsic asymmetry in lowresolution videos: spatial appearance is highly sensitive to resolution degradation, illumination, and background variations, whereas temporal dynamics are more closely tied to action semantics and therefore exhibit greater cross-domain stability.

Assumption 2 (Label-Conditional Invariance). We assume that, given the learned representation Z, the relationship between representations and labels is invariant across domains. Formally, the conditional label distributions induced by Z are consistent among domains Di and Dj:

This assumption can be viewed as a conditional relaxation of domain-invariant representations (Definition 3). Rather than enforcing full marginal alignment of Z across domains, it only requires invariance of the label conditional distribution P(Y | Z). Such conditional invariance is widely adopted in domain generalization and forms the basis for learning transferable predictors.

Under the domain generalization setting defined in Definition 2, we analyze the target-domain risk under the frozen representation evaluation protocol.

Theorem 1 (Target Risk Bound with Frozen Representation). Let g : X →Z be a fixed feature extractor and ℓ(·, ·) ∈ [0, 1] a bounded loss. Under label-conditional invariance across domains, i.e., Ps(y | z) = Pt(y | z), the target-domain risk of any classifier h satisfies:

Theorem 1 shows that, under frozen representation evaluation, target-domain performance is governed by the crossdomain discrepancy of the learned representation. As target data are unavailable during training, we reduce this discrepancy indirectly by exploiting the spatial-temporal asymmetry of domain shifts (Assumption 1), emphasizing temporally stable motion cues while suppressing domain-sensitive spatial appearance variations.

Lemma 1 (Dual-Branch Disentanglement). Under Assumption 1, the dual-branch architecture with spatial branch ϕs and temporal branch ϕt satisfies:

for any domains Di and Dj .

Lemma 1 provides a theoretical justification for disentangling spatial and temporal representations in a dual-branch architecture. Motivated by this result, we design a temporal branch to prioritize cross-domain stable motion cues, while the spatial branch captures domain-sensitive appearance information.

Proposition 1 (Temporal Features as Label-Determining Factors). Under Assumptions 1 and 2, temporal features are more closely aligned with label-determining factors than spatial features.

Proof. See Appendix C.3

Proposition 1 complements Lemma 1 by indicating that temporal features are not only more domain-stable but also more predictive of action labels. Accordingly, we treat the temporal branch as the primary source of domain-invariant semantics, while spatial features are incorporated as complementary cues through controlled interaction and prediction fusion.

Domain-Invariant Representation. A remaining challenge is to ensure that the learned representation is domaininvariant not only at the feature extraction stage but also at the final representation level, which requires that decision boundaries in the representation space depend solely on domaininvariant components.

Assumption 3 (Label Stability under Feature Perturbation). Following the notion of virtual domains induced by featurelevel intervention (Definition 4), we assume that small perturbations in the learned representation do not alter the underlying action semantics. Formally, for a perturbed representation ˜ Z = Z + ϵ with ϵ ∼ N(0, σ2 I) and sufficiently small σ, the induced label distribution remains stable:

This assumption characterizes a local stability property of domain-invariant representations in the feature space. It enables enforcing consistency between predictions under feature perturbations, which serves as a practical surrogate for decision-level domain invariance.

Theorem 2 (Invariance Induced by Feature Interference). Based on the virtual domain construction in Definition 4, under Assumption 3, minimizing the expected loss over featureperturbed virtual domains:

encourages learning representations that are invariant to feature-level perturbations.

Theorem 2 provides a principled way to promote domaininvariant representations without explicitly measuring crossdomain discrepancy. Motivated by this result, we implement Gaussian feature interference during training to enforce prediction consistency across virtual domains and improve robustness to unseen domain variations.

Corollary 1. Under the conditions of Theorem 2, minimizing the expected loss over virtual domains leads to representations whose predictions are locally stable with respect to feature perturbations.

Corollary 1 indicates that feature interference encourages locally stable representations under domain-induced perturbations. This supports our use of Gaussian feature interference in Module 3 to improve robustness without explicitly modeling domain shifts.

Based on the prediction consistency objective defined in Definition 5, we establish the following result.

Theorem 3 (Consistency-Induced Invariant Decision Boundaries). Based on the prediction consistency objective in Definition 5, under Assumptions 2 and 3, minimizing prediction inconsistency encourages decision boundaries to rely primarily on domain-invariant components of the learned representation.

Theorem 3 provides a theoretical justification for enforcing prediction consistency as a mechanism to promote decisionlevel domain invariance. Motivated by this result, we design a consistency loss Lcons that penalizes prediction divergence between original and perturbed features, encouraging robust decision boundaries under domain shifts.

Theorem 4 (Generalization Bound with Frozen Representation [Ben-David et al., 2009; Muandet et al., 2013]). If there exists a representation ϕ : X →Z such that the conditional distribution P(Z | Y,D) is invariant across domains and a classifier h achieves low empirical risk on source domains, then the target-domain risk is bounded by:

where ϵ depends on domain divergence and sample complexity.

Theorem 4 provides a general theoretical context for domain generalization under frozen representations, highlighting the importance of conditional invariance for transferable prediction. Our framework instantiates this principle through a set of complementary mechanisms that progressively promote representation- and decision-level invariance.

Based on above, Fig. 2 manifests ActLDR architecture with domain-invariant feature extraction and representation. Specially, utilize dual-branch feature disentanglement and resolution-aligned spatial-temporal feature exchange to extract domain-invariant features. Then, use Gaussian feature interference and consistency-driven learning to generate domain-invariant representations for action recognition.

3.4 Dual-Branch Feature Disentanglement

Motivated by the spatial-temporal asymmetry(Assumption 1) in low-resolution videos, domain-invariant feature extraction requires explicitly separating domain-sensitive appearance cues from more stable motion patterns. To this end, we design a dual-branch architecture that disentangles spatial and temporal information streams, enabling specialized modeling of complementary features.

Given an input video x ∈ RT×H×W×C , ActLDR adopts a dual-branch backbone consisting of a Spatial Branch and a Temporal Branch, producing features ϕs(x) ∈ Rds and ϕt(x) ∈ Rdt , respectively.

• Spatial Branch. The spatial branch focuses on appearance cues and local context, employing convolutional kernels with larger spatial extent (ks × ks) and limited temporal support (kt = 1). While sensitive to domain shifts, these features provide complementary discriminative information.
• Temporal Branch. The temporal branch emphasizes motion dynamics and long-range temporal dependencies using larger temporal receptive fields (kt ≥ 3) with compact spatial footprints (ks = 3). Such temporal patterns are more stable across domains and closely aligned with action semantics.

Through this disentanglement, the temporal branch serves as the primary carrier of domain-invariant and label-relevant information, as theoretically justified in Proposition 1, while the spatial branch complements it with appearance cues that are subsequently regularized through cross-branch interaction.

3.5 Resolution-Aligned Spatial-Temporal Feature Exchange

While dual-branch disentanglement separates complementary information streams, Theorem 1 indicates that improving target-domain performance requires reducing cross-domain representation discrepancy. We therefore introduce controlled cross-branch interaction, allowing stable temporal cues to regularize domain-sensitive spatial features.

At selected depths l ∈ {l1, l2, . . . , lL}, we introduce a Spatial-Temporal Feature Exchange (STFE) module. Let ϕ(l) s ∈ Rd(l) s and ϕ(l) t ∈ Rd(l) t denote the intermediate features at depth l from spatial and temporal branches, respectively.

The STFE module performs cross-branch feature exchange:

where Fst and Fts are learnable transformation functions implemented as:

Here, Attention(·, ·) denotes cross-attention mechanism, ⊙ is element-wise multiplication, and αst, αts ∈ [0, 1]

are learnable gating coefficients. Resolution-aligned projections ensure correspondence between branches, while crossattention highlights mutually consistent patterns. This exchange steers both branches toward a shared invariant subspace, reducing representation discrepancy across domains.

From a theoretical standpoint, this exchange explicitly reduces cross-domain representation discrepancy, as formalized in Proposition 2 (Appendix C.7).

3.6 Gaussian Feature Interference

To explicitly reduce representation sensitivity to domain shifts, we instantiate Definition 4 and Theorem 2 by constructing virtual domains via feature-level perturbation.

For an intermediate representation h(l) ∈ Rd(l) at depth l, Gaussian Feature Interference (GFI) generates perturbed features:

where σ2 l is a depth-dependent variance parameter. By minimizing the expected loss over these virtual domains, the model suppresses residual domain-sensitive components, as implied by Theorem 2.

From a theoretical perspective, this perturbation-based training enforces local smoothness in the representation space and penalizes features that are highly sensitive to perturbations. This property is formalized in Proposition 3, with a complete proof provided in Appendix C.8.

3.7 Consistency-Driven Learning Objective

Following Theorem 3, enforcing prediction consistency under feature perturbation encourages invariant decision boundaries.

We fuse spatial and temporal predictions:

where ps(y | x) = hs(ϕs(x)) and pt(y | x) = ht(ϕt(x)) are predictions from spatial and temporal branches, respectively, and α ∈ [0, 1] is a fusion weight.

We enforce consistency between clean and perturbed predictions:

where ˜ pf(y | x) is the prediction from perturbed features ˜ ϕs(x) and ˜ ϕt(x).

The final training objective combines classification loss and consistency loss:

where Lcls is the cross-entropy loss on the fused prediction, and λcons > 0 balances empirical risk minimization and invariance promotion.

Theoretical analysis shows that this consistency constraint enforces decision boundaries that depend only on domaininvariant components (Propositions 4 and 5 provided in Appendix C.9.).

4 Experiments

We conduct comprehensive experiments to evaluate the effectiveness of the proposed ActLDR framework under three complementary protocols: (1) In-Domain, (2) Cross-Domain Generalization, and (3) Cross-Species Transfer Learning. These protocols are designed to systematically assess resolution robustness, domain invariance, and generalization under increasingly severe distribution shifts.

An overview of the experimental results is summarized in Fig. 3, where Fig. 3(a) reports performance under the normal low-resolution setting, and Fig. 3(b) presents cross-domain and cross-species generalization results. Detailed numerical results and experimental settings are provided in appendix D.

We evaluate our method on four standard human action recognition benchmarks and one animal action dataset: NTU RGB+D 120 [Liu et al., 2019], used as the primary pretraining dataset; UCF101 [Soomro et al., 2012], HMDB51 [Kuehne et al., 2012], and NW-UCLA [Wang and Nie, 2014], used for within-dataset and cross-dataset evaluation; and LoTE-Animal [Liu et al., 2023], used to assess cross-species generalization. All videos are uniformly resized to 56 × 56 to simulate low-resolution inputs.

4.1 In-Domain

We evaluate our method under the In-Domain, where models are trained and tested on the same dataset at low resolution (56 × 56). Implementation details, dataset-specific protocols are provided in Appendix D.1. As summarized in Fig. 3(a) and Table 1, ActLDR consistently outperforms recent CNN- and Transformer-based baselines across all evaluated RGB benchmarks under low-resolution conditions. Notably, ActLDR achieves strong and stable gains on both largescale (NTU120) and small-scale (HMDB51, NW-UCLA) datasets, while using significantly fewer parameters than most Transformer-based models.

4.2 Cross-Domain/Species Generalization

We further evaluate the generalization ability of ActLDR under out-of-domain settings, including both cross-dataset and cross-species transfer. Implementation details, datasetspecific protocols are provided in Appendix D.2. As illustrated in Fig. 3(b) and table 2, ActLDR consistently outperforms CNN- and Transformer-based baselines when transferring from a human source dataset to unseen human datasets as well as to animal action recognition. Notably, competitive performance is maintained even under human-toanimal transfer without any target-domain supervision, indicating that the learned representations emphasize motioncentric and domain-invariant spatiotemporal cues rather than appearance-specific patterns.

4.3 Ablation Study

We conduct ablation studies on NW-UCLA and HMDB51 under the 56×56 low-resolution setting to systematically analyze the contribution of each component in our framework. All variants are trained using the same protocol. In particular, we examine the impact of dual-branch disentanglement, spatial-temporal feature exchange depth and gaussian feature interference by selectively removing each module. Quantitative results are summarized in Table 3, while a more finegrained analysis is provided in Appendix E.

5 Conclusion

In this work, we studied generalized action recognition under low-resolution and cross-species conditions, where severe visual degradation and domain heterogeneity challenge conventional human-centric models. We proposed ActLDR, a unified framework that exploits spatial-temporal asymmetry by disentangling resolution-sensitive appearance cues from stable motion dynamics and enabling their interaction through feature exchange. Building on domain-invariant representation, ActLDR enforces decision-level invariance via feature perturbation and consistency-driven learning, yielding stable predictions. Extensive in-domain, cross-domain, and cross-species experiments show that ActLDR achieves strong performance, robustness, and generalization, with human-toanimal transfer highlighting the effectiveness of temporally grounded and domain-invariant representations.

Future work will strengthen temporal modeling under extreme low resolutions and improve scalability to broader actions and species. We also plan to explore large-scale pretraining and weak or multimodal supervision for better realworld generalization.

Acknowledgments

This work has been partially supported by grants from the Natural Science Foundation of Hebei Province, No.F2025203033; the National Natural Science Foundation of China, No.62276226.