BEAT2AASIST: BEATs Feature Splitting with Dual-Branch AASIST for Environmental Sound Deepfake Detection

1 Introduction

Recent progress in audio generation technologies has enabled the synthesis of highly realistic soundscapes extending beyond speech to a wide range of environmental sounds. Modern text-to-audio (TTA) and audio-to-audio (ATA) generative models are now capable of producing immersive and convincing audio content, which has accelerated applications in film production, gaming, and virtual or augmented reality. At the same time, these advances raise serious security concerns. Fabricated environmental sounds can be maliciously injected into recordings or disseminated as misleading evidence, undermining the credibility of audio-based information. These emerging threats have intensified the demand for reliable Environmental Sound Deepfake Detection (ESDD) systems.

While deepfake detection has been extensively studied in the speech domain, ESDD remains relatively underexplored. In speech-based spoofing detection, substantial progress has been achieved through the adoption of largescale self-supervised learning (SSL) foundation models, particularly when combined with multi-layer fusion of transformer features and vocoder-based data augmentation [Han et al., 2023; Mart´ın-Do˜nas et al., 2024; Wu et al., 2024; Shin et al., 2024; Wang and Yamagishi, 2023; Wang and Yamagishi, 2024]. Models such as wav2vec2 [Baevski et al., 2020], HuBERT [Hsu et al., 2021], and WavLM [Chen et al., 2022] have become dominant front-end representations. These approaches have consistently delivered state-of-the-art results in major challenges such as ADD 2023 [Yi et al., 2023] and ASVspoof5 [Wang et al., 2026], demonstrating the effectiveness of SSL pre-training for robust spoofing detection.

In contrast, ESDD poses additional challenges due to the significantly higher variability of acoustic conditions and the diversity of environmental sound categories. Environmental audio encompasses a broad range of non-linguistic events with less structured temporal patterns than speech, making generalization to unseen conditions substantially more difficult. To address these challenges, recent studies have explored SSL foundation models tailored for environmental audio, most notably BEATs [Chen et al., 2023] and EAT [Chen et al., 2024]. Among them, BEATs pre-trained on the largescale AudioSet-2M corpus has shown strong generalization across diverse sound events and has emerged as a particularly effective representation for environmental sound analysis.

This research direction is formalized by the Environmental Sound Deepfake Detection Challenge (ESDD 2026) [Yin et al., 2026], which introduces the first large-scale benchmark dedicated to ESDD based on the EnvSDD dataset. EnvSDD consists of 45.25 hours of real audio and 316.7 hours of fake audio, covering a wide range of environmental sound types. The challenge is structured into two tracks reflecting real-life scenarios. Track 1 evaluates generalization to unseen TTA / ATA generators, while Track 2 simulates a more challenging black-box setting with extreme uncertainty and limited data, where only 1% of black-box training data is available. The official baseline, BEATs-AASIST, highlights the importance of combining environmental-sound-centric SSL representations with robust anti-spoofing back-end architectures.

In this paper, we propose BEAT2AASIST, an enhanced framework that extends the BEATs-AASIST baseline to better capture the heterogeneous characteristics of environmental sound deepfakes. Unlike the baseline, which treats the BEATs output as a flattened sequence of tokens, inherently obscuring the explicit spectrotemporal structure, our approach introduces a dual branch AASIST architecture. By explicitly splitting the representations along either the frequency or channel dimension, we recover and independently model these structural components. This design enables specialized modeling of distinct acoustic artifacts, such as frequency dependent or channel specific spoofing patterns, before aggregating the branch outputs for final prediction.

To further enhance representation quality, we introduce three multi-layer fusion strategies that aggregate information from the top-k transformer layers of BEATs. By leveraging intermediate representations through concatenation, CNNgated, and SE-gated fusion mechanisms, the model captures multi-scale acoustic cues beyond the final-layer embedding. In addition, we apply vocoder-based data augmentation using several high-quality, publicly available GAN-based vocoders to expose the model to diverse synthesis artifacts, improving robustness to both unseen-generator and black-box scenarios.

We evaluate our approach on the EnvSDD dataset under both challenge tracks. Our system achieves competitive performance, ranking 3rd in Track 2 with EER 0.35% and 4th in Track 1 with EER 1.60% in the ESDD 2026 Challenge, demonstrating strong generalization to unseen generators as well as robustness under black-box low-resource conditions. These results validate the effectiveness of our architectural and training strategies for ESDD.

2 Backgrounds

2.1 FoundationModels for Spoofing Detection

With increasingly realistic spoofing speech from advanced text-to-speech (TTS) and voice conversion (VC) systems, spoofing speech detection (SSD) has become a key task in speech processing. Recent SSD approaches have shifted from handcrafted features to end-to-end models operating on raw waveforms (e.g., RawNet2 [Tak et al., 2021a]) and, more prominently, to SSL foundation models that provide strong transferable representations. Because spoofing artifacts can be subtle and localized, attention mechanisms and graph-based methods (e.g., RawGAT-ST [Tak et al., 2021b], AASIST) are often employed to capture complex spectrotemporal dependencies. Thus, hybrid systems that combine SSL front-ends with attentive/graph back-ends have become an effective design for robust spoofing detection.

2.2 BEATs-AASIST

BEATs (Bidirectional Encoder representation from Audio Transformers) [Chen et al., 2023] is a general audio foundation model designed to learn semantic representations from large-scale unlabeled audio. It introduces an iterative pretraining framework that alternates between an acoustic tokenizer and a Transformer-based audio SSL model, formulating masked prediction over discrete acoustic tokens rather than continuous reconstruction. Through mutual refinement via knowledge distillation, the tokenizer and SSL model progressively capture high-level semantics and discard the redundant details for audio. BEATs employs a Transformer encoder to model temporal and spectral context and is compatible with various masked audio modeling backbones. Trained on large-scale audio corpora such as AudioSet-2M, it achieves strong performance across diverse audio understanding tasks. Owing to its robust transfer learning capability, BEATs serves as an effective front-end for ESDD, where modeling heterogeneous acoustic patterns and background conditions is critical.

AASIST (Audio Anti-Spoofing using Integrated SpectroTemporal Graph Attention Networks) [Jung et al., 2022] is an end-to-end deep learning framework specifically developed for voice spoofing detection. AASIST models audio representations as spectro-temporal graphs and employs graph attention mechanisms to capture discriminative features across frequency and time dimensions. By jointly learning temporal and spectral relationships, the model effectively identifies subtle artifacts introduced by spoofing and synthesis processes. AASIST is typically initialized randomly and optimized through supervised training on spoofing detection tasks, enabling it to adapt its graph-based reasoning to taskspecific spoofing characteristics.

BEATs-AASIST integration combines self-supervised acoustic representations with graph-based anti-spoofing reasoning. In this framework, the BEATs encoder serves as a robust feature extractor that provides high-level, environmentaware embeddings, while the AASIST back-end operates as a discriminative classifier that models spectro-temporal relationships within these embeddings. This hybrid architecture has been adopted as the official baseline in the ESDD 2026

Challenge and has demonstrated strong generalization to diverse and unseen environmental sound manipulations.

2.3 Multi-layer Fusion of Transformer Layers

Recent studies have shown that using only the final layer of a large pre-trained Transformer is often suboptimal for downstream tasks. Because Transformer encoders learn hierarchical representations from low-level acoustic patterns in lower layers to higher-level semantic information in higher layers, aggregating features from multiple layers generally yields richer and more robust representations than relying on a single layer.

This insight is supported by the SUPERB benchmark for speech, which employs the learnable weighted-sum of the features from all transformer layers for downstream evaluation, where the layer-weights are task-specific [Yang et al., 2024]. Similarly, WavLM adopts layer-wise aggregation in speaker recognition, demonstrating improved robustness to acoustic variability [Chen et al., 2022]. Multi-layer fusion following the SUPERB setup has also proven effective in audio deepfake detection [Mart´ın-Do˜nas et al., 2024].

Motivated by these findings, our work adopts a top-k layer fusion strategy to exploit complementary information across different transformer layers, offering stronger generalization and robustness than single-layer representations.

2.4 Neural Vocoders

Neural vocoders have emerged as an efficient and scalable source of synthetic audio for data augmentation in spoofing and deepfake detection. Since constructing comprehensive spoofing datasets is challenging due to diverse realworld attack pipelines, prior studies show that effective countermeasures can be trained using vocoder-generated audio alone, without relying on full TTS or voice conversion systems [Wang and Yamagishi, 2023; Wang and Yamagishi, 2024].

HiFi-GAN [Kong et al., 2020] is a GAN-based neural vocoder that achieves computationally efficient and highfidelity speech synthesis by effectively modeling periodic patterns in audio signals. Its strong generalization to unseen speakers and noisy conditions makes it particularly suitable for large-scale spoofed audio generation. Recent GAN-based vocoders such as UnivNet [Jang et al., 2021] and BigVGAN [gil Lee et al., 2023] further advance high-fidelity audio synthesis and generalization. UnivNet employs a multiresolution spectrogram discriminator leveraging spectral representations at different scales, enabling robust waveform generation for both seen and unseen speakers. BigVGAN focuses on universal generalization by introducing a periodic activation function and anti-aliased representation, allowing stable synthesis across varied environments without fine-tuning.

Overall, neural vocoders provide a practical means of generating diverse spoofing examples, supporting the development of robust and generalizable deepfake detection systems when real attack data is limited.

3 Methods

Our proposed framework BEAT2AASIST extends the official BEATs-AASIST baseline by introducing targeted architectural and training-level modifications that improve the modeling of diverse acoustic cues inherent in environmental sound deepfake detection.

In the baseline pipeline, the self-supervised BEATs encoder takes a mel-spectrogram as input and partitions it into frequency-time patches of size 16×16. Each patch is embedded into a 768-dimensional token, producing a flattened feature sequence of shape (H×W, 768), where H and W correspond to the number of patches along the frequency and time axes, respectively. However, this flattening process inherently obscures the explicit spectrotemporal structure of the input. Consequently, the resulting representations are directly forwarded to the AASIST back-end without explicit spatial or channel distinction, potentially limiting the modeling of local frequency dependencies.

Building upon this baseline, our framework introduces three complementary improvements:

1. a dual branch AASIST architecture that reorganizes the flattened BEATs representations to recover frequency or channel wise structural information and processes them with parallel AASIST branches;
2. multi-layer fusion strategies that aggregate information from the top-k transformer layers instead of relying solely on the final layer in the BEATs; and
3. vocoder-based data augmentation to improve robustness against diverse and previously unseen spoofing attacks during training.

3.1 BEAT2AASIST Architecture

The proposed BEAT2AASIST extends BEATs-AASIST by splitting BEATs-derived representations along frequency or channel dimension and processing them with a dual-branch AASIST back-end as illustrated in Fig. 2. Given the BEATs output feature sequence of shape (H×W, 768), the proposed architecture divides the representation into two disjoint subsets, each of which is processed by an independent AASIST network. The outputs from the two AASIST branches are then concatenated to produce the final prediction. Unlike the baseline BEATs-AASIST pipeline that feeds a single unified representation into one classifier, our design explicitly separates the feature space and enables specialized modeling of distinct spoofing characteristics. We investigate two complementary splitting strategies that target different dimensions of the BEATs representation.

Frequency-based Feature Split

For frequency-aware modeling, the BEATs output tensor is first reshaped into (H, W, 768) and evenly divided along the frequency axis, yielding low- and high-frequency feature subsets. Each subset is then flattened back to (H/2×W, 768) and processed by a separate AASIST branch. By explicitly decoupling low- and high-frequency components, this strategy allows each branch to focus on distinct spectral characteristics, improving sensitivity to frequency-dependent spoofing artifacts that often arise from synthesis operations.

Channel-based Feature Split

Alternatively, the BEATs representation is partitioned along the channel dimension into two tensors of size (H×W, 384), which are independently fed into parallel AASIST networks. This channel-wise decomposition encourages each branch to specialize in different subspaces of the learned BEATs embeddings, enabling the model to capture complementary channel-specific cues embedded in the high-dimensional representation. Such specialization is particularly beneficial for modeling subtle acoustic inconsistencies that may not be uniformly distributed across embedding channels.

3.2 Multi-layer Fusion

Transformer-based encoders capture complementary acoustic information across different layers, ranging from low-level spectral patterns to higher-level semantic cues. Relying only on the final layer may therefore limit the expressive capacity of the learned representation. To address this limitation, we adopt a multi-layer fusion strategy that aggregates the representations, say h13-k t , . . . , h12 t ∈R768 for each token t with 1 ≤ t ≤H×W, from the top-k layers among the 12 transformer layers of BEATs. Three different fusion strategies are illustrated in Fig. 3.

Concatenation Fusion

In the simplest fusion scheme, the outputs of the selected topk layers, each with shape (H×W, 768), are stacked along a newly introduced layer dimension to form a tensor of shape (H×W, k, 768). This fused representation is directly forwarded to the downstream deepfake detection network without additional layer-weighting. Despite its simplicity, this approach preserves all layer-wise information from the selected layers and allows the classifier to implicitly learn inter-layer relationships.

CNN-gated Fusion

To adaptively weight transformer layers based on input characteristics, we design a convolutional gating network conditioned on the input mel-spectrogram. The spectrogram is processed by a lightweight CNN composed of three 2D convolution layers, each followed by batch normalization, non-linear activation, and max pooling. The resulting feature map is then passed through global average pooling and a linear layer with softmax activation to produce a normalized k layer-weights, say α13-k , . . . , α12 , which are applied to the corresponding transformer layer outputs to compute a weighted-sum,

thereby enabling the model to emphasize transformer layers that are more informative for a given acoustic input. The unified feature representation ˜ht is subsequently split along the frequency or channel dimension and processed by the dualbranch AASIST back-end.

SE-gated Fusion

Unlike the CNN-gated approach conditioned on the input spectrogram, the SE-gated fusion strategy derives layerweights directly from the transformer layer outputs themselves. Specifically, the top-k layer representations are first aggregated and then passed through a Squeeze-andExcitation (SE) module [Hu et al., 2018], which captures global dependencies across the selected layers. The resulting layer-wise attention weights α13-k , . . . , α12 are then used to combine the layer representations into a unified embedding, enabling coherent acoustic patterns while selectively emphasizing the most relevant layers for deepfake detection.

3.3 Vocoder-based Data Augmentation

To enhance robustness against diverse spoofing conditions defined in the ESDD 2026 Challenge, we adopt vocoderbased data augmentation during training. Real-world environmental sound deepfakes can be generated by a wide range of unseen TTA/ATA generators, making it difficult to construct sufficiently diverse training data from limited generators alone. Neural vocoders provide an efficient way to simulate such variability by introducing distinct synthesis artifacts while preserving the semantic content of the original audio.

In accordance with the challenge design, we employ different augmentation strategies for the two tracks. For Track 1 (ESDD in Unseen Generators), which focuses on generalization to unseen TTA / ATA models, we use HiFi-GAN to generate spoofed audio via copy-synthesis. HiFi-GAN produces high-quality waveforms with realistic periodic structures, allowing the model to learn general spoofing cues without overfitting to specific generators.

For the more challenging Track 2 (Black-Box LowResource ESDD), which simulates extreme uncertainty and limited training data, we further expand augmentation by incorporating UnivNet and BigVGAN in addition to HiFiGAN. These vocoders exhibit complementary generalization behaviors across acoustic conditions and recording environments, thereby introducing a broader spectrum of synthesis artifacts. This diversity is particularly important under blackbox settings, where the test-time generation process is entirely unknown.

By leveraging multiple vocoders with different inductive biases, our augmentation strategy encourages the model to focus on intrinsic spoofing characteristics rather than vocoderspecific artifacts, leading to improved generalization across both unseen-generator and black-box scenarios.

4 Experiments

4.1 Dataset

The EnvSDD dataset [Yin et al., 2026] is the first large-scale dataset designed for ESDD, consisting of 45.25 hours of real audio and 316.7 hours of fake audio. The training set for Track 1 contains 27,811 real audio samples and 111,244 fake samples, where the fake audio is synthesized using a fixed set of known TTA/ATA generators. The Track 1 test set includes 1,000 real and 3,000 fake samples, with fake audio generated by previously unseen models, thereby explicitly evaluating generalization to unseen generators [Yin et al., 2026]. The training set for Track 2 includes all samples from the Track 1 training set, along with an additional 270 real audio samples and 1,083 fake samples. The generation process of these additional fake samples is undisclosed, and thus they are referred to as black-box data. The Track 2 test set consists of 1,994 real and 7,980 fake audio samples, all of which are generated under black-box conditions, making Track 2 representative of real-world deployment scenarios. Table 1 summarizes the detailed statistics of the EnvSDD dataset across different usage splits and tracks, highlighting the significant imbalance between real and fake samples as well as the increased scale and uncertainty introduced in Track 2. This design allows systematic evaluation of both unseen-generator generalization (Track 1) and robustness to unknown synthesis pipelines (Track 2). All audio samples are resampled to 16 kHz and have a duration of 4 seconds. For model training, we used only the training datasets of Track 1 and Track 2, respectively.

4.2 Experimental Setup

For training the BEAT2AASIST model, we employed the BEATs-iter3 version, which was pre-trained on the AudioSet dataset, and fine-tuned all parameters. Input audio clips were either trimmed or padded through repetition to a fixed length of 4 seconds. Each audio sample was converted into a melspectrogram with a frame length of 25 ms, a frame shift of

10 ms, and 128 mel bins. For data augmentation, we applied SpecAug [Park et al., 2019] and additionally generated fake audio using vocoders. As vocoders, we used HiFi-GAN, UnivNet, and BigVGAN. All vocoder-generated fake audio was resampled to 16 kHz before being used. We trained the model using the Adam optimizer with a learning rate of 10-6 and weight decay of 10-4 . The batch size was set to 32, and training was conducted for 20 epochs. Because the dataset contains significantly more fake audio than real audio, we applied class weighting to the cross-entropy loss, setting the weights to (fake, real) = (0.1, 0.9).

4.3 Experimental Results

We evaluated our models on the official test sets of both tracks using equal error rate (EER) as the evaluation metric. The Eval EER corresponds to performance on the progressphase data, whereas the Test EER reflects the final rankingphase results. Table 2 summarizes the Track 1 performance, where Systems 1-3 are individual models and Systems 4-5 are ensemble variants constructed from them. Table 2 also presents the Track 2 results, with Systems 1-4 denoting individual models and Systems 5-7 representing their corresponding ensembles. The specific weights for the ensemble models were empirically tuned based on the individual model accuracy observed during the challenge participation. Detailed model configurations are provided in the table descriptions, and the vocoders used for generating synthetic spoofed audio are noted where applicable.

The evaluation results demonstrate the competitiveness of our proposed approach. In particular, the model exhibits strong performance on Track 2, which focuses on detecting black-box fake audio.

4.4 Ablation Study

To investigate how much each component of the proposed BEAT2AASIST contributes to performance improvement, we conducted additional ablation experiments. Using the single BEAT2AASIST models employed in Track 1 and Track 2, we evaluated the performance degradation caused by individually removing the feature split architecture, multi-layer fusion, and vocoder augmentation components. All additional experiments were conducted under the same experimental settings, except for the use of a smaller learning rate 7 × 10-7 for more fine-grained training. This adjustment of the learning rate led to slight performance improvements in several models. The results of the ablation experiments are summarized in Table 3.

Across most single BEAT2AASIST models, the largest performance degradation was observed when vocoder augmentation was removed. This effect is particularly pronounced in Track 2, indicating that vocoder augmentation plays a critical role in enhancing robustness under black-box conditions, where attack generators and synthesis processes are unknown at training time. By exposing the model to diverse vocoder-induced artifacts, this component substantially improves generalization to unseen spoofing strategies.

In contrast, removing the feature split architecture or the multi-layer fusion module consistently degrades performance across both Track 1 and Track 2, suggesting that these components provide track-agnostic structural benefits. The feature split architecture enables more effective discrimination of heterogeneous spoofing characteristics by introducing subspace specialization, while multi-layer fusion facilitates richer representation learning from the transformer-based encoder. Together, these results indicate that vocoder augmentation primarily addresses distributional uncertainty in blackbox scenarios, whereas feature splitting and multi-layer fusion contribute complementary and robust gains across different evaluation conditions.

4.5 Comparison with other Top-Performing Systems

As summarized in Table 4, top-performing systems in the ESDD 2026 Challenge are predominantly built upon strong environmental audio foundation models such as EAT, BEATs and SSLAM, often combined with advanced back-end architectures and ensemble inference. The highest-ranked approaches in both tracks employ large ensembles (up to five models), indicating that model diversity plays a key role in achieving peak performance. The SSLAM+FFN system [Guo et al., 2025], which ranked 3rd in Track 1 and 4th in Track 2, demonstrates that competitive results can be obtained using a frozen SSL encoder with lightweight backend modeling and intermediate-layer fusion. In contrast, BEAT2AASIST adopts a task-specific architectural design that explicitly models heterogeneous spoofing cues via feature splitting and multi-layer fusion. Despite using fewer ensemble components, our method achieves 4th place in Track 1 and 3rd place in Track 2, suggesting that architectural inductive biases can partially compensate for reduced ensemble size and offer an efficient alternative to large-scale ensemblebased systems.

5 Conclusion

This paper introduced BEAT2AASIST, an enhanced framework for environmental sound deepfake detection that addresses the growing risks posed by increasingly realistic audio generation technologies. Beyond the specific architecture, our approach highlights the importance of explicitly modeling heterogeneous acoustic subspaces when applying tokenbased audio foundation models such as BEATs to spoofing detection tasks. By adopting a dual-branch AASIST design that splits BEATs representations along frequency or channel dimensions, the proposed method captures complementary spoofing cues that may be attenuated in unified representations. Multi-layer transformer fusion further enriches acoustic representations, while vocoder-based data augmentation improves robustness to unseen and black-box attacks.

Experiments on the EnvSDD dataset demonstrate the effectiveness of BEAT2AASIST, achieving competitive performance in the ESDD 2026 Challenge across both tracks. These results suggest that architectural inductive biases and targeted data augmentation can partially compensate for reliance on large-scale ensemble inference, offering a practical and efficient alternative for real-world ESDD scenarios.

Beyond the challenge performance, BEAT2AASIST is particularly relevant for real-world deployment, where environmental sounds exhibit high variability in source, context, and recording conditions, and where attack models are often unknown in advance. The modular design of our framework allows it to flexibly adapt to different operating conditions, making it suitable for applications such as forensic audio analysis, multimedia content verification, and security monitoring systems.

Several promising directions remain for future work. First, incorporating temporal or event-level localization could enable not only detection but also attribution of spoofed segments within long audio streams. Second, extending the framework to jointly leverage multimodal cues, such as synchronized visual or textual information, may further improve robustness against sophisticated attacks. Finally, exploring more adaptive or continual learning strategies could help the model remain effective as audio generation technologies continue to evolve.

Acknowledgments

This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government (MSIT) (No. RS-2024-00341075, RS-2026-25477127)