NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms

1 Introduction

The pursuit of musicality is a core objective in music generation research, as it fundamentally shapes how we perceive and experience musical compositions. Symbolic music abstracts music into discrete symbols such as notes and beats, with performance signals (i.e., MIDI) and sheet music (e.g., ABC notation, MusicXML) being the two predominant modalities. Both of them can efficiently model melody, harmony, instrumentation, etc., all of which are crucial for musicality.

Training tokenized representations with language model architectures, such as Transformers [Vaswani et al., 2017], has emerged as a powerful paradigm for symbolic music generation [Huang et al., 2018; Casini and Sturm, 2022]. However, several challenges persist. First, the scarcity of highquality music data [Hentschel et al., 2023] hinders the ability of deep learning models to generate sophisticated compositions. Second, when optimizing a language model's loss function, the focus typically lies in minimizing the discrepancy between the predicted and the ground-truth next tokens, potentially neglecting holistic musical aspects like music structure and stylistic coherence.

Figure 1: An overview of NotaGen's training paradigms.

Insights from the Natural Language Processing (NLP) domain provide a promising approach to overcoming the challenges inherent in symbolic music generation. The success of Large Language Models (LLMs) [Dubey et al., 2024] has established the paradigm of pre-training, fine-tuning, and reinforcement learning as a widely acknowledged framework to improve the quality of text generation and align output with human preferences. These techniques have been successfully adapted for music generation. To overcome the scarcity of high-quality data, large-scale pre-training followed by finetuning on smaller, task-specific datasets has been employed effectively [Donahue et al., 2019; Wu et al., 2024a]. Reinforcement Learning from Human Feedback (RLHF) [Stiennon et al., 2020], transcending next-token prediction approaches, has also shown promising results in music generation [Cideron et al., 2024]. However, to the best of our knowledge, the complete pipeline of LLM training paradigms has not been fully implemented in symbolic music generation. Furthermore, the high cost of RLHF for human annotation highlights the necessity for more efficient and automated solutions.

In this work, we introduce NotaGen (Musical Notation Generation), a symbolic music generation model focused on classical sheet music. Compared to MIDI generation, sheet music generation not only aims to produce artistically refined music, but also emphasizes proper voice arrangement and notation to create well-formatted sheets for performance and analysis. Furthermore, the challenge of sheet music generation is exacerbated by the diverse instrumentation and rich musicality inherent in classical music. The success of LLMs has motivated us to apply the training paradigms to sheet music generation. NotaGen is pre-trained on a corpus of over 1.6M sheets in ABC notation, and fine-tuned on a collection of approximately 9K high-quality classical pieces from 152 composers with "period-composer-instrumentation" (e.g."Baroque-Bach, Johann Sebastian-Keyboard") prompts guiding conditional generation. During reinforcement learning, we introduce the CLaMP-DPO method to further optimize NotaGen's musicality and controllability using the Direct Preference Optimization (DPO) [Rafailov et al., 2024] algorithm. In this approach, CLaMP 2 [Wu et al., 2024b], a multimodal symbolic music information retrieval model, assigns generated samples as "chosen" or "rejected" based on references from the finetuning dataset. Our contributions are two-fold:

• We introduce NotaGen, a symbolic music generation model implementing LLM training paradigms, which significantly outperforms baseline models in subjective A/B tests against human compositions.
• We propose CLaMP-DPO, a reinforcement learning approach that integrates the DPO algorithm with CLaMP 2 feedback, enhancing musicality and controllability of symbolic music generation without relying on human annotation or predefined rewards. This potential is showcased across symbolic music generation models with varying architectures and encoding schemes.

2 RelatedWorks

2.1 Sheet Music Generation

Sheet music generation has been widely studied, with a focus on encoding methods and composition modeling. Score Transformer [Suzuki, 2021] introduces a tokenized representation for sheet music and applies it to piano music generation. Measure by Measure [Yan and Duan, 2024] models sheet music as grids of part-wise bars and employs hierarchical architectures for generation. Compared to the intricate representations used by the models above, ABC notation, a comprehensive text-based sheet music representation, simplifies encoding and facilitates composition modeling, gaining increasing adoption in recent research. The following models utilize the ABC notation: FolkRNN [Sturm et al., 2016], Tradformer [Casini and Sturm, 2022], and Tunesformer [Wu et al., 2023a], specializing in folk melody generation; MuPT [Qu et al., 2024], a large-scale pre-trained model for sheet music, which explores multitrack symbolic music generation.

2.2 Pre-training in Symbolic Music Generation

The success of pre-training in NLP has inspired the application of this technique in symbolic music generation. LakhNES [Donahue et al., 2019] enhances chiptune music generation by pre-training on the Lakh MIDI Dataset [Raffel, 2016]. MuseBERT [Wang and Xia, 2021] adopts masked language modeling [Devlin et al., 2019], while MelodyGLM [Wu et al., 2023c] implements auto-regressive blank infilling [Du et al., 2021] for generation. MelodyT5 [Wu et al., 2024a] leverages multi-task learning [Raffel et al., 2020]. These studies highlight the effectiveness of pre-training in enhancing music generation performance.

2.3 Reinforcement Learning inMusic Generation

Reinforcement learning has long been recognized as a promising approach for enhancing the musicality of music generation models. It has been successfully applied in RL Tuner [Jaques et al., 2017] for melody generation, RL-Duet [Jiang et al., 2020] for online duet accompaniment, RLChord [Ji et al., 2023] for melody harmonization, and [Guo et al., 2022] for multi-track music generation. However, these methods either base their rewards on music theory, which limits flexibility, or tailor them to specific music styles, hindering their generalization to a broader range of music generation tasks. To tackle this problem, MusicRL [Cideron et al., 2024] adopts the RLHF method with extensive human feedback to align the generated compositions with human preference.

3 NotaGen

3.1 Data Representation

ABC notation sheets consist of two parts: the tune header, which contains metadata such as tempo, time signature, key, and instrumentation; the tune body, where the musical content for each voice is recorded. We adopt a modified version-interleaved ABC notation [Wu et al., 2024b; Qu et al., 2024]. In this format, different voices of the same bar are rearranged into a single line and differentiated using voice indicators "[V:]". This ensures alignment of duration and musical content across voices. Furthermore, we remove bars with full rests (containing only "z" or "x" notes), reducing the length to 80.7% on average, while increasing information density.

We employ stream-based training and inference methods to enable long musical piece generation. We annotate the current and countdown bar indices before each tune body line using the label "[r:]". During training, we randomly segment the tune body and concatenated it with the tune header for longer pieces; during inference, we enforce the generation to start from scratch using the bar annotations. If the piece is incomplete within the current context length, we concatenate the generated tune header with the second half of the tune body and continue generating until the final bar.

3.2 Model Architecture

NotaGen utilizes bar-stream patching [Wang et al., 2024] and the Tunesformer architecture [Wu et al., 2023a]. Building upon bar patching [Wu et al., 2023b], bar-stream patching divides the tune header lines and bars into fixed-length patches (padded when necessary), striking a balance between musicality of generation and computational efficiency among sheet music tokenization methods.

NotaGen consists of two hierarchical GPT-2 decoders [Radford et al., 2019]: a patch-level decoder and a characterlevel decoder. Each patch is flattened by concatenating onehot character vectors and then passed through a linear layer to obtain the patch embedding. The patch-level decoder captures the temporal relationships among patches, and its final hidden states are passed to the character-level decoder, which auto-regressively predicts the characters of the next patch. The data representation and model architecture are illustrated in Figure 2.

3.3 Training Paradigms

Pre-training

Pre-training enables NotaGen to capture fundamental musical structures and patterns through next-token prediction on a large, diverse dataset spanning various genres and instrumentations.

The pre-training stage utilized a carefully curated internaluse dataset comprising 1.6M ABC notation sheets. We also preprocessed the text annotations, retaining music-related content such as tempo and expression hints, while removing irrelevant content like lyrics and background information.

All music sheets were transposed to 15 keys (including F♯, G♭, C♯, C♭) for data augmentation. During training, a randomly selected key was used for each piece in every epoch.

Fine-tuning

NotaGen was fine-tuned on high-quality classical music sheet data to further enhance musicality in generation. Spanning from the intricate contrapuntal orchestra suites of the Baroque period to the melodious and harmonically nuanced piano pieces of the Romantic era, classical music encompasses a diverse array of compositional styles and instrumentations, all characterized by exceptional musicality.

Thus, we curated a fine-tuning dataset comprising 8,948 classical music sheets, from DCML corpora [Neuwirth et al., 2018; Hentschel et al., 2021a; Hentschel et al., 2021b; Hentschel et al., 2023], OpenScore String Quartet Corpus [Gotham et al., 2023], OpenScore Lieder Corpus [Gotham and Jonas, 2022], ATEPP [Zhang et al., 2022], KernScores [Sapp, 2005], and internal resources, as listed in Table 1. Sheets with more than 16 staves were excluded due to generation complexity. Each work was assigned with three labels: period, composer and instrumentation. The data distribution is provided in supplementary materials, and the details of each label are explained as follows:

· Period:

• Baroque (1600s-1750s): e.g., Bach, Vivaldi.
• Classical (1750s-1810s): e.g., Mozart, Beethoven.
• Romantic (1810s-1950s): e.g., Chopin, Liszt.
• Composer: The official names of a total of 152 composers, as listed on IMSLP 1 , were included.

· Instrumentation:

• Keyboard: piano and organ works.
• Chamber: instrumental music typically for a small group of performers, each playing a unique part.
• Orchestral: instrumental music for orchestra.

1 https://imslp.org/

• Art Song: vocal music typically for solo or duet voices with piano accompaniment.
• Choral: vocal music for a choir.
• Vocal-Orchestral: works involving both vocal and orchestral elements, including Cantata, Oratorio, and Opera.

In fine-tuning, a "period-composer-instrumentation" prompt was prepended to each piece for conditional generation. This approach challenges NotaGen to produce high-quality compositions, imitate the styles of composers across different periods, and conform to specified instrumentation requirements.

To facilitate NotaGen's learning of appropriate pitch ranges for each instrument while optimizing data utilization, data augmentation during fine-tuning was restricted to the six nearest key transpositions of the original. Keys farther from the original were selected with decreasing probability.

Reinforcement Learning

To refine both the musicality and the prompt controllability of the fine-tuned NotaGen, we present CLaMP-DPO. This method builds upon the principles of Reinforcement Learning from AI Feedback (RLAIF) [Lee et al., 2024] and implements Direct Preference Optimization (DPO) [Rafailov et al., 2024]. In CLaMP-DPO, CLaMP 2 serves as the evaluator within the DPO framework, distinguishing between chosen and rejected musical outputs to optimize NotaGen.

CLaMP 2 is a multimodal symbolic music information retrieval model supporting both ABC notation and MIDI formats. Leveraging contrastive learning, CLaMP 2 extracts semantic features that encapsulate global musical properties. These features encompass comprehensive musical information, including style, instrumentation, and compositional complexity. Meanwhile, they are consistent with human subjective perceptions, as validated by [Retkowski et al., 2024]. In the context of music generation, the objective is to produce pieces which closely resemble the ground truth. Accordingly, it is critical to ensure the alignment of the semantic features between the generated pieces and authentic references.

We introduce the CLaMP 2 Score to quantify the similarity among pieces. To elaborate, we denote P as the set of prompts for NotaGen. For each prompt p ∈ P, Yp represents the corresponding set of ground truth with an average semantic feature ¯ zp. Similarly, each prompt p has a generated set Xp, where each piece xp is associated with a semantic feature zxp .

The CLaMP 2 Score c for a generated piece xp is defined as the cosine similarity between zxp and ¯ zp:

Our goal is to maximize the average, ¯ cxp over Xp, thereby ensuring the music generated for prompt p aligns semantically with the ground truth. It is achieved by employing the DPO algorithm to improve ¯ cxp .

The DPO algorithm optimizes a language model based on preference data, which consists of paired chosen and rejected examples under the same prompts. It eliminates the need of explicit reward modeling. In the proposed CLaMP-DPO algorithm, the fine-tuned model first generates data across the prompt set P . For each generated set Xp, the pieces xp ∈ Xp are sorted according to cxp , with the top 10% selected as chosen set Xpw and the bottom 10% as rejected set Xpl. Additional criteria, such as syntax error checks or the exclusion of ground-truth plagiarism, can be applied to refine these sets. Finally, the chosen and rejected pairs (xpw, xpl) are randomly selected and combined into preference data for optimization.

Given a prompt p, an auto-regressive language model predicts the next token based on its policy πθ, where θ represents the model parameters. The probability of generating a chosen data xpw is πθ(xpw|p), and that of generating a rejected data xpl is πθ(xpl|p). To prevent excessive drift from the initial model that generates the preference data and ensure diversity in the generated content, the initial model policy, i.e., the reference model policy πref is introduced and kept frozen during optimization. The objective function to be minimized in DPO is given by:

where σ is the sigmoid function, D is the preference dataset, and β is the hyperparameter that controls the deviation between πθ and πref.

The optimization process increases the relative log probability of chosen data over rejected data. However, we observed a decrease in πθ(xpw|p), leading to degraded outputs. To mitigate this issue, we adopt the DPO-Positive (DPOP) objective function [Pal et al., 2024], which incorporates a penalty term to stabilize πθ(xpw|p):

where the hyperparameter λ controls the impact of penalty.

Algorithm 1: Iterative CLaMP-DPO

Input: Fine-tuned policy π0 θ , CLaMP 2 model C, prompt set P , fine-tuning dataset Y Parameter: Iterations K, DPO hyperparameter β, DPOP hyperparameter λ, optimization steps N, learning rate η Output: Optimized policy πK θ # Initialize ground-truth features foreach prompt p ∈ P do ¯ zp ←Avg(C(yp)), ∀yp ∈ Yp end # Iterative Optimization for k ← 1 to K do # Construct preference data foreach prompt p ∈ P do Xk-1 p ← πk-1 θ (p) # Generate on p foreach piece xk-1 p ∈ Xk-1 P do zxk-1 p ←C(xk-1 p ) cxk-1 p ←Eq. (1)(zxk-1 p , ¯ zp) end Xk-1 pw ,Xk-1 pl ← Select(Xk-1 p , Sort(cxk-1 p )) end # Optimize using DPO πref ← πk-1 θ for i ← 1 to N do Sample prompt p ∼ P Sample pairs (xpw, xpl) ∼ (Xk-1 pw ,Xk-1 pl ) θ ← θ - η∇θLDPOP(πθ , πref, xpw, xpl, p, β, λ) end πk θ ← πθ end return πK θ

The fine-tuned model is optimized by minimizing LDPOP in Eq.(3) for a specified number of steps, completing the process of CLaMP-DPO algorithm. Notably, CLaMP-DPO supports iterative optimization. After the first round, the model generates a new set X′p . Using CLaMP 2, we construct new chosen and rejected sets, X′pl and X′pw , allowing the model to undergo further optimization via Eq.(3).

4 Experiments

4.1 Settings

The experiments are divided into two parts. The first part assesses CLaMP-DPO's ability to improve the controllability and musicality of symbolic music models. The second part compares the musicality of NotaGen with baseline models. Along with the pre-trained NotaGen, we selected two pre-trained symbolic music generation models as baselines: MuPT [Qu et al., 2024] and Music Event Transformer (MET) 2 [SkyTNT, 2024]. All models adopt language model architectures and are trained auto-regressively. A brief overview of their architectures and pre-training procedures follows:

2 https://huggingface.co/skytnt/midi-model-tv2o-medium

• NotaGen features a 20-layer patch-level decoder and a 6-layer character-level decoder, with a context length of 1024 and a hidden size of 1280, totaling 516M parameters. It was pre-trained on 1.6M ABC notation sheets, augmented to 15 key transpositions. The AdamW optimizer [Loshchilov and Hutter, 2019] was utilized with a learning rate of 1e-4 and a 1,000-step warm-up phase. The pre-training was performed on 8 NVIDIA H800 GPUs, with a batch size of 4 per GPU.
• MuPT utilizes Synchronized Multi-Track ABC notation (SMT-ABC) as data representation. SMT-ABC is equivalent to interleaved ABC notation, as both merge multitrack voices into a single sequence. Byte Pair Encoding (BPE) is used for tokenization. MuPT-v1-8192-550M, the chosen baseline model, consists of a 16-layer Transformer decoder with a hidden size of 1024 and a context length of 8192, totaling 505M parameters. MuPT was pre-trained on a corpus of 33.6B tokens.
• MET encodes MIDI events into token sequences and uses hierarchical Transformer decoders for generation, including a event-level decoder and a token-level decoder. Details on the encoding and model architecture are provided in the supplementary materials. MET consists of a 12-layer event-level decoder and a 3-layer token-level decoder, with a context length of 4096 and a hidden size of 1024, totaling 234M parameters. It was pre-trained on three MIDI datasets: Los Angeles MIDI Dataset [Lev, 2024], Monster MIDI Dataset 3 , and SymphonyNet Dataset [Liu et al., 2022].

We applied fine-tuning and reinforcement learning to these models using their pre-trained weights.

Fine-tuning. The fine-tuning dataset for NotaGen and MuPT comprises 8,948 classical music pieces, referred to as the sheet ground truth set (sheet-GT). All data were formatted to match the pre-training data of different models, each preceded by a "period-composer-instrumentation" prompt. Due to the challenges in converting between MIDI and ABC notation, only the keyboard subset, consisting of 3,104 pieces was used for fine-tuning MET, referred to as the MIDI ground truth set (MIDI-GT). Each piece was preceded by a "periodcomposer" prompt.

Reinforcement learning. Considering that the accuracy of prompt semantic feature ¯ zp in CLaMP-DPO relies on a sufficient amount of ground truth data in Yp, we defined the prompt set P to only include prompts p that appear more than ten times in the fine-tuning dataset (Yp > 10). The detailed list of P can be referred in supplementary materials. For NotaGen and MuPT, P contained 112 prompts, covering 86.4% of the data; for MET, P contained 29 prompts, covering 90.5%. The number of iterations K was set to 3, with approximately 100 pieces generated per prompt as Xp in each iteration. The chosen and rejected sets were constructed based on CLaMP 2 Scores. Sheets where staves for the same instrument were not grouped together were excluded from the chosen set. The hyperparameters β = 0.1 and λ = 10 were used, with N = 10, 000 optimization steps. The learning rate was fixed at 1e-6 for NotaGen and MET, and 1e-7 for MuPT, yielding stable CLaMP-DPO performance.

3 https://huggingface.co/datasets/projectlosangeles/ Monster-MIDI-Dataset

Given the challenge of establishing objective metrics that fully capture musicality, we conducted subjective A/B tests in both experiments to evaluate different models and settings. For each question, two pieces were generated using identical prompts; videos were rendered from sheet music using Sibelius and MIDI files using MIDIVisualizer 4 . Participants were instructed to evaluate musicality from multiple perspectives and select the piece they found more musically appealing, or indicate no preference if they perceived no differences. The evaluation criteria included melodic appeal, harmonic fluency, orchestral balance, counterpoint correctness, and structural coherence, and, for sheet music, notation formatting quality. A total of 92 participants from music colleges took part in the assessment. At least 35 valid responses were recorded for each test group, ensuring statistical reliability.

4.2 Ablation Studies on CLaMP-DPO

This experiment evaluates the impact of the proposed CLaMP-DPO algorithm in enhancing the controllability and musicality of generated music for NotaGen, MuPT, and MET. In the objective assessment, we selected several metrics for both the fine-tuned models (denoted as K = 0) and the models after K iterations of CLaMP-DPO optimization. We also assessed a subset of these metrics on the fine-tuning datasets (sheet-GT and MIDI-GT) for reference. The metrics are as follows:

• Average CLaMP 2 Score (ACS): The average CLaMP 2 Score across generated outputs. For sheet-GT and MIDI-GT, the score is computed over the corpus.
• Label Accuracy (LA): The alignment with specified period (per.) and instrumentation (inst.) prompts. We extracted features from the fine-tuning dataset via a multimodal symbolic music encoder-M3[Wu et al., 2024b], then trained two linear classifiers to predict the period and instrumentation labels. LA is defined as the classification accuracy, where for the fine-tuning dataset, it measures the accuracy on the test set, and for generated outputs, it reflects the match between predicted labels and prompt labels.
• Bar Alignment Error (BAE): The proportion of bars where duration is misaligned, occurring in either the generated output or the fine-tuning corpus. This metric applies only to sheet data.
• Perplexity (PPL): A language model metric, where lower PPL indicates better prediction capability.

We conducted subjective A/B tests on each model before and after three optimization iterations with CLaMP-DPO to appraise its efficacy in enhancing the musical quality of generated outputs. The results of the objective and subjective tests are presented in Table 2 and Figure 3, respectively.

4 https://github.com/kosua20/MIDIVisualizer

The ACS, as the primary optimization goal, exhibited a monotonic increase across iterations of its DPO-based process. Though significant improvements were observed in early iterations, subsequent gains exhibited diminishing returns.

LA for period and instrumentation classification exceeded 90% on the test set of fine-tuning data, validating the reliability of the label assignments and the performance of the classifiers. Following the CLaMP-DPO method, all models demonstrated a noticeable improvement in LA, indicating enhanced prompt controllability and better alignment with the intended musical styles. NotaGen exhibited the highest controllability among the models, further confirming its superior adaptability to specified prompts.

Regarding BAE, NotaGen maintained a relatively low error rate throughout optimization, indicating its character-level prediction is more robust at managing duration consistency. In contrast, MuPT's increased error rate is likely due to the use of BPE tokenization, which may merge duration with other musical elements, such as pitch, into single tokens. It may lead to inaccuracies in duration prediction after CLaMPDPO adjusts token probabilities.

Subjective A/B tests showed that all models exhibited improvement in musicality after applying the CLaMP-DPO algorithm, with post-optimization outputs receiving more votes than their pre-optimization counterparts. However, it is noteworthy that PPL increased after optimization. It suggests that PPL may not be a suitable indicator for model performance in symbolic music generation, highlighting the limitations of traditional language model metrics in assessing musical quality.

In summary, the CLaMP-DPO algorithm efficiently enhanced both the controllability and the musicality across three models, irrespective of their data modalities, encoding schemes, or model architectures. This underscores CLaMPDPO's broad applicability and potential for auto-regressively trained symbolic music generation models.

4.3 Comparative Evaluations

This experiment compares the musicality of three models after the LLM training paradigm. For baseline comparison, we constructed the reference set using human-authored musical pieces from the fine-tuning dataset, which represent professional compositional standards. The subjective A/B tests were organized into three groups, each containing the generated results of a model and the ground truth. For comparison involving MET, all data were converted to MIDI to eliminate format-based bias. The results are shown in Figure 4.

Human compositions consistently outperformed all modelgenerated outputs in voting due to their exceptional musicality. Nevertheless, NotaGen achieved the highest voting rate against the ground truth among the three models, suggesting its superior perceived musicality relative to other systems in human evaluations.

Overall, NotaGen outperformed the baseline models. The superior performance of NotaGen compared to MuPT is attributed to well-designed data representation and tokenization. Despite its architectural similarities to MET, NotaGen achieved better musicality, benefiting from the efficiency and structural integrity of sheet music representation compared to MIDI.

5 Limitations and Challenges

While NotaGen shows promising advancements in symbolic music generation, limitations and challenges still warrant discussion.

We once introduced a post-training stage between pretraining and fine-tuning, refining the model with classicalstyle subset of the pre-training dataset. While it accelerated the fine-tuning convergence and improved ACS for NotaGen, the impact was less pronounced on MuPT and MET.

Furthermore, the prerequisite for evaluating generated results using CLaMP 2 Score is that the model has been well trained and is capable of generating reasonable compositions.

For corrupted or syntactically flawed pieces, the CLaMP 2 Score may not reliably indicate the true musical similarity.

Finally, we found that modeling orchestral music presents greater challenges compared to smaller ensembles (e.g. solo piano or string quartets). While rest-bar omission during data pre-processing addresses the degeneration due to excessive blank bars in ensemble writing, NotaGen's performance in orchestral music generation still lags behind. More effective methods are expected for generating large ensemble compositions.

6 Conclusions

In this work, we present NotaGen, a symbolic music generation model designed to advance the musicality of generated outputs through a comprehensive LLM-inspired training paradigm. By integrating pre-training, fine-tuning, and reinforcement learning with the proposed CLaMP-DPO algorithm, NotaGen demonstrates superior performance in generating compositions that align with both the music style specified by prompts and human-perceived musicality. Our experiments validate two key findings: (1) CLaMP-DPO efficaciously enhances controllability and musicality across diverse symbolic music models, regardless of their modality, architectures, or encoding schemes, without requiring human annotations or predefined rewards; (2) NotaGen outperforms baseline models in subjective evaluations, achieving the highest voting rate against human-composed ground truth.

NotaGen establishes the viability of adapting LLM training paradigms to symbolic music generation, while addressing domain-specific challenges, including data scarcity and demand for high-quality music outputs. Future work could extend this framework with CLaMP-DPO to broader musical genres such as jazz, pop, and ethnic music; as well as exploring its compatibility with emerging music generation models.

Acknowledgments

We would like to express our sincere gratitude to SkyTNT for his valuable support on this project. We also acknowledge Yuling Yang, Xinran Zhang, Jiafeng Liu, Yuqing Cheng, and Yuhao Ding from Central Conservatory of Music for their support, especially on subjective tests and paper writing.

This work was supported by the following funding sources: Special Program of National Natural Science Foundation of China (Grant No. T2341003), Advanced Discipline Construction Project of Beijing Universities, Major Program of National Social Science Fund of China (Grant No. 21ZD19), and the National Culture and Tourism Technological Innovation Engineering Project (Research and Application of 3D Music).

Contribution Statement

Yashan Wang, Shangda Wu, and Jianhuai Hu are co-first authors with equal contribution. Maosong Sun is the corresponding author.