METEOR: Melody-aware Texture-controllable Symbolic Music Re-Orchestration via Transformer VAE

METEOR: Melody-aware Texture-controllable Symbolic Music Re-Orchestration via Transformer VAE

Dinh-Viet-Toan Le, Yi-Hsuan Yang

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence
AI, Arts & Creativity. Pages 10126-10134. https://doi.org/10.24963/ijcai.2025/1125

Re-orchestration is the process of adapting a music piece for a different set of instruments. By altering the original instrumentation, the orchestrator often modifies the musical texture while preserving a recognizable melodic line and ensures that each part is playable within the technical and expressive capabilities of the chosen instruments. In this work, we propose METEOR, a model for generating Melody-aware Texture-controllable re-Orchestration with a Transformer-based variational auto-encoder (VAE). This model performs symbolic instrumental and textural music style transfers with a focus on melodic fidelity and controllability. We allow bar- and track-level controllability of the accompaniment with various textural attributes while keeping a homophonic texture. With both subjective and objective evaluations, we show that our model outperforms style transfer models on a re-orchestration task in terms of generation quality and controllability. Moreover, it can be adapted for a lead sheet orchestration task as a zero-shot learning model, achieving performance comparable to a model specifically trained for this task.
Keywords:
Application domains: Music and sound
Methods and resources: AI systems for collaboration and co-creation
Methods and resources: Machine learning, deep learning, neural models, reinforcement learning