Improving Scientific Formula Verbalization in Large Speech Language Models for Accessible Learning

1 Introduction

Accessible learning aims to provide fair educational opportunities in which students with different abilities can fully participate, and it has become an important part of online education [Hadi Mogavi et al., 2023; Shi et al., 2024]. Many online learning systems use AI tutors to support students, especially blind or low-vision students who rely on spoken feedback as their primary way of accessing course content [Pal Chowdhury et al., 2024; Anderer et al., 2025]. When courses involve scientific formulas, students depend entirely on how the formulas are read to understand the expression, which highlights the importance of accurate formula verbalization in AI tutors [Gorniak et al., 2024; Abram, 2025; Chen et al., 2025].

∗Corresponding author: Teng Guo.

Previous research on scientific formula verbalization has mainly used cascaded pipelines, which are often adopted in AI tutors. For example, Sieun Hyeon et al. propose a pipeline that integrates automatic speech recognition (ASR) models with language models to correct errors in mathematical expressions and accurately convert spoken expressions to speech output by text-to-speech (TTS) models [Hyeon et al., 2025]. With the development of large language models (LLMs) [Vaswani et al., 2017], recent studies have introduced end-to-end large speech language models (LSLMs) that extend the natural language understanding and representation capabilities of LLMs to the speech modality through multimodal alignment techniques [Zeng et al., 2025; Liu et al., 2026]. Compared to cascaded pipelines, end-to-end LSLMs have attracted increasing attention due to their low latency, capability for streaming generation and potential for industrial deployment. However, when directly applying these LSLMs to AI tutors, their performance in scientific formula verbalization remains suboptimal, with frequent mispronunciations of certain symbols (as shown in Figure 1). Achieving accurate scientific formula verbalization with LSLMs continues to pose an open research challenge.

To improve scientific formula verbalization in end-to-end LSLMs, a promising approach is to directly incorporate high-quality paired data of formula text and corresponding speech during training. Similar to large-scale LLM pretraining, aligning the textual and speech modalities in LSLMs during pretraining also requires massive amounts of data. For example, GLM-4-Voice and Kimi-Audio are trained on over 700,000 hours of speech data [Zeng et al., 2025; Ding et al., 2025]. However, large-scale text-speech paired datasets specifically curated for scientific formulas are difficult to obtain, making formula-oriented large-scale pretraining impractical. This motivates a shift toward the supervised fine-tuning (SFT) stage, where task-specific capabilities can be improved with a smaller amount of high-quality paired data. As LSLMs are developed on top of LLMs, the alignment between general text and speech patterns has largely been established through pretraining, equipping these models with strong capabilities in understanding scientific formulas in textual form, but a small yet critical gap remains in aligning certain scientific symbolic expressions with corresponding speech. To bridge this gap, we construct a high-quality text-speech paired dataset for scientific formulas and design a lightweight and effective method to investigate the following research question: How to align the text and speech modalities in LSLMs for scientific formula verbalization with limited text-speech paired data?

In this paper, we propose Formula-Speech, the first endto-end LSLM designed to enhance scientific formula verbalization by efficiently aligning symbolic text with its corresponding speech. Specifically, we construct two high-quality datasets covering three scientific domains including mathematics, physics and chemistry, in collaboration with educational experts. One dataset consists of paired formula-speech data for aligning symbolic expressions with spoken language, while the other comprises multi-turn teacher-student dialogues in a speech-to-speech format to support the learning of conversational speech features. We further introduce a lightweight and effective two-stage training framework based on low-rank adaptation (LoRA). In the first stage, we perform SFT with LoRA to help the model learn fundamental mapping patterns between scientific formulas and their spoken representations. In the second stage, we employ a reinforcement learning (RL) algorithm, i.e., group relative policy optimization (GRPO), combined with a custom designed LLM-as-a-Judge reward function. This reward function evaluates ASR-transcribed outputs and guides the model toward human-preferred verbalization. Experimental results show that the proposed Formula-Speech consistently outperforms strong baselines across all three domains and achieves stateof-the-art performance in scientific formula verbalization despite being trained on only a small amount of data.

Overall, this paper makes the following contributions:

• We propose the first end-to-end LSLM optimized for scientific formula verbalization. It adopts a lightweight and effective two-stage training framework that requires only a small amount of scientific formula data.
• We construct and release two high-quality datasets covering mathematics, physics and chemistry to facilitate future research in the community.
• We conduct comprehensive experiments across three scientific domains to evaluate the effectiveness of the proposed Formula-Speech, demonstrating state-of-the-art performance on scientific formula verbalization.

2 RelatedWork

2.1 Formula Verbalization

In online learning systems, AI tutors often use spoken explanations to help students access course content, especially those who cannot easily rely on visual materials. This motivates research on scientific formula verbalization, which aims to convert symbolic expressions such as LaTeX formulas into clear and standardized spoken language. Early approaches primarily treated formula verbalization as a speech processing task, leveraging traditional methods such as hidden Markov models and rule based systems [Prylipko et al., 2012; Sak et al., 2013]. In pursuit of better performance and robustness, recent research has shifted toward adopting cascaded pipelines, focusing predominantly on the mathematical domain. Ritchie et al. introduced a unified grammar based on finite-state transducers to jointly support ASR and TTS, aiming for consistent cross-lingual formula rendering [Ritchie et al., 2019]. Dong et al. proposed a dictionaryguided generation model that significantly outperformed rule based baselines in the textual verbalization of mathematical formulas [Dong et al., 2022]. Hyeon et al. developed a cascaded system that combines ASR and symbolic correction with a language model, achieving superior performance in formula transcription [Hyeon et al., 2025]. Roychowdhury et al. conducted a systematic evaluation of TTS models for formula pronunciation, revealing that current systems struggle with complex mathematical structures [Roychowdhury et al., 2025]. Different from previous approaches, we focus on improving scientific formula verbalization in end-to-end LSLMs across the domains of mathematics, physics and chemistry.

2.2 Large Speech Language Models

Current speech interaction in AI tutors mostly adopts cascaded pipelines, typically composed of ASR, LLMs and TTS modules. While this architecture is functional, it suffers from several limitations, including high latency and error accumulation across stages [Fang et al., 2025; Xu et al., 2025; Huang et al., 2025]. To address these challenges, recent research has focused on extending LLMs to the speech modality, giving rise to end-to-end LSLMs. These models aim to unify speech understanding and generation within a single framework, enabling seamless and low latency speech interaction. Zeng et al. introduced an interleaved speech-text training paradigm and a supervised speech tokenizer, achieving strong performance in spoken question answering and conversational ability [Zeng et al., 2025]. Li et al. further advanced the field with a multi-codebook tokenizer and independent speech head, along with a two-stage pretraining strategy to preserve LLM capabilities during speech integration [Li et al., 2025]. Building on these advances, Long et al. tackled the latency bottleneck by proposing a fast interleaved token generation mechanism, achieving zero-delay speech response via efficient token prediction modules [Long et al., 2025]. While these end-to-end LSLMs have demonstrated impressive progress in general conversational tasks, they largely overlook domain-specific challenges such as scientific formula verbalization, which is the primary focus of our work.

3 ProblemDefinition

To support accessible learning with AI tutors in online learning systems, we aim to convert a given scientific formula into spoken language that accurately verbalizes each symbolic component. Formally, we define a scientific formula x ∈ X as a sequence of n tokens, x = {t1, t2, . . . , tn}, where each ti denotes a symbolic token and n is the formula length. Given a paired dataset D = {(x(i), y(i))}Ni=1 of N examples, where x(i) is a formula and y(i) ∈ Y is its corresponding spoken form represented as speech tokens, we train a speech language model G : X → Y that maps input formulas x to generated speech ˆ y = G(x). Unlike natural language, scientific formulas contain compact syntax, hierarchical structure and domain-specific semantics, posing unique challenges for general-purpose speech models.

4 Method

4.1 Dataset Construction

To improve scientific formula verbalization in LSLMs, highquality text-speech paired datasets are essential for aligning symbolic representations with corresponding speech patterns. However, existing resources are limited in both domain coverage and data quality. To bridge this gap, we construct two datasets, namely EduDialogue and SciFormula, following a standardized data construction pipeline with the involvement of educational experts to ensure data quality and procedural consistency [Tirumala et al., 2023].

Formula Collection and Normalization

To ensure broad domain coverage, we collect formulas and multi-turn teacher-student dialogues from an intelligent tutoring system developed by TAL Education Group, a leading educational technology company in China, following standard protocols for data collection and privacy protection. The data covers mathematics, physics and chemistry. All dialogue content is anonymized to remove personally identifiable information while preserving educational relevance. We apply rigorous data cleaning to remove non-scientific, malformed or incomplete formulas. Preprocessing includes whitespace normalization, HTML tag removal, emoji and special character stripping and n-gram-based deduplication to eliminate redundant content. Additionally, all Arabic numerals are converted into textual form to enhance consistency and speech synthesis quality. Valid formulas are then normalized and converted into LaTeX format for downstream processing.

Verbalization and Human Annotation

To obtain accurate speech representations of scientific formulas, we leverage the language understanding capabilities of LLMs to interpret symbolic expressions and generate standardized spoken language. Specifically, we design carefully crafted prompts to instruct Claude Sonnet 4 to transform textual inputs containing formulas into natural and fluent verbalized forms suitable for speech synthesis. For the multi-turn dialogue dataset EduDialogue, every utterance from teachers and students is converted into fluent and complete spoken language to ensure natural continuity in conversation. For the scientific formula dataset SciFormula, each scientific expression in LaTeX format is individually processed to preserve both semantic meaning and structural accuracy. To ensure the quality and correctness of the verbalization, all generated outputs are manually reviewed and refined by educational experts to mitigate potential biases introduced during model generation.

Speech Synthesis

To generate high-quality speech data, we synthesize speech from the spoken language representations obtained in the previous step using CosyVoice2 [Du et al., 2024], a state-ofthe-art TTS model known for its clarity and expressiveness. The model takes the standardized spoken language as input and produces corresponding speech waveforms. This step enables the construction of paired text-speech data with natural and accurate pronunciation, supporting effective training of LSLMs for scientific formula verbalization. All synthesized speech is manually reviewed by educational experts and any samples judged to be unsatisfactory are regenerated.

In total, we construct two datasets for scientific formula verbalization. The EduDialogue dataset contains 3417 multiturn dialogues between teachers and students. These dialogues cover subject-specific instructional scenarios and include both speech and the corresponding spoken transcripts. On average, each dialogue has 15.15 turns, with the longest dialogue reaching 40 turns. The distribution of dialogue lengths is diverse, with the highest proportions concentrated at 2 turns accounting for 19.2 percent, 12 turns accounting for 13.7 percent and 14 turns accounting for 13.3 percent. The SciFormula dataset consists of 8678 paired samples spanning mathematics, physics and chemistry. Among them, 4339 samples are in Chinese and 4339 samples are in English. In terms of subject distribution, chemistry contributes 5434 samples which is 62.6 percent of the total, mathematics contributes 2506 samples which is 28.9 percent and physics contributes 738 samples which is 8.5 percent.

4.2 Model Architecture

We build our Formula-Speech upon the GLM-4-Voice backbone to improve scientific formula verbalization in both speech and text modalities [Zeng et al., 2024]. The architecture is designed to align symbolic formula representations with speech while preserving general conversational capabilities. As illustrated in Figure 2, the model takes either speech or text as input and produces outputs in an interleaved textspeech manner across both modalities.

Speech and Text Encoder

Given an input speech sequence S or a text sequence T, the model first transforms them into discrete representations. A pretrained speech encoder Encsp maps the speech input into a sequence of speech tokens, denoted as tsp, while another text encoder Enctext processes the text input into a sequence of text tokens ttext:

Formula Semantic Adaptor

To enable fine-grained multimodal alignment between symbolic structures and their spoken representations, we introduce a lightweight formula semantic adaptor (FSA) that integrates into the attention layers of a frozen LSLM. The FSA is implemented using low-rank parameter updates [Hu et al., 2022], which specialize the query, key, and value projections for symbol-aware speech modeling while keeping all original parameters θ fixed. We denote the resulting model as LSLM FSA θ .

where ˜tsp and ˜ttext represent the aligned output token sequences generated by the LSLM.

Speech and Text Decoder

The aligned token sequences are then passed to two modalityspecific decoders. The speech decoder Decsp generates the output speech ysp in an autoregressive manner and the text decoder Dectext produces the output text ytext:

where ysp denotes the predicted speech sequence for the verbalized formula and ytext denotes the text segment in the same interleaved output that corresponds to this speech sequence.

4.3 Training Strategy

To enable Formula-Speech to learn accurate verbalization of scientific formulas with only a small amount of data, we adopt a two-stage LoRA training framework. This lightweight strategy allows the model to efficiently acquire formula-to-speech alignment patterns while preserving the general language understanding capabilities of the backbone LSLM. This training framework is inspired by recent advances in RL and adaptation methods for LLMs, such as direct preference optimization (DPO) [Ouyang et al., 2022; Rafailov et al., 2023] and GRPO [Shao et al., 2024]. We introduce several modifications tailored to the formula verbalization task, including custom reward functions and a simplified yet effective training pipeline.

Stage I: Supervised Fine-Tuning with LoRA

In the first stage, we apply SFT with LoRA on our proposed datasets to guide the model in learning basic alignment patterns between scientific symbolic inputs and their corresponding spoken outputs.

Let DSFT = (xi, yi)Ni=1 denote the dataset, where xi is the symbolic formula input and yi is the corresponding spoken output in the form of speech tokens. We optimize the model using the standard maximum likelihood estimation objective:

where LSFT denotes the SFT loss, ω represents the LoRA adapted model parameters, N is the number of training examples, (xi, yi) refers to the i-th symbolic-spoken pair with xi as the symbolic formula input and yi as the corresponding speech token output and Pω(yi | xi) is the conditional likelihood of generating yi given xi under the model.

Stage II: RL with Custom Rewards

To further enhance output quality and reduce errors such as token duplication or overly lengthy generations, we adopt an RL stage based on policy-gradient optimization. We use the GRPO training framework implemented in TRL 1 , which samples multiple candidate outputs, normalizes their reward values within each group and updates the policy using a weighted log-likelihood gradient. This lightweight design improves stability and efficiency for speech generation tasks.

Our RL objective maximizes the expected reward over sampled outputs:

where LRL denotes the RL loss and ˆ y is a sampled speech token sequence drawn from the model distribution Pω(· | x). The scalar reward r(x, ˆ y) evaluates the quality of ˆ y conditioned on x and provides learning signals for improving verbalization quality.

To estimate gradients, we apply a standard policy-gradient update of the form:

where the final update used by GRPO incorporates groupwise reward normalization and clipping for improved training stability.

Although the model generally learns the correct formulato-speech mappings after SFT, it often produces redundant or overly lengthy speech tokens. To address this, we design a composite reward function r(x, ˆ y) as the weighted average of two custom reward components: (1) a reward for nonrepetition and length control that penalizes repeated n-grams in speech token outputs, and (2) a verbalization-quality reward computed via an LLM-as-a-Judge mechanism. The latter transcribes generated speech back to text using ASR and evaluates its correctness relative to the reference interpretation.

To penalize redundant or overly lengthy speech tokens, we define a reward function that computes the n-gram duplication rate in the generated token sequence and applies an exponential penalty when the repetition level exceeds a tolerance threshold.

Let ˆ y = {t1, t2, . . . , tT} denote the generated sequence of speech, where T is the sequence length and ti is the i-th speech token. We compute the n-gram duplication rate d to quantify redundancy in ˆ y. An n-gram is a contiguous subsequence of n tokens from ˆ y, i.e., g = (ti, ti+1, . . . , ti+n-1) for 1 ≤ i ≤ T - n + 1. The duplication rate d is defined as:

1 https://github.com/huggingface/trl.

where G is the set of all n-grams in ˆ y, count(g) is the number of times n-gram g appears in ˆ y and ϵ is a small constant added for numerical stability.

The duplication reward rdup is computed as:

and is clipped to the range [-1, 1] during implementation to ensure numerical stability. d denotes duplication rate, τ is the threshold for duplication rate and α is the penalty scale.

To further enhance verbalization quality, inspired by previous work [Ye et al., 2025; Ma et al., 2025], we adopt an LLMas-a-Judge framework to assess the correctness of generated outputs. Specifically, the generated speech is first converted into waveform and passed through an ASR module to obtain a predicted transcript ˆ t. The transcript is then compared with a reference transcription t∗ using a pretrained LLM to evaluate the fidelity of verbalization with respect to the original formula f .

To improve ASR accuracy, we adopt different ASR models based on language. Paraformer-zh is used for transcribing Chinese utterances [Gao et al., 2022], while Whisper-Largev3 is used for English utterances [Radford et al., 2023]. In addition, we employ Claude Sonnet 4 as the judging model and design task specific prompts to guide its evaluation of verbalization quality.

Let s ∈ [0, smax] be the raw score assigned by the LLM. We normalize this score to a reward value in [-µ, µ] as follows: rqual = 2µ/smax · s-µ, where s is the model-assigned raw score, µ is the reward scale and smax is the maximum score the model can assign.

In total, the final reward is computed as:

where λ ∈ [0, 1] is a balancing coefficient. By optimizing this reward under the GRPO training framework with LoRA updates, the model is guided to generate concise and highquality verbalization.

5 Experiments

5.1 Experimental Setup

Datasets

To effectively train and evaluate our model, we use two datasets including the multi-turn teacher-student dialogue dataset EduDialogue and the scientific formula verbalization dataset SciFormula. For the EduDialogue dataset, we use the entire dataset for SFT. For the SciFormula dataset, we first divide the data by subject into three subsets including Math2k, Physics1k and Chemistry5k. Each subset is then split into an SFT training set, a RL training set and a test set using a 7:2:1 ratio. We ensure that the proportion of Chinese and English examples is approximately 1:1 within each split. All dataset splits are performed randomly.

Baselines

To evaluate the effectiveness of Formula-Speech, we compare it with state-of-the-art end-to-end LSLMs, including GLM-4-Voice [Zeng et al., 2024], Baichuan-Audio [Li et al., 2025], Step-Audio [Huang et al., 2025], Qwen2.5-Omni [Xu et al., 2025], Kimi-Audio [Ding et al., 2025] and VITAAudio [Long et al., 2025].

Evaluation Metrics

To comprehensively evaluate the quality of scientific formula verbalization, following prior work [Gao et al., 2022; Cho et al., 2025], we adopt both automatic and human-centered metrics:

• WER (Word Error Rate): It measures the transcription accuracy of the generated speech via ASR. Lower values indicate clearer pronunciation and fewer recognition errors. When insertion errors dominate and many extra words are inserted, the WER value can exceed 1.
• BLEU (Bilingual Evaluation Understudy): It captures the n-gram overlap between ASR-transcribed outputs and reference transcripts, reflecting lexical similarity.
• NMOS (Normalized Mean Opinion Score): A normalized variant of the standard MOS that automatically estimates speech naturalness from acoustic features, with a maximum score of 5. In this paper, we use NMOS estimator from [Dubey et al., 2023].
• LLMS: It evaluates the correctness and relevance of verbalized formulas using Claude Sonnet 4, which scores each output from 0 to 3 based on the original formula, reference transcript and ASR-transcribed result. Scoring criteria are shown in Table 1. This metric naturally accommodates cases where a single formula admits multiple valid verbalizations.
• HumanS: It measures verbalization accuracy through human annotation using the same rubric as LLMS, with final scores averaged across three raters. Human evaluation further helps mitigate potential biases introduced by LLM based evaluation.
• HumanF : It assesses speech fluency and naturalness based on predefined criteria in Table 1, with ratings averaged from three raters.

Implementation Details

We adopt a two-stage training strategy for the proposed Formula-Speech. In the first stage, we perform SFT using the full EduDialogue dataset and the SFT training set of the SciFormula dataset, training for 20 epochs with a learning rate of 5e-4 under a cosine schedule. The batch size is 1 per device with gradient accumulation of 4. In the second stage, we apply reinforcement learning on the RL training set of SciFormula using the GRPO training framework implemented in TRL. Training runs for 10 epochs with a learning rate of 1e-6, batch size 8 and accumulation steps 8. We generate 8 responses per prompt using top-p sampling (p = 0.95), temperature 0.6 and up to 512 new tokens. LoRA is applied to the query-key-value modules with rank 64, alpha 32 and dropout 0.05. To guide learning, we use two scalar reward components. The repetition suppression reward penalizes frequent

2-grams using a duplication threshold τ = 0.05, an exponential penalty scale α = 6 and a small constant ϵ for numerical stability, with values clipped to [-1, 1]. The formula verbalization reward is scored by Claude Sonnet 4 based on ASR transcriptions and linearly mapped to [-µ, µ] with µ = 1. The final reward is a weighted sum with coefficient λ. All random seeds are fixed to 42 for reproducibility. All experiments are conducted on a cluster with 8 nodes, each equipped with 8 NVIDIA A100 GPUs.

5.2 Overall Performance

Tables 2 - 3 present the overall performance. The best results are highlighted in bold and the second-best results are underlined. The "zh" denotes Chinese and "en" represents English. Based on the tables, we make the following observations: (1) Our proposed Formula-Speech achieves the best performance across nearly all evaluation metrics, demonstrating its strong capability in scientific formula verbalization. For example, on the Math2k dataset in Chinese, our model attains a WER of 0.3421 and a BLEU score of 0.7082 in Table 2, as well as an LLMS of 1.5032 and a HumanS of 1.7140 in Table 3, significantly outperforming the second-best model, Step-Audio, which scores 0.9846, 0.4210, 1.1019, and 1.1190, respectively. These results suggest that Formula-Speech is particularly effective at aligning symbolic formulas with their corresponding speech patterns, even in complex domains such as mathematics. (2) Step-Audio also demonstrates competitive performance, ranking second across all metrics and consistently outperforming the remaining baselines. This can largely be attributed to its substantial parameter count of 130B, which is considerably larger than those of other models such as Baichuan-Audio (7B) and GLM-4-Voice (9B). These results suggest that larger model capacity tends to yield better performance in scientific formula verbalization. However, in real-world applications, deploying extremely large models may result in significant resource consumption and higher inference costs, which limits their practicality.

5.3 Comparisons with Cascaded Pipeline

Previous approaches to formula verbalization have predominantly relied on cascaded pipeline where ASR, LLM and TTS modules are connected sequentially. Although effective, these systems often suffer from high latency and cumulative error propagation across stages. Recent research has increasingly focused on end-to-end LSLMs that provide lower latency, enable streaming generation and offer greater potential for real world deployment. To assess the effectiveness of our end-to-end Formula-Speech, we compare it with a strong S Human en

S

Figure 3: Performance comparisons with cascaded pipeline.

cascaded pipeline using four averaged evaluation metrics including WER, BLEU, LLMS and HumanS. The cascaded pipeline integrates Whisper-Large-v3 for ASR, Qwen2.5-7BInstruct for text generation and CosyVoice2 for TTS synthesis. Experimental results are shown in Figure 3. The LLMen S and LLMzh S denote the metric LLMS in English and Chinese respectively; others follow accordingly. As illustrated in the figure, Formula-Speech consistently outperforms the cascaded pipeline across all metrics. These findings demonstrate that our end-to-end approach not only enhances verbalization performance but also provides a practical alternative to traditional cascaded pipelines for scientific formula verbalization.

5.4 Speech Quality Evaluation

To assess the overall quality of speech generated by LSLMs, we report the average NMOS scores and collect human ratings on fluency. The evaluation results are presented in Figure 4. From the figure, we observe that most LSLMs exhibit similar speech quality. Among all models, Qwen2.5Omni achieves the highest average scores on both NMOS and HumanF across Chinese and English tasks. This performance is attributed to the Talker module in its architecture, which enhances the naturalness and smoothness of speech by effectively cloning speaker characteristics. In comparison, Formula-Speech and GLM-4-Voice yield similar results. This outcome is expected since Formula-Speech is built upon GLM-4-Voice without additional optimization for speech fluency or naturalness. Improving speech quality through targeted fluency and voice naturalness enhancement will be explored in future work.

5.5 Impact of Different Training Strategies

To effectively align LSLMs with limited text-speech paired data for scientific formula verbalization, we adopt a two-stage LoRA based training strategy in our Formula-Speech. To verify the effectiveness of this approach, we conduct an ablation study comparing five different training strategies. Full-SFT refers to SFT with all model parameters updated. LoRASFT performs SFT using LoRA while keeping the base model frozen. LoRA-GRPO applies RL using the GRPO algorithm with LoRA without prior fine-tuning. Full-SFT+GRPO combines full parameter fine-tuning followed by RL. LoRASFT+GRPO represents our complete two-stage training strategy. As shown in Table 4, LoRA-SFT achieves the lowest WER by reducing symbol and structure verbalization errors while preserving the base model's speech generation ability. LoRA-SFT+GRPO further improves BLEU and LLMS, with only a small WER increase caused by slight lexical variations introduced during RL optimization. These results suggest that the two-stage strategy balances transcription-level accuracy with higher-level semantic and pedagogical alignment.

6 Conclusion

In this paper, we focus on scientific formula verbalization for AI tutors, which is essential for accessible learning in online learning systems. Built upon two carefully curated datasets, the proposed Formula-Speech is optimized through a two-stage training strategy combining SFT and RL. Experimental results across multiple scientific domains demonstrate that our model significantly improves the verbalization performance of LSLMs. These findings highlight that end-toend speech models can be efficiently adapted to handle symbolic content, offering a practical path toward more accessible spoken interaction in educational settings.

7 Stakeholder Involvement

This work was shaped by a real need from online AI tutor development in collaboration with TAL Education Group, a leading educational technology company in China. Existing end-to-end LSLMs often fail to reliably verbalize scientific formulas, limiting their use in AI tutors for accessible learning. Educational experts helped translate this practical need into a research task by selecting educationally meaningful formula-reading cases, refining verbalizations for semantic correctness and pedagogical appropriateness, and shaping the human evaluation criteria for accuracy and fluency.

Acknowledgments

This work was supported in part by Beijing Natural Science Foundation under Grant No. L252010; in part by NSFC under Grant No. 62477025; in part by Key Laboratory of Smart Education of Guangdong Higher Education Institutes, Jinan University (2022LSYS003); in part by the Guangdong University Student Science and Technology Innovation Cultivation Special Fund under Grant No. pdjh2026ak027; in part by Guangdong Hong Kong Joint Laboratory for Data Security and Privacy Protection under Grant No. 2023B1212120007 and in part by the Fundamental Research Funds for the Central Universities.