MELLA: Bridging Linguistic Capability and Cultural Groundedness for Low-Resource Language MLLMs
MELLA: Bridging Linguistic Capability and Cultural Groundedness for Low-Resource Language MLLMs
Yufei Gao, Jiaying Fei, Nuo Chen, Ruirui Chen, Guohang Yan, Yunshi Lan, Botian Shi
Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence
Main Track. Pages 1115-1123.
https://doi.org/10.24963/ijcai.2026/125
Multimodal Large Language Models (MLLMs) perform strongly in high-resource languages, yet often produce fluent but culturally "thin" descriptions in low-resource settings. We argue that this failure is not merely a linguistic limitation: culture-specific visual knowledge depends on native visual-textual alignments that translation-centric pipelines rarely provide.
We present MELLA, a multimodal dataset across eight low-resource languages, designed to jointly support linguistic fluency and cultural groundedness. MELLA uses a dual-source strategy that combines native web image-alt-text pairs for culture-grounded supervision with generated-and-translated image descriptions for linguistically rich supervision, explicitly separating two learning signals often conflated in multilingual multimodal data.
Through controlled diagnostic fine-tuning on multiple MLLM backbones, we show that MELLA mitigates cultural hallucination by helping models recognize and articulate culturally specific entities overlooked by translation-based adaptation. Our findings highlight data alignment, rather than model modification alone, as a key path toward culturally grounded multimodal understanding in low-resource languages.
Keywords:
Computer Vision: Vision, language and reasoning
