M^2LLM: Multi-view Molecular Representation Learning with Large Language Models

1 Introduction

Molecular property prediction is vital in cheminformatics [Yang et al., 2019] and drug discovery [Drews, 2000], enabling the estimation of key characteristics like blood-brain barrier permeability, solubility, and toxicity. Traditional approaches rely heavily on predefined molecular descriptors, such as Extended-Connectivity Fingerprints (ECFPs) [Rogers and Hahn, 2010], derived from SMILES(Simplified Molecular Input Line Entry System) [Weininger, 1988]. While these methods are efficient, their fixed feature sets limit their ability to capture the complex relationships needed for specific chemical tasks. Graph Neural Networks (GNNs) have demonstrated strong capabilities across a wide range of domains [Bu et al., 2024; Yu et al., 2024; Wang et al., 2024a; Wang et al., 2024b] and have also shown effectiveness in capturing structural and physicochemical patterns from molecular graphs [Koh et al., 2024; Yu et al., 2025; Du et al., 2024]. However, they rely heavily on dataset-driven learning, which limits their ability to generalize across diverse chemical tasks. Similarly, recent studies [Sadeghi et al., 2024; Shirasuna et al., 2024] leverage text encoder [Medsker et al., 2001] or Transformer-based language model [Kenton and Toutanova, 2019] using SMILES representations, which focus solely on string patterns and lack the ability to capture contextual relationships or molecular semantics.

† Corresponding author.

To address these limitations, large language models (LLMs) offer a transformative approach to molecular representation by leveraging semantic and contextual knowledge from diverse pretraining corpora, including scientific literature and domain-specific datasets [Zheng et al., 2024c]. In addition, LLMs demonstrate emergent abilities such as contextual reasoning, relational understanding, and the ability to extrapolate patterns, making them uniquely suited for tasks that require deep semantic understanding [Kojima et al., 2022; Zheng et al., 2025]. These capabilities far surpass those of earlier language models or text encoders. While those methods demonstrate the utility of LLMs in capturing molecular representations, they are inherently constrained by their exclusive reliance on SMILES as input. As a result, the potential of LLMs to enhance molecular representation by leveraging their pretrained scientific knowledge and emergent abilities remains underexplored.

In this work, we propose M 2LLM, a novel multi-view molecular representation learning framework that addresses these gaps by fully exploiting the power of LLMs. The framework is organized into two key modules: Molecular Embedding Generation and Molecular Feature Curation, each contributing distinct yet complementary perspectives on molecular data. The molecular embedding generation module leverages the semantic embedding ability of LLMs to represent molecular information frommultiple views. This module includes the molecular structure view, which encodes structural information from SMILES sequences to capture structurespecific insights. This structure view can be further extended to incorporate additional insights, broadening the scope and enriching the molecular representation. Additionally, the molecular task view contextualizes molecules within specific prediction tasks to provide task-relevant guidance. Together, these views harness the LLM's pretrained semantic knowledge to generate comprehensive embeddings.

The molecular feature curation module, on the other hand, utilizes the reasoning ability of LLMs to derive interpretable features. This module introduces the molecular rules view, which generates rule-based features informed by scientific knowledge and observed data patterns, facilitating a deeper understanding of molecular properties. To unify these representations, M 2LLM employs a dynamic fusion mechanism that adaptively combines the contributions of each view based on the requirements of the task and the characteristics of individual molecules. The fused representation is then used for downstream prediction tasks, with a multi-layer perceptron (MLP) designed for both classification and regression. Through extensive experiments, we demonstrate that M 2LLM achieves state-of-the-art performance across multiple molecular property prediction benchmarks, highlighting the potential of LLMs to redefine molecular representation learning.

Our contributions of this work are as follows: (1) We propose M 2LLM, a novel multi-view molecular representation learning framework that integrates diverse molecular perspectives through molecular structure view, molecular task view, and molecular rules view. These views are fused dynamically to create a unified representation tailored to each prediction task. (2) We explore the potential of representations derived from LLMs using M 2LLM for molecular property prediction, demonstrating that LLMs can achieve high performance by leveraging their dual capabilities: molecular embedding generation through their encoding abilities and molecular feature curation through their reasoning capabilities, addressing a significant gap in current research. (3) We show that M 2LLM achieves state-of-the-art performance across multiple molecular property prediction benchmarks, demonstrating its adaptability, scalability, and effectiveness in advancing the use of LLMs for molecular representation learning.

2 RelatedWork

Several molecular representation methods have been developed, including molecular graph, ECFPs, and string line annotations such as SMILES. With the advancement of AI, machine learning models are now extensively used for property prediction through traditional and deep learning approaches.

In the conventional approach, traditional machine learning models, such as random forests [Breiman, 2001], rely on computed molecular fingerprints to predict properties by capturing relationships between molecular substructures [Jeon and Kim, 2019], though these predefined fingerprints may not fully capture complex molecular structural patterns and interactions. On the other hand, GNNs have been widely applied across domains [Zheng et al., 2024a; Wu et al., 2024; Zhang et al., 2025; Zhang et al., 2019], they have also been effectively used to model molecular graphs [You et al., 2020; Wang et al., 2022; Xia et al., 2022], capturing hierarchical structural information and uncovering complex molecular patterns. However, these approaches often fail to integrate broader contextual, and may overlook knowledge already discovered in scientific literature and encoded within LLMs.

LLMs like GPT-4 [OpenAI et al., 2023] and Galactica [Taylor et al., 2022], trained on diverse scientific and chemical datasets, capture semantic and contextual relationships beyond traditional text encoders' syntactic patterns. Moreover, LLMs exhibit emergent abilities such as reasoning [Kojima et al., 2022], relational understanding [Mirza et al., 2024], and pattern recognition [Zheng et al., 2025], making them powerful tools for extracting insights from molecular text representations. Mirza et al. [2024] indicate that even a 7B-parameter LLM can achieve average human scores, while advanced models like GPT-4 can surpass the highest human scores in chemical reasoning.

Previous studies [Wang et al., 2019; Fabian et al., 2020; Ross et al., 2022] have explored encoding SMILES using LLMs as molecular embeddings, demonstrating their effectiveness in capturing meaningful representations and performance in downstream property prediction tasks [Sadeghi et al., 2024]. Researchers [Luo et al., 2024; Zheng et al., 2024b; Rollins et al., 2024] have also investigated combining text encoders with GNNs to leverage both contextual and structural information. However, these methods remain limited by their reliance on SMILES as the sole input, failing to fully exploit the semantic depth of LLMs.

3 Multi-viewMolecular Representation Learning with Large Language Models

In this section, we introduce M 2LLM, a novel multi-view framework designed to leverage LLMs for molecular representation learning. As illustrated in Figure 2, the framework consists of two main components: Embedding Generation and Feature Curation, which collectively provide three distinct views of molecular information. These views are fused in a Multi-View Fusion Module, generating a unified representation optimized for downstream prediction tasks.

3.1 Molecular Embedding Generation

The Molecular Embedding Generation component leverages LLMs to encode sequence inputs, providing two views: the Molecular Structure View and the Molecular Task View.

Molecular Structure View

The molecular structure view uses LLMs to generate embeddings that capture key physical and chemical properties of molecules. LLMs have demonstrated strong capabilities in encoding semantic and contextual knowledge. These models generate outputs sequentially, relying on previous input tokens to reason about the current context. Instead of directly providing SMILES as input, we frame specific questions about the molecule alongside its SMILES representation. This approach gives the LLM additional contextual information, allowing it to "think about" the molecule in relation to the queried property, resulting in richer and more meaningful representations. To achieve this, we define three example insights as questions targeting various structural aspects of the molecule, which can be adapted or replaced as needed.

Structure Insight 1: How does the molecule's 3D shape change in different environments, and what are the effects of these changes?

Structure Insight 2: What are the key intermolecular forces that govern the behavior of this molecule in various contexts?

Sructure Insight 3: How does the molecule contribute to the overall chemical equilibrium in its different environments?

Let si denote the SMILES for molecule i, and qj denote one of the three structure insight questions. The structural embedding zstruct ij for molecule i with question qj is computed as:

where fEncode(·) is the LLM encoder used to process the input, [qj ; xi] represents the concatenation of the question qj and the SMILES si. The embeddings generated for the three questions are then concatenated to form the final structure view representation:

Molecular structure view not only enriches the molecular structure representation but also provides a flexible framework for generating diverse and comprehensive embeddings. By incorporating structural views, M 2LLM focuses solely on analyzing molecular information without being tied to specific tasks or datasets. This design allows the LLM to encode the input in a way that leverages the semantic knowledge learned during the pretraining process, capturing more contextual and meaningful information than using SMILES alone. Additionally, the modular nature of the structural view facilitates straightforward extensions by introducing new questions to create additional views.

Molecular Task View

The molecular task view encodes task-specific information by framing molecular analysis as a natural language processing problem, inspired by the pre-training process of the Galactica model [Taylor et al., 2022] that has demonstrated the ability to process and reason about molecular representations effectively when supplemented with contextual information, such as questions or prompts. To leverage this capability, the task view combines a molecule's SMILES representation with a task-specific question, guiding the LLM to generate a taskaware representation by retrieving and processing semantic information encoded during pretraining. For example, consider the task of predicting blood-brain barrier penetration for a given molecule. The input to the LLM would take the following form:

Here is a SMILES formula: [START I SMILES]C1= CC=C(C=C1)C(=O)O[END I SMILES]

Question: Will the chemical compound penetrate the blood-brain barrier?

Unlike the molecular structure view, which focuses on general analysis of molecular properties, the task view explicitly tailors its input to the prediction problem at hand. The generated embedding ztask i incorporates both the SMILES sequence si and the task-specific question t, and is computed as follows:

3.2 Molecular Feature Curation

The Molecular Feature Curation component introduces an additional view of molecular representation through the Molecular Rules View. This view captures domain-specific knowledge and patterns by leveraging LLMs to generate rules based on pretrained knowledge and data-driven insights. These rules are then transformed into features using a Rule-toFeature Code Translator, providing a numerical representation that complements the semantic representations generated by Molecular Embedding Generation module.

Molecular Rules View

The Molecular Rules View captures both pretrained knowledge from scientific literature and patterns derived from molecular datasets. The generated rules are transformed into numerical features, creating a representation that complements the semantic representations.

Scientific Rule Generation with LLMs: LLM has builtin knowledge and an understanding of various tasks from its pre-training. To make use of this, we assign the LLM a specific persona, such as an experienced chemist, and instruct it to generate rules based on its extensive exposure to scientific literature. These rules are generated independently of any specific molecule and are instead tailored to the requirements of a given task. By leveraging its pretraining on vast scientific datasets, the LLM produces rules that reflect well-established principles and patterns relevant to the task at hand. Let t represent the specific task, the generated rules, Rsci, for task t can be expressed as Rsci(t) = f sci Reason (t), where f sci Reason (·) leverages the LLM's pretrained reasoning capabilities to derive rules. The example below demonstrates how the LLM generates scientifically grounded rules for predicting bloodbrain barrier penetration in Scientific Rule Generation phase.

Persona: Assume you are an experienced Chemist. Please come up with 20 rules that are important to predict if a molecule can penetrate the blood-brain barrier.

LLM Answer Example:

Rule 1: Molecular weight < 500 Da

Rule 2: LogP value between 1 and 3

Rule 3: Presence of aromatic rings

...

Data Pattern Rule Observation with LLMs: Beyond their extensive knowledge base, LLMs exhibit strong abilities in identifying patterns and relationships [Zheng et al., 2025], making them well-suited for recognizing task-relevant trends within molecular data. In this phase, several randomly selected subsets of SMILES strings in the training data {si}mi=1 ∈ Strain with their corresponding label yi are then provided to the LLM. By analyzing these subsets, additional rules Rdata based on observed patterns and relationships within the molecular structures for the specific task t: Rdata(t) = ⋃Kk=1 f data Reason ({si, yi}mi=1 , t)k, where K is the number of subsets analyzed, m is the number of molecule in each subset, and f data Reason (·) leverages the LLM's emergent reasoning capabilities to identify patterns and generate rules. The following example illustrates how the LLM generates task-specific rules by analyzing randomly selected one subset of molecular training data paired with their corresponding labels.

Persona: Assume you are a very experienced Chemist. In the following data, with label 1, it means the smiles string is BBBP. With label 0, it means the smiles string is not BBBP. Please infer step-by-step to come up with 3 rules that directly relate the properties/structures of a molecule to predict if it can be BBBP.

[SMILES strings along with corresponding labels]

LLM Answer Example:

Rule 1: The presence of a benzene ring in the molecule is essential for predicting whether it can be BBBP.

Rule 2: The presence of a carbonyl group (-C=O) in the molecule is also important for predicting whether it can be BBBP.

Rule 3: Number of Hydrogen Bond Donors (HBD)

Feature-based Representation

The combined set of rules, R(t) = Rsci(t) ⋃ Rdata(t), is transformed into numerical features through a Ruleto-Feature Code Translator, which maps the rules,

{r1, r2, . . . , rn} ∈ R(t), into a feature vector for each SMILES string s. This translation process leverages LLMs to transform text-based rules into code-based features and can be defined as:

or alternatively as a numerical function fi(s) ∈ Rn if the rule outputs a continuous value (e.g., molecular weight). The LLM is tasked with translating these rules into executable functions fi(·), which are applied to the input s to extract a set of molecular features. This process generates a feature vector zrule i for a molecule s, where each feature corresponds to a rule:

3.3 Multi-viewRepresentation Fusion

The M 2LLM framework then integrates three views into a unified representation through a fusion mechanism. The fused representation is subsequently passed through a prediction module, which adapts to the requirements of either classification or regression tasks.

For a given molecule si, let zstruct i , ztask i , and zrule i denote the representations obtained from the structure view, task view, and rules view, respectively. To combine these representations into a single, comprehensive vector, our proposed framework employs a weighted sum mechanism. Each view's contribution is modulated by a set of learnable weights αstruct i , αtask i , and αrule i , which are specific to each molecule. The fused representation zfused i is then computed as a weighted sum of the individual view representations:

where weights satisfy: αstruct i + αtask i + αrule i = 1, αstruct i , αtask i , αrule i ≥ 0. The fused representation zfused i is then used as input to a multi-layer perceptron (MLP), which performs the final prediction. This process is mathematically defined as:

where ˆ yi is the predicted output for molecule si, and fMLP(·) represents the function of the multi-layer perceptron.

The framework is trained to optimize the weights αstruct i , αtask i , and αrule i , and the MLP parameters using task-specific loss functions. For classification tasks, the cross-entropy loss is minimized:

For regression tasks, the root mean squared error (RMSE) loss is used:

where N denotes the number of molecules, yi represents the binary ground truth label or the true value, and ˆ yi is the predicted probability or the predicted value.

4 Experiment

4.1 Experimental Setup

Dataset Our framework is evaluated on 8 datasets spanning 34 tasks fromMoleculeNet [Wu et al., 2018], including physiology-related tasks like BBBP [Martins et al., 2012], ClinTox [Gayvert et al., 2016], and 27 SIDER tasks [Kuhn et al., 2016] for adverse drug reaction prediction. Additionally, we evaluate classification tasks from BACE [Subramanian et al., 2016] and HIV [Wu et al., 2018], as well as regression tasks from ESOL [Delaney, 2004], FreeSolv [Mobley and Guthrie, 2014], and Lipophilicity [Wu et al., 2018]. We use the scaffold splitting method recommended by MoleculeNet [Wu et al., 2018], which assigns molecules with distinct structural scaffolds to separate training, validation, and test sets. This method, unlike random splitting, ensures structural dissimilarity between sets, creating a more challenging evaluation scenario. Detailed dataset descriptions are provided in Appendix A.1.

Baselines For the traditional model, we use Random Forest [Breiman, 2001] with ECFP4 [Rogers and Hahn, 2010] as the input feature set. For deep learning models, as shown in Table 1, we select the most representative GNNs pretraining baselines and transformer-based architecture. To ensure a fair comparison, We rerun all baseline models with the same random seed across 10 iterations.

BackboneModel Our framework is built on an LLM-based multi-view architecture, utilizing a state-of-the-art LLM as its backbone. Specifically, for the molecular structural and task views, we use Galactica models (6.7B and 30B parameters) [Taylor et al., 2022], LLaMa-3.1 models (8B and 8B-instruct) [Dubey et al., 2024], and OpenAI's closedsource text embedding models (small and large configurations) [OpenAI, 2024]. For the molecular rules view, rule generation is performed using the Galactica models, leveraging their extensive pretraining on scientific literature to produce high-quality task-specific rules.

4.2 Performance on Classification Tasks

We evaluate M 2LLM on five classification datasets with 31 subtasks, as shown in Table 1. We report the mean and standard deviation from 10 random seeds using the evaluation metric, receiver operating characteristic-area under the curve (ROC-AUC) (%), where higher scores indicate better performance. One result for the same LLM backbone architecture is presented for the comparison with other state-of-the-art baselines. Full results for different backbone architectures are provided in the Appendix A.2.

As shown in Table 1, our framework demonstrates superior performance, surpassing existing baselines with significant improvements. Notably, our framework exhibits exceptional performance on the Clintox dataset, achieving a nearperfect accuracy of 99.5% and 99.4%, this result significantly outperforms all other models. Moreover, on the BBBP, HIV, and SIDER datset, our framework variants achieve the best and second-best results, outperforming all GNN-based and Transformer-based baselines. This result further enhances the credibility of LLM-based approaches in molecular property prediction tasks. Full results on 27 tasks for the SIDER

Dataset can be found in Appendix A.3. In the case of the BACE dataset, M 2LLM achieves 80.0%, which, while competitive, remains below the highest baseline result of 85.0% achieved by the RF model. The potential reasons may be the BACE dataset assigns binary labels for molecular inhibitors of human β-secretase 1 (BACE-1), based on an arbitrary threshold of quantitative potency values (IC50) set at 7 [Wu et al., 2018]. However, potency values can vary significantly depending on the assay settings [Landrum and Riniker, 2024], lower potency values can still indicate strong inhibition of BACE-1 [Harding et al., 2024]. We hypothesize that this arbitrary threshold and label ambiguity hinder LLMs' ability to reason effectively.

4.3 Performance on Regression Tasks

We evaluate M 2LLM on three regression tasks, as shown in Table 2. We report the RMSE for regression, where lower values signify better result. Results presented in Table 2 demonstrate the superior performance of our proposed framework compared to all baselines across three datasets. Specifically, M 2LLM demonstrates strong performance, achieving an RMSE of 2.01 on FreeSolv dataset, reducing the error by 15.5% compared to the best baseline value of 2.38. Furthermore, on ESOL dataset, it achieves an RMSE of 0.44, a remarkable 56.9% reduction in error compared to the best baseline value of 1.02. Additionally, on the Lipophilicity dataset, it achieves state-of-the-art results with an RMSE of 0.66, while our other variants demonstrate competitive performance against baseline models.

4.4 Multi-viewComponent Contribution Analysis

In this section, we analyze the contribution of each view component to the final decision, as illustrated in Figure 3, based on three classification datasets and three regression tasks. Interestingly, the molecular structure view component contributes more significantly to the classification tasks, whereas the molecular rule view component and molecular task view component play a larger role in the regression tasks. This suggests that classification tasks may benefit from a detailed representation of molecular structures, as these tasks often rely on recognizing specific structural features critical for distinguishing between categories. On the other hand, regression tasks, which predict continuous values such as molecular properties, appear to benefit more from the molecular rule and task views, which capture broader context and relationships. This demonstrates that our framework effectively automates the learning of appropriate weights for each component, dynamically optimizing the contribution of each view for individual molecules based on the task requirements.

From the LLM architecture perspective, we find that LLMs with the same architecture tend to exhibit similar component contributions across different datasets to the final predictions. However, on the ESOL and FreeSolv datasets, even though the LLMs heavily rely on one or two components, Galactica6.7B and 30B demonstrate different behavior, despite having the same architectural design, these models exhibit different component contributions for their final decisions. This pattern highlights the flexibility and adaptability of our proposed multi-view representation learning framework.

Furthermore, the results for the ClinTox dataset, as reported in Table 1, demonstrate that we achieve near-perfect scores across all backbone settings. However, this component analysis reveals intriguing insights into how different model architectures rely on various components to make their predictions. The OpenAI models heavily depend on the molecular structure view, indicating that their decision-making process is primarily driven by structural understanding and extensive pre-trained knowledge. In contrast, the Galactica models rely more on task-specific components, likely because our task-specific thinking process is closely aligned with their pre-training dataset and methodology. The LLaMa-3.1 models demonstrate a relatively balanced utilization of all three components to make accurate predictions.

4.5 Effectiveness of Multi-viewRepresentation

To better understand the performance gains afforded by our proposed multi-view representation, we first evaluate a baseline configuration using the SMILES-only representation. Specifically, only the SMILES string of a molecule is fed into the best-performing LLMmodel. This approach relies solely on the LLM's general understanding of a molecule and does not prompt the model to reason through the contextual diversity provided by a multi-view approach.

As shown in Figure 4, our proposed method consistently improved the scores across all six datasets. On the FreeSolv regression dataset, where a lower RMSE indicates better performance, we achieved a substantial reduction in prediction error, decreasing it from 4.29 to 2.01, representing a 53.3% improvement. This improvement underscores the effectiveness of our framework, particularly in molecular property regression tasks. Similarly, for other regression and classification tasks, such as ESOL and BBBP, our method demonstrated a measurable improvement compared to the SMILESonly baseline. This trend is also consistently observed for all other LLM backbone models as illustrated in Appendix A.4.

In the case of the ClinTox dataset, the SMILES-only baseline achieves nearly perfect results, with our framework offering a marginal improvement. This suggests that in tasks where the LLM already possesses sufficient understanding of the molecular domain through SMILES-based encoding alone, the additional views provide less pronounced benefits. This observation, nonetheless, further reinforces the strength of LLMs as text encoders for molecular property prediction.

Overall, these results highlight the key contribution of M 2LLM: the integration of complementary molecular views consistently enhances predictive performance compared to a single-view SMILES-only approach. By dynamically incorporating diverse representations, M 2LLM captures richer molecular features and demonstrates superior adaptability across tasks of varying complexity, firmly establishing itself as a state-of-the-art framework for molecular property prediction.

5 Conclusion

In this paper, we introduce M 2LLM, a multi-view learning framework that harnesses the capabilities of LLMs to generate rich molecular representations, enabling state-of-theart performance in molecular property prediction. By utilizing the strong reasoning capabilities, extensive pre-trained knowledge, and powerful encoding abilities of LLMs, the framework delivers exceptional results across several benchmark tasks. Unlike methods that rely solely on SMILES as input, M 2LLM dynamically integrates multiple views to capture complex molecular features, enabling the learned representations to generalize effectively across diverse classification and regression tasks. These results underscore the transformative potential of M 2LLM in advancing molecular property prediction, offering a scalable and versatile solution for a wide range of applications in molecular science and beyond.

Acknowledgments

This research was partly funded by Australian Research Council (ARC) under grants FT210100097 and DP240101547.