Graph-based Molecular Representation Learning

1 Introduction

The interaction between machine learning and chemical science has received great attention from researchers in both areas. Remarkable progress has been made by applying machine learning in various chemical applications including molecular property prediction [Guo et al., 2020; Sun et al., 2021; Yang et al., 2021b; Liu et al., 2022c], reaction prediction [Jin et al., 2017; Do et al., 2019], molecular graph generation [Jin et al., 2018a; Jin et al., 2020b] and drugdrug interaction prediction [Lin et al., 2020]. Molecular representation learning (MRL) is an important step in bridging the gap between these two fields. MRL aims to utilize deep learning models to encode the input molecules as numerical vectors, which preserve relevant information about the molecules and serve as feature vectors for downstream chemical applications. While general representation learning models were earlier adapted to represent molecules, MRL algorithms have been recently designed to better incorporate chemical domain knowledge. However, it is non-trivial to have a seamless integration of domain knowledge into representation learning models. Given the tremendous effort in this rapidly-developing area, we are motivated to provide a systematic review of recent MRL methods, which are based on graph deep learning methods and integrate various types of chemical domain knowledge.

We focus on graph-based MRL for two reasons. First, molecules naturally lead themselves to graph representations, as they are essentially atoms and bonds interconnecting atoms. Compared with SMILES, a line-based representation (i.e., string) of molecules, molecular graphs provide richer information for MRL models to learn from. Accordingly, graph-based MRL models evolve much faster than sequence-based MRL models. Second, graph neural networks (GNN) have shown exceptional capacity and promising performance in handling graph structural data [Kipf and Welling, 2017; Hamilton et al., 2017; Zhang et al., 2019; Guo et al., 2023], certainly including those applied on molecular graphs [Gilmer et al., 2017; Hu et al., 2020; You et al., 2020]. It is thus urgent to summarize the effort of leveraging GNN on molecular graphs with domain knowledge and open this topic with more discussion.

This survey paper will be a contribution to both fields of machine learning and chemistry. A variety of moleculecentered problems can be formulated as predictive or generative tasks, e.g., molecule property prediction, reaction prediction, and molecule generation. The machine learning-enabled solutions for these problems have a common ground in learning high-quality representations for molecules. However, researchers in chemistry are overwhelmed by the big group of MRL models to choose from, not to mention that new models are rapidly presented. This survey provides an up-todate overview of MRL models regarding the input graph to the models and the feasible downstream applications. Researchers in chemistry can easily find out the MRL models that match their application needs. For researchers in machine learning, a lack of understanding of the chemical domain knowledge is the barrier to addressing the representation learning for molecules. Treating molecular graphs as regular attributed graphs would overlook the special substructure patterns of molecules, such as motifs and functional groups. This survey summarizes the existing strategies for introducing chemistry-related domain knowledge into representation learning and will inspire researchers in machine learning to design more effective MRL models.

We organize the review in the following structure. We first introduce the input expression of molecules by 2D and 3D graphs. Following these categories of input, we summarize representative MRL algorithms regarding the usage of domain knowledge, the learning strategies, the application tasks, and code links if available. We then look into each application task and introduce in detail the usable MRL models and benchmark datasets. In the end, we open the discussion for future research directions and conclude.

2 Expression of Molecules

When machine learning was first introduced for molecular analysis, hand-crafted features based on predetermined fingerprint extraction rules were used to identify and represent significant information about molecules [Ahneman et al., 2018]. However, this feature engineering process can be time-consuming, expert-dependent, and may not always provide the best results. To address these challenges, deep learning models, known for representation learning, have been developed to learn important molecular features automatically. Molecule input to deep learning models can be presented in two kinds of expressions: molecular sequences and graphs. The expression of molecular sequence, such as simplified molecular-input line-entry system (SMILES) [Weininger et al., 1989] and SELF-referencing Embedded Strings (SELFIES) [Krenn et al., 2020], may separate two connected atoms at two distant positions and lead to inferior representation. In contrast, the expression by graph naturally incorporates additional information in nodes (atoms) and edges (bonds), which can be easily leveraged by the rich suite of graph-based models (e.g., graph neural networks). Therefore, MLR on molecular graphs is becoming commonly used and will be the focus of this survey.

In this section, we provide clarification on the distinction between 2D molecular graphs and 3D molecular graph representations, as shown in Figure 1. We analyze the characteristics of each representation and explore their applications and limitations when used in deep learning models.

2.1 2DMolecular Graphs

A graph is typically composed of nodes connected by edges. Similarly, in a molecule, atoms can be seen as nodes and bonds as edges interconnecting them. Thus, each molecule has a natural graph structure. This renders molecular graphs to be the most feasible input for deep learning models and leads to their extensive use. The most common form of molecular graphs is described by three matrices: the node feature matrix, edge feature matrix, and adjacency matrix. Molecules are usually stored as SMILES for convenience and then converted to molecular graphs for computation using specific tools, such as RDKit [Landrum, 2020]. The commonly used features of nodes and edges are listed in Table 1, including mandatory features such as atom and bond types, and other optional features can be added as in need of different tasks [Tang et al., 2020; Saebi et al., 2023]. Among these features, the atom's chirality tag cannot be learned from the common 2D molecular graph representation without 3D geometric information, while all other features are learnable from both 2D and 3D structures. Each bond is considered as a bidirectional edge, i.e., a bond between atoms A and B is given as two edges in the adjacency list: one from A to B, and the other from B to A. With the description matrices, 2D molecular graphs can be treated as homogeneous [Gilmer et al., 2017; Guo et al., 2021; Coley et al., 2019] or heterogeneous networks [Shui and Karypis, 2020] for learning molecular representations via leveraging graph neural networks (GNN).

Despite that GNN can be conveniently used on 2D molecular graphs, the learned representation neglects the spatial direction and torsion between atoms in molecules. This is mainly due to the limitation of 2D graph structure, which only presents the type of bond connecting two atoms, and has no information like torsion angle, bond length, and stereoisomerism. The missing information is important to catch the subtle difference that may cause a significant discrepancy in chemical problems. For instance, reactants with the same 2D graph structures can have different products, because the reactants may differ on torsion angle and bond length, which are not reflected in their 2D graphs.

2.2 3DMolecular Graphs

The 3D molecular graphs provide the missing geometric information by explicitly encoding the spatial structure of molecules. The 3D graphs present the atomic structure as a set of atoms along with their 3D coordinates, which includes more spatial information about atoms. As a result, this representation format has received increasing attention in MRL [Liu et al., 2022c]. The key difference between 2D and 3D molecular graphs lies in the determination of edges between atoms. In 2D molecular graphs, edges (bonds) are pre-determined, indicating the existence of a connection or not. The soft edges of atomic interactions in 3D graphs can be determined by the distance and also the angle between two atoms using their coordinates. To incorporate more complicated spatial relationships, spherical graph neural network [Liu et al., 2022c] is designed to learn molecule structure from 3D graphs.

3 Methodologies of MRL

In this section, we summarize MRL methods into three categories based on the types of input molecules: 2D-based, 3Dbased, and knowledge graph-based MRL methods. We introduce the encoding method for each category and point out recent representative methodologies (summarized in Table 2).

3.1 2D-basedMRLMethods

2D molecular graphs are the most widely used inputs for graph-based MRL. Here, we introduce the general graph neural networks to learn molecular representations with 2D graphs. Then, following several chemical substructure definition clarifications, we will chart the path from the general representation learning methods to the representative methods incorporating molecular structures and chemistry-related domain knowledge.

Encoding methods. Formally, each molecule generally is considered as an undirected graph G = (V, E, X) with node features xv ∈ X for v ∈ V and edge features euv ∈ E for (u, v) ∈ E [Brockschmidt, 2020]. Here, nodes represent atoms and edges represent bonds. Generally, graph-based learning methods can fit into Message Passing Neural Networks (MPNN) [Gilmer et al., 2017] scheme. Therefore, we take MPNN as an example to illustrate the learning process. The forward pass consists of three operations: message passing, node update, and readout. During the message passing phase, node features are updated iteratively according to their neighbors in the graph for T times. By initializing the embedding of node v as h0v = xv, node hidden states ht+1 v at step t + 1 are obtained based on messages mt+1 v , which are represented as: mt+1 v = ∑u∈N(v) Mt(htv , htu , euv) and ht+1 v = Ut(htv , mt+1 v ), where Mt denotes the message function, Ut is the node update function, and N(v) is the set of node v's neighbors in the graph. After updating the node features T times, the readout function R computes the whole graph embedding vector by ˆ y = R(hTv | v ∈ V). Note that R is invariant to the order of nodes so that the framework can be invariant to graph isomorphism. ˆ y is the representation for the molecule and can be passed to downstream tasks. All functions Mt, Ut, and R are neural networks with learnable weights to update during the training process.

Besides MPNN, different variants of graph neural networks like GCN [Kipf and Welling, 2017], GIN [Xu et al., 2019], GAT [Veliˇckovi´c et al., 2018], GGNN [Li et al., 2016], GraphSage [Hamilton et al., 2017], HetGNN [Zhang et al., 2019] and feature-wise GNN-Film [Brockschmidt,

2020] can also be used directly to learn molecular representations. These methods are widely utilized as the base encoder for molecular representation learning in various downstream tasks, such as reaction prediction [Coley et al., 2019], property prediction [Brockschmidt, 2020] and drug discovery [Jin et al., 2020c]. Hu et al. [Hu et al., 2020] conduct a comparative study on GNNs in property prediction and find that GIN usually achieves the best results. While these models are powerful in learning graph structures, chemical traits, and other chemical domain knowledge are largely neglected.

Chemical Substructures. Molecular graphs have substructures that are relevant to certain molecular properties or represent molecular generation constraints. These substructures are not just subgraphs, and convey special domain knowledge. They are clarified and distinguished next. Chemical substructures [Jin et al., 2018a] could be clusters of atoms, e.g., rings. Motifs [Rong et al., 2020; Sun et al., 2021] are recurrent sub-graphs among the input graphs. The functional group is an important component of motifs and encodes rich domain knowledge of molecules. This type of motif can be detected by RDKit [Landrum, 2020]. A Rationale [Jin et al., 2020c; Liu et al., 2022a] is a sub-graph that has a particular molecular property. Scaffolds [Maziarz et al., 2022] are predefined chemical sub-graphs. Structures sharing the same scaffold can always be considered to be generated following the same synthetic pathway.

Learning Strategies. The weights in MRL encoder can be trained in an end-to-end fashion by attaching the encoder with a downstream task, e.g., the representation after encoding is sent to make property prediction by a fully connected layer. The training thus takes a supervised manner. To enhance the training process, molecular substructure-related or chemical domain knowledge can be utilized. JT-VAE [Jin et al., 2018a], RationaleRL [Jin et al., 2020c], and HierVAE [Jin et al., 2020b] take advantages of chemical substructures, rationales, and motifs respectively for the molecular generation. Yang et. al [Yang et al., 2021b] proposed PhysChem. This method is composed of a physicist network to learn the molecular conformation and a chemist network to learn the molecular properties. It shows good performance on property prediction benchmarks by fusing both chemical and physics information. Wang et. al [Wang et al., 2021] involved task information and proposed a property-aware embedding method for molecular property prediction.

Although the learned representation may perform well for the specific downstream task, it is not generally usable for other tasks. In addition, supervised training requires a sufficient set of annotated training samples, which are often difficult to acquire. Recent research has seen a foray into selfsupervised learning strategies [Jin et al., 2020a] that propose reconstruction tasks for pre-training. PreGNN [Hu et al., 2020] uses two self-supervised strategies (context prediction and node/edge attribute masking) to pre-train GNN. GROVER [Rong et al., 2020] involves molecular-specific self-supervised methods: contextual property prediction and graph-level motif prediction. Zhang et. al [Zhang et al., 2021] also designed a motif-based graph self-supervised strategy, which predicted the motif's topology and label during the motif tree generation process.

Table 2: Representative graph-based MRL algorithms with open-source code. CL and RT stand for pre-training by two self-supervised learning methods, contrastive learning and reconstruction tasks respectively. PP, MG, RP and DDI stand for property prediction, molecule generation, reaction prediction and drug-drug interaction respectively. "/" indicates not applicable.

Contrastive learning is another common self-supervised learning technique, where augmented graphs are biased to keep close to the anchor graph (positive pair) and away from other graphs (negative pairs). It can help graph encoder models to produce graph representations with better generalizability, transferability, and robustness. The general graph augmentation methods (node dropping, edge perturbation, attribute masking and subgraph) were proposed by You et al. [You et al., 2020], which could be applied to molecular datasets as well. InfoGraph [Sun et al., 2020] trains the model by maximizing the mutual information between the representations of the entire graph and substructures of different granularity. Unlike general contrastive learning strategies, the following models incorporate chemical domain knowledge. MoCL [Sun et al., 2021] has two proposed molecular graph augmentation methods: one is replacing a valid substructure with a similar physical or a chemical property-related substructure. The other one is changing a few general carbon atoms. Wang et. al [Wang et al., 2022a] were inspired by the relation of equivalence between reactants and products in a chemical reaction. They proposed MolR to preserve the equivalence relation in the embedding space. It forces the sum of reactant embeddings and the sum of product embeddings to be equal. The reactant and the product from different reactions could be a negative pair for this contrastive learning.

3.2 3D-basedMRLMethods

Molecular spatial information, especially geometric information, attracts wide attention and has been increasingly involved in the MRL in recent years, especially when the model needs to learn the physical and chemical properties of molecules associated with the 3D positions of atoms.

Encoding methods. 3D molecule is a dynamic structure since atoms are in continual motion in 3D space. The local minima on the potential energy surface are called conformer, or conformation. In nature, a molecule contains multiple lowenergy conformers exhibiting different chemical properties. Therefore, the 3D molecular graph implicitly encodes spatial positions of atoms to learn better representation. In general, the 3D conformer of a molecule can be denoted as G3D = (V, C) where V is the node set, and C is the 3D-coordinate matrix. Various encoding methods are explored to enhance GNNs with a proper 3D-coordinate matrix. GeomCL [Li et al., 2022] and GRAPHMVP [Liu et al., 2022b] utilize conformers generated from RDkit through the stochastic optimization algorithm using the Merck Molecular Force Field (MMFF) to encode spatial information such as angles, interatomic distances, torsion, etc. SphereNet [Liu et al., 2022c]

employs sphere coordinates and designs a spherical message passing as a powerful scheme for 3Dmolecular learning. Follow this work, ComENet [Wang et al., 2022b] is a messagepassing paradigm on 3D molecular graphs with better completeness (bijectivity) by leveraging rotation angles. To circumvent the challenge that true 3D coordinates of molecules are difficult to calculate and sometimes non-deterministic, DimeNet [Klicpera et al., 2019], GemNet [Klicpera et al., 2021a] and Directional MPNN [Klicpera et al., 2021b] generate synthetic coordinates in molecules through computing molecular distance bound and corresponding angles between atom pairs, where directional message passing networks are applied to learn the enhanced representation.

Learning Strategies. In addition to supervised learning, recent MRL studies propose self-supervised learning techniques by using both 2D and 3D molecular graphs. UnifiedPML [Zhu et al., 2022] uses three pre-training reconstruction tasks: reconstruction of masked atom and coordinates, 3D conformation generation based on 2D graph, and 2D graph generation based on 3D conformation. For contrastive learning, molecular 2D and 3D graph representations are naturally two augmented views of molecules. Using this characteristic, GeomGCL [Li et al., 2022], GraphMVP [Liu et al., 2022b], and 3D-Informax [St¨ark et al., 2022] were proposed to train the molecular representations by keeping the consistency between 2D and 3D graph information. GeomGCL utilizes 2D geometric information, while GraphMVP and 3DInformax use 2D topological information. Different from the other two methods, 3D-Informax utilizes multiple 3D conformers instead of one.

3.3 Knowledge Graph-basedMRLMethods

The knowledge graph-based methods are proposed to involve molecular-structure-invariant but rich external knowledge in the model. KCL [Fang et al., 2022] is a molecular augmentation method for contrastive learning with an external knowledge graph, which is formed by triples in the form of (chemical element, relation, attribute), such as (Gas, isStateOf, Cl). Then a new node "Gas" connecting with "Cl" atom will be generated to the original 2D molecular graph. After the augmentation, the model will be trained by maximizing the agreement between two views of molecular graphs with a contrastive loss function. In contrast to KCL and the above MRL methods, which take atoms as nodes and bonds as edges forming a graph, KGNN [Lin et al., 2020] and MDNN [Lyu et al., 2021] explore the knowledge graph consisting of molecules as nodes and connection relationship between molecules as edges. In this case, molecular representations are learned from the knowledge graph structures instead of molecular structures.

4 Application Tasks of MRL

In this section, we discuss four real applications of MRL, and present the details of representative work in Table 3.

4.1 Property Prediction

Molecular property prediction plays a fundamental role in drug discovery to identify potential drug candidates with target properties. Generally, this task consists of two phases: a molecular encoder to generate a fixed-length molecular representation and a predictor. A predictor is utilized to predict whether the molecule has the target property or predict the reaction of molecules to the target property based on learned molecular representation. Property prediction results can reflect the quality of learned molecular representation directly. General graph learning papers [Hu et al., 2020; Gilmer et al., 2017; Brockschmidt, 2020; You et al., 2020] thus take property prediction tasks to evaluate the performance of their algorithms. Compared with these general methods, the pre-training methods (e.g., GROVER, MoCL, and MGSSL) involving molecular substructure-related designs are more appropriate and can mostly achieve better prediction performance. Some recent work proposes supervised learning models specifically for property prediction. GREA [Liu et al., 2022a] defines the complementary subgraph of rationale in each molecular graph as an environmental subgraph of that molecule, which has no relationship with the target property. Yang et al. [Yang et al., 2022] proposed MoleOOD to guide the encoder to focus on the environmentinvariant substructures to improve its generalization ability. Among the above methods, UnifiedPML [Zhu et al., 2022] achieves SOTA performance by leveraging the unified view of both 2D and 3D molecular graphs. Besides, the insufficient available molecular dataset is a common problem existing in chemistry. Guo et al. [Guo et al., 2021] and Wang et al. [Wang et al., 2021] proposed meta-learning methods to deal with this low-data problem on property prediction.

The property prediction task is mainly conducted as classification and regression. The classification task is to predict a discrete class label, for which the Area under the ROC curve (ROC-AUC) and accuracy (ACC) are selected evaluation metrics [Sun et al., 2020; Rong et al., 2020]. The regression task is to predict a continuous quantity for each molecule, where the mean absolute error (MAE) and Root Mean Square Error (RMSE) are commonly used evaluation metrics to provide the estimate of the accuracy for the target [Rong et al., 2020; Zhu et al., 2022].

4.2 Molecular Generation

The key challenge of drug discovery is to find target molecules with the target properties, which heavily relies on domain experts. The molecular generation is to automate this process. Two steps are necessary to complete this task: one is designing an encoder to represent molecules in a continuous manner, which is beneficial to optimize and predict property; the other is proposing a decoder to map the optimized space to a molecular graph with the optimized property. Atom-byatom generations may generate atypical chemical substructures such as partial rings. To avoid invalid states [Jin et al., 2018a], most studies generate graphs fragment by fragment instead of node by node. JT-VAE [Jin et al., 2018a] and VJTNN [Jin et al., 2018b] decompose the molecular graph into the junction tree first, based on substructures in the graph. Then they encode the junction tree using a neural network. Next, they reconstruct the junction tree and assemble nodes in the tree back to the original molecular graph. These methods are highly complex and frequently fail for substructures involving more than 10 atoms. Different from the above substructures, motifs have much larger and more flexible substructures. Therefore, to deal with the problems existing in the previous methods, Jin et al. [Jin et al., 2020b] proposed HierVAE, which relies on motifs instead of substructures and generates molecular graphs hierarchically based on motifs. Following this direction, MoLeR [Maziarz et al., 2022] was proposed to generate molecules by combining atom-by-atom and motif-by-motif generation. What's more, it supports the scaffolds as an initial seed for molecular generation, which outperforms previous methods on scaffold-constrained tasks. Besides substructures and motifs, RationaleRL [Jin et al., 2020c] utilizes Monte Carlo Tree Search to extract the subgraphs from the molecules and construct rationale vocabulary for each property with their predictive models. It composes molecules from the sampled rationales to preserve the properties of interest with a variational auto-encoder. In contrast to the above encoder-decoder framework, GraphAF [Shi et al., 2020] is a flow-based auto-regressive model to generate the molecular graph in a sequential process, which allows it to leverage chemical valency constraints in each generation step and also enjoys efficient parallel computation in the training process. ConfVAE [Xu et al., 2021] and UnifiedPML [Zhu et al., 2022] were proposed for molecular 3D conformation generation tasks.

For molecular generation tasks, validity is the percentage of the chemically valid generated molecules [Jin et al., 2018a]. Diversity measures the diversity of the generated positive compounds by computing their distance in chemical space. Reconstruction accuracy is utilized to evaluate how often the model can reconstruct a given molecule from the latent embedding [Jin et al., 2020b]. Frechet ChemNet Distance (FCD) is utilized to measure how much the sampled molecules ensemble the training molecules [Maziarz et al., 2022]. For property-constraints molecular generation tasks, the model also needs to measure the fraction of generated molecules that match the target property. For molecular conformation generation tasks, coverage score (COV) and matching score (MAT) are usually utilized for evaluation [Xu et al., 2021; Zhu et al., 2022].

4.3 Reaction Prediction

Reaction prediction and retrosynthesis prediction are fundamental problems in organic chemistry. Reaction prediction means using reactants to predict reaction products. The process of retrosynthesis prediction is the opposite of reaction prediction. When taking SMILES as input, the reaction prediction task is analogous to a translation task. When taking molecular graphs as input, there are two steps to do both for reaction prediction and retrosynthesis prediction. Like WLDN [Jin et al., 2017] and WLDN++ [Coley et al., 2019], the model needs to predict the reaction center first and then predict the potential products which is the major product. These tasks will be evaluated by coverage (whether the candidates cover the correct product) and accuracy (whether the model can select the correct product). Different from previous work, MolR [Wang et al., 2022a] formulates the task of reaction prediction as a ranking problem. All the products in the test set are put in the candidate pool. MolR ranks these candidate products based on the embedding learned from given reactant sets, using mean reciprocal rank (MRR) and hits ratio (Hits) as the evaluation metrics.

4.4 Drug-drug Interactions

Detecting drug-drug interaction (DDI) is an important task that can help clinicians make effective decisions and schedule appropriate therapy programs. Accurate DDI can not only help medication recommendations but also effectively identify potential adverse effects, which is critical for patients and society. AttSemiGAE [Ma et al., 2018] predicts DDI by measuring drug similarity with multiple types of drug features. SafeDrug [Yang et al., 2021a] designs global and local two modules to fully encode the connectivity and functionality of drug molecules to predict DDI. MoleRec [Yang et al., 2023] proposes a molecular substructure-aware representation learning strategy for DDI. Both KGNN [Lin et al., 2020] and MDNN [Lyu et al., 2021] build the drug knowledge graph to improve the accuracy of DDI. DDI prediction is evaluated according to Accuracy (ACC), ROC-AUC (AUC), P-AUC (area under the precision-recall-curve), and F1 scores.

5 Molecular Datasets and Benchmarks

We summarize representative molecular representation learning algorithms in Table 2. To conveniently access the empirical results, each paper is attached with code links if available. Encoding algorithms, pre-training methods, and the utilized domain knowledge are also listed. Here, pre-training methods specify contrastive learning and reconstruction tasks we discussed in Section 3. We also present the representative methods for each application and their corresponding evaluation metrics in Table 3. The SOTA for each application is also labeled. In addition, we summarize commonly used datasets for different chemical tasks in Table 4.

6 Future Directions

Graph-based methods for MRL develop fast. Although MRL has achieved satisfactory results in various applications, there are still some challenges that remain to be solved. We list several future directions for reference.

6.1 Graph-basedMRL with Spatial Learning

The 3D geometric information attracts great attention recently in graph-based MRL. There are several ways to encode 3D information. One is an equivariant graph neural network, like SE(3)-transformers [Fuchs et al., 2020]. Another category of methods takes relative 3D information as input, like the directional message passing methods [Klicpera et al., 2019; Klicpera et al., 2021a] introduced in Section 3, which include distances between atoms and angles between bonds as features to learn geometric information. SphereNet [Liu et al., 2022c] leverages spherical message passing to learn 3D molecular representation. Nevertheless, how different geometries contribute to molecular representation learning still lacks rigorous justification. There is no established standard spatial information learning method for now. It should be a promising future research direction for MRL.

6.2 Graph-basedMRL with Explainabitity

The explainability is always a challenge for MRL. To break down the gap between machine learning and chemical science, a well-designed MRL model to produce competitive prediction or generation results on chemical tasks is important but not the end of MRL research. Which molecular features play the most important part in MRL? How can MRL be helpful on explaining the process of reaction? How can MRL support the transparent generation of new drugs? The answers to these questions will facilitate the discovery and innovation in chemical science and engineering, as well as improving the trustworthiness of machine learning methods. AttSemiGAE[Ma et al., 2018], E2E[Gao et al., 2018] and GCNN[Henderson et al., 2021] own initial strategies to improve their model's explainability. However, explainable MRL remains a challenging research problem.

6.3 Graph-basedMRL with Insufficient Data

Reliable data collection and annotation are time-consuming and expensive via experiments in the laboratory. As a result, data scarcity is a common problem in chemistry, and highly hinders the development of MLR. Self-supervised and metalearning have been considered promising solutions in recent years. Guo et al. [Guo et al., 2021] and Wang et al. [Wang et al., 2021] proposed meta-learning algorithms to deal with few-shot molecule problems, which appeals to some following work. While only specific application tasks have been investigated, novel MRL algorithms should be further developed to deal with insufficient data problems.

7 Conclusion

Molecular representation learning builds a strong and vital connection between machine learning and chemical science. In this work, we introduce the problem of graph-based MRL and provide a comprehensive overview of the recent progresses on this research topic. To facilitate reproducible research, we take the first step to summarize and release the representative molecular representation learning benchmarks and commonly used datasets for the research community. This survey paper will be a useful resource for researchers in both chemistry and machine learning to advance the study of MRL and other molecular application tasks.

Acknowledgements

This work was supported by National Science Foundation under the NSF Center for Computer Assisted Synthesis (CCAS), grant number CHE-2202693.