Proceedings

PDDFormer: Pairwise Distance Distribution Graph Transformer for Crystal Material Property Prediction

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 7724-7732.

https://doi.org/10.24963/ijcai.2025/859

Abstract

Crystal structures can be simplified as a periodic point set that repeats across three-dimensional space along an underlying lattice. Traditionally, crystal representation methods rely on descriptors such as lattice parameters, symmetry, and space groups to characterize the structure. However, in reality, atoms in materials always vibrate above absolute zero, causing their positions to fluctuate continuously. This dynamic behavior disrupts the fundamental periodicity of the lattice, making crystal graphs based on static lattice parameters and conventional descriptors discontinuous under slight perturbations. Chemists proposed the pairwise distance distribution (PDD) method to address this. However, the completeness of PDD requires defining a large number of neighboring atoms, leading to high computational costs. Additionally, PDD does not account for atomic information, making it challenging to apply it directly to crystal material property prediction tasks. To tackle these challenges, we introduce the atom-weighted Pairwise Distance Distribution (WPDD) and Unit cell Pairwise Distance Distribution (UPDD) for the first time, applying them to the construction of multi-edge crystal graphs. We demonstrate the continuity and general completeness of crystal graphs under slight atomic position perturbations. Moreover, by modeling PDD as global information and integrating it into matrix-based message passing, we significantly reduce computational costs. Comprehensive evaluation results show that WPDDFormer achieves state-of-the-art predictive accuracy across tasks on benchmark datasets such as the Materials Project and JARVIS-DFT.

  1. [Ahmad et al., 2018] Zeeshan Ahmad, Tian Xie, Chinmay Maheshwari, Jeffrey C Grossman, and Venkatasubrama- nian Viswanathan. Machine learning enabled computa- tional screening of inorganic solid electrolytes for suppres- sion of dendrite formation in lithium metal anodes. ACS central science, 4(8):996-1006, 2018.
  2. [Ba, 2016] Jimmy Lei Ba. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.
  3. [Balasingham et al., 2022] Jonathan Balasingham, Viktor Zamaraev, and Vitaliy Kurlin. Compact graph represen- tation of molecular crystals using point-wise distance dis- tributions. arXiv preprint arXiv:2212.11246, 2022.
  4. [Balasingham et al., 2024] Jonathan Balasingham, Viktor Zamaraev, and Vitaliy Kurlin. Material property predic- tion using graphs based on generically complete isometry invariants. Integrating Materials and Manufacturing In- novation, pages 1-14, 2024.
  5. [Bartók et al., 2013] Albert P Bartók, Risi Kondor, and Gábor Csányi. On representing chemical environments. Physical Review B-Condensed Matter and Materials Physics, 87(18):184115, 2013.
  6. [Batzner et al., 2022] Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):2453, 2022.
  7. [Carstens et al., 1999] Hans-Georg Carstens, Walter A Deu- ber, Wolfgang Thumser, and Elke Koppenrade. Geomet- rical bijections in discrete lattices. Combinatorics, Proba- bility and Computing, 8(1-2):109-129, 1999.
  8. [Chen and Ong, 2022] Chi Chen and Shyue Ping Ong. A universal graph deep learning interatomic potential for the periodic table. Nature Computational Science, 2(11):718- 728, 2022.
  9. [Chen et al., 2019] Chi Chen, Weike Ye, Yunxing Zuo, Chen Zheng, and Shyue Ping Ong. Graph networks as a univer- sal machine learning framework for molecules and crys- tals. Chemistry of Materials, 31(9):3564-3572, 2019.
  10. [Choudhary and DeCost, 2021] Kamal Choudhary and Brian DeCost. Atomistic line graph neural network for improved materials property predictions. npj Computa- tional Materials, 7(1):185, 2021.
  11. [Choudhary et al., 2020] Kamal Choudhary, Kevin F Gar- rity, Andrew CE Reid, Brian DeCost, Adam J Biacchi, Angela R Hight Walker, Zachary Trautt, Jason Hattrick- Simpers, A Gilad Kusne, Andrea Centrone, et al. The joint automated repository for various integrated simula- tions (jarvis) for data-driven materials design. npj compu- tational materials, 6(1):173, 2020.
  12. [Das et al., 2022] Kishalay Das, Bidisha Samanta, Pawan Goyal, Seung-Cheol Lee, Satadeep Bhattacharjee, and Niloy Ganguly. Crysxpp: An explainable property predic- tor for crystalline materials. npj Computational Materials, 8(1):43, 2022.
  13. [Das et al., 2023a] Kishalay Das, Pawan Goyal, Seung- Cheol Lee, Satadeep Bhattacharjee, and Niloy Ganguly. Crysmmnet: multimodal representation for crystal prop- erty prediction. In Uncertainty in Artificial Intelligence, pages 507-517. PMLR, 2023.
  14. [Das et al., 2023b] Kishalay Das, Bidisha Samanta, Pawan Goyal, Seung-Cheol Lee, Satadeep Bhattacharjee, and Niloy Ganguly. Crysgnn: Distilling pre-trained knowledge to enhance property prediction for crystalline materials. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 37, pages 7323-7331, 2023.
  15. [Groom et al., 2016] Colin R Groom, Ian J Bruno, Matthew P Lightfoot, and Suzanna C Ward. The cambridge structural database. Structural Science, 72(2):171-179, 2016.
  16. [Ioffe, 2015] Sergey Ioffe. Batch normalization: Accelerat- ing deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.
  17. [Kurlin, 2024] Vitaliy Kurlin. Mathematics of 2-dimensional lattices. Foundations of Computational Mathematics, 24(3):805-863, 2024.
  18. [Lin et al., 2023] Yuchao Lin, Keqiang Yan, Youzhi Luo, Yi Liu, Xiaoning Qian, and Shuiwang Ji. Efficient ap- proximations of complete interatomic potentials for crystal property prediction. In International Conference on Ma- chine Learning, pages 21260-21287. PMLR, 2023.
  19. [Louis et al., 2020] Steph-Yves Louis, Yong Zhao, Alireza Nasiri, Xiran Wang, Yuqi Song, Fei Liu, and Jianjun Hu. Graph convolutional neural networks with global atten- tion for improved materials property prediction. Physical Chemistry Chemical Physics, 22(32):18141-18148, 2020.
  20. [Patterson, 1944] A Lindo Patterson. Ambiguities in the x- ray analysis of crystal structures. Physical Review, 65(5- 6):195, 1944.
  21. [Rubner et al., 2000] Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. The earth mover's distance as a met- ric for image retrieval. International journal of computer vision, 40:99-121, 2000.
  22. [Song et al., 2024] Zixing Song, Ziqiao Meng, and Irwin King. A diffusion-based pre-training framework for crys- tal property prediction. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 38, pages 8993- 9001, 2024.
  23. [Taniai et al., 2024] Tatsunori Taniai, Ryo Igarashi, Yuta Suzuki, Naoya Chiba, Kotaro Saito, Yoshitaka Ushiku, and Kanta Ono. Crystalformer: Infinitely connected attention for periodic structure encoding. In The Twelfth Interna- tional Conference on Learning Representations, 2024.
  24. [Wang et al., 2022] Rui Wang, Robin Walters, and Rose Yu. Approximately equivariant networks for imperfectly sym- metric dynamics. In International Conference on Machine Learning, pages 23078-23091. PMLR, 2022.
  25. [Wassermann et al., 2010] Anne Mai Wassermann, Mathias Wawer, and Jurgen Bajorath. Activity landscape represen- tations for structure-activity relationship analysis. Journal of medicinal chemistry, 53(23):8209-8223, 2010.
  26. [Widdowson and Kurlin, 2022] Daniel Widdowson and Vi- taliy Kurlin. Resolving the data ambiguity for periodic crystals. Advances in Neural Information Processing Sys- tems, 35:24625-24638, 2022.
  27. [Widdowson et al., 2022] Daniel Widdowson, Marco M Mosca, Angeles Pulido, Andrew I Cooper, and Vitaliy Kurlin. Average minimum distances of periodic point sets-foundational invariants for mapping periodic crystals. MATCH Commun. Math. Comput. Chem, 87(3):529-559, 2022.
  28. [Xie and Grossman, 2018] Tian Xie and Jeffrey C Gross- man. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material proper- ties. Physical review letters, 120(14):145301, 2018.
  29. [Yan et al., 2022] Keqiang Yan, Yi Liu, Yuchao Lin, and Shuiwang Ji. Periodic graph transformers for crystal mate- rial property prediction. Advances in Neural Information Processing Systems, 35:15066-15080, 2022.
  30. [Yan et al., 2024a] Keqiang Yan, Cong Fu, Xiaofeng Qian, Xiaoning Qian, and Shuiwang Ji. Complete and efficient graph transformers for crystal material property predic- tion. In The Twelfth International Conference on Learning Representations, 2024.
  31. [Yan et al., 2024b] Keqiang Yan, Alexandra Saxton, Xi- aofeng Qian, Xiaoning Qian, and Shuiwang Ji. A space group symmetry informed network for o(3) equivariant crystal tensor prediction. In Forty-first International Con- ference on Machine Learning, 2024.
  32. [Zwart et al., 2008] Peter H Zwart, Ralf W Grosse- Kunstleve, Andrey A Lebedev, Garib N Murshudov, and Paul D Adams. Surprises and pitfalls arising from (pseudo) symmetry. Acta Crystallographica Section D: Biological Crystallography, 64(1):99-107, 2008.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards