Escaping Feature Twist: A Variational Graph Auto-Encoder for Node Clustering

Escaping Feature Twist: A Variational Graph Auto-Encoder for Node Clustering

Nairouz Mrabah, Mohamed Bouguessa, Riadh Ksantini

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
Main Track. Pages 3351-3357. https://doi.org/10.24963/ijcai.2022/465

Most recent graph clustering methods rely on pretraining graph auto-encoders using self-supervision techniques (pretext task) and finetuning based on pseudo-supervision (main task). However, the transition from self-supervision to pseudo-supervision has never been studied from a geometric perspective. Herein, we establish the first systematic exploration of the latent manifolds' geometry under the deep clustering paradigm; we study the evolution of their intrinsic dimension and linear intrinsic dimension. We find that the embedded manifolds undergo coarse geometric transformations under the transition regime: from curved low-dimensional to flattened higher-dimensional. Moreover, we find that this inappropriate flattening leads to clustering deterioration by twisting the curved structures. To address this problem, which we call Feature Twist, we propose a variational graph auto-encoder that can smooth the local curves before gradually flattening the global structures. Our results show a notable improvement over multiple state-of-the-art approaches by escaping Feature Twist.
Keywords:
Machine Learning: Unsupervised Learning
Machine Learning: Autoencoders
Machine Learning: Clustering
Machine Learning: Representation learning
Machine Learning: Self-supervised Learning