A Dual Input-aware Factorization Machine for CTR Prediction

A Dual Input-aware Factorization Machine for CTR Prediction

Wantong Lu, Yantao Yu, Yongzhe Chang, Zhen Wang, Chenhui Li, Bo Yuan

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence
Main track. Pages 3139-3145. https://doi.org/10.24963/ijcai.2020/434

Factorization Machines (FMs) refer to a class of general predictors working with real valued feature vectors, which are well-known for their ability to estimate model parameters under significant sparsity and have found successful applications in many areas such as the click-through rate (CTR) prediction. However, standard FMs only produce a single fixed representation for each feature across different input instances, which may limit the CTR model’s expressive and predictive power. Inspired by the success of Input-aware Factorization Machines (IFMs), which aim to learn more flexible and informative representations of a given feature according to different input instances, we propose a novel model named Dual Input-aware Factorization Machines (DIFMs) that can adaptively reweight the original feature representations at the bit-wise and vector-wise levels simultaneously. Furthermore, DIFMs strategically integrate various components including Multi-Head Self-Attention, Residual Networks and DNNs into a unified end-to-end model. Comprehensive experiments on two real-world CTR prediction datasets show that the DIFM model can outperform several state-of-the-art models consistently.
Keywords:
Machine Learning: Recommender Systems
Multidisciplinary Topics and Applications: Recommender Systems
Data Mining: Classification, Semi-Supervised Learning