Cuckoo Feature Hashing: Dynamic Weight Sharing for Sparse Analytics

Cuckoo Feature Hashing: Dynamic Weight Sharing for Sparse Analytics

Jinyang Gao, Beng Chin Ooi, Yanyan Shen, Wang-Chien Lee

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 2135-2141. https://doi.org/10.24963/ijcai.2018/295

Feature hashing is widely used to process large scale sparse features for learning of predictive models. Collisions inherently happen in the hashing process and hurt the model performance. In this paper, we develop a feature hashing scheme called Cuckoo Feature Hashing(CCFH) based on the principle behind Cuckoo hashing, a hashing scheme designed to resolve collisions. By providing multiple possible hash locations for each feature, CCFH prevents the collisions between predictive features by dynamically hashing them into alternative locations during model training. Experimental results on prediction tasks with hundred-millions of features demonstrate that CCFH can achieve the same level of performance by using only 15%-25% parameters compared with conventional feature hashing.
Keywords:
Machine Learning: Feature Selection ; Learning Sparse Models
Computer Vision: Big Data and Large Scale Methods
Machine Learning: Dimensionality Reduction and Manifold Learning