Proceedings

Robust Flexible Feature Selection via Exclusive L21 Regularization

Download BibTeX Cite

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Main track. Pages 3158-3164.

https://doi.org/10.24963/ijcai.2019/438

© 2019 International Joint Conferences on Artificial Intelligence

Abstract

Recently, exclusive lasso has demonstrated its promising results in selecting discriminative features for each class. The sparsity is enforced on each feature across all the classes via L12-norm. However, the exclusive sparsity of L12-norm could not screen out a large amount of irrelevant and redundant noise features in high-dimensional data space, since each feature belongs to at least one class. Thus, in this paper, we introduce a novel regularization called "exclusive L21", which is short for "L21 with exclusive lasso", towards robust flexible feature selection. The exclusive L21 regularization is the mix of L21-norm and L12-norm, which brings out joint sparsity at inter-group level and exclusive sparsity at intra-group level simultaneously. An efficient augmented Lagrange multipliers based optimization algorithm is proposed to iteratively solve the exclusive L21 regularization in a row-wise fashion. Extensive experiments on twelve benchmark datasets demonstrate the effectiveness of the proposed regularization and the optimization algorithm as compared to state-of-the-arts.

  1. [Bhattacharjee et al., 2001] Arindam Bhattacharjee, William G. Richards, Jane Staunton, Cheng Li, et al. Classification of human lung carcinomas by mrna expres- sion profiling reveals distinct adenocarcinoma subclasses. Proceedings of the National Academy of Sciences, 98(24):13790-13795, 2001.
  2. [Campbell and Allen, 2017] Frederick Campbell and Genev- era I. Allen. Within group variable selection through the exclusive lasso. Electronic Journal of Statistics, 11(2):4220-4257, 2017.
  3. [Cavazza et al., 2018] Jacopo Cavazza, Pietro Morerio, Ben- jamin Haeffele, Connor Lane, et al. Dropout as a low-rank regularizer for matrix factorization. In Proceedings of the Twenty-First International Conference on Artificial Intel- ligence and Statistics, pages 435-444, 2018.
  4. [Chang and Lin, 2011] Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011.
  5. [Ciarelli and Oliveira, 2009] Patrick M. Ciarelli and Elias Oliveira. Agglomeration and elimination of terms for di- mensionality reduction. In 2009 Ninth International Con- ference on Intelligent Systems Design and Applications, pages 547-552, Nov 2009.
  6. [Ding and Peng, 2003] Chris Ding and Hanchuan Peng. Minimum redundancy feature selection from microarray gene expression data. In Computational Systems Bioin- formatics. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003, pages 523-528, Aug 2003.
  7. [Gui et al., 2017] Jie Gui, Zhenan Sun, Shuiwang Ji, Dacheng Tao, and Tieniu Tan. Feature selection based on structured sparsity: A comprehensive study. IEEE Transactions on Neural Networks and Learning Systems, 28(7):1490-1507, July 2017.
  8. [Guyon and Elisseeff, 2003] Isabelle Guyon and André Elis- seeff. An introduction to variable and feature selec- tion. Journal of Machine Learning Research, 3:1157- 1182, March 2003.
  9. [Guyon et al., 2002] Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. Gene selection for can- cer classification using support vector machines. Machine Learning, 46(1):389-422, Jan 2002.
  10. [Kong et al., 2014] Deguang Kong, Ryohei Fujimaki, Ji Liu, Feiping Nie, and Chris Ding. Exclusive feature learning on arbitrary structures via ℓ 1,2 -norm. In Advances in Neu- ral Information Processing Systems 27, pages 1655-1663, 2014.
  11. [Lecun et al., 1998] Yann Lecun, Léon Bottou, Yoshua Ben- gio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, Nov 1998.
  12. [Liu et al., 2009] Jun Liu, Shuiwang Ji, and Jieping Ye. Multi-task feature learning via efficient ℓ 2,1 -norm mini- mization. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pages 339-348, 2009.
  13. [Ming et al., 2019] Di Ming, Chris Ding, and Feiping Nie. A probabilistic derivation of lasso and l12-norm feature selections. In Proceedings of the Thirty-Third AAAI Con- ference on Artificial Intelligence (AAAI-19), 2019.
  14. [Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris Ding. Efficient and robust feature selection via joint ℓ 2,1 -norms minimization. In Advances in Neural Informa- tion Processing Systems 23, pages 1813-1821, 2010.
  15. [Nutt et al., 2003] Catherine L. Nutt, D. R. Mani, Re- becca A. Betensky, Pablo Tamayo, et al. Gene expression- based classification of malignant gliomas correlates better with survival than histological classification. Cancer Re- search, 63(7):1602-1607, 2003.
  16. [Peng et al., 2005] Hanchuan Peng, Fuhui Long, and Chris Ding. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min- redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8):1226-1238, Aug 2005.
  17. [Quattoni et al., 2009] Ariadna Quattoni, Xavier Carreras, Michael Collins, and Trevor Darrell. An efficient projec- tion for ℓ 1,∞ regularization. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 857-864, 2009.
  18. [Ramaswamy et al., 2001] Sridhar Ramaswamy, Pablo Tamayo, Ryan Rifkin, Sayan Mukherjee, et al. Multiclass cancer diagnosis using tumor gene expression signa- tures. Proceedings of the National Academy of Sciences, 98(26):15149-15154, 2001.
  19. [Robnik-Šikonja and Kononenko, 2003] Marko Robnik- Šikonja and Igor Kononenko. Theoretical and empirical analysis of relieff and rrelieff. Machine Learning, 53(1):23-69, Oct 2003.
  20. [Sim et al., 2002] Terence Sim, Simon Baker, and Maan Bsat. The cmu pose, illumination, and expression (pie) database. In Proceedings of Fifth IEEE International Con- ference on Automatic Face Gesture Recognition, pages 53-58, 2002.
  21. [Tibshirani, 1996] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58(1):267-288, 1996.
  22. [Wang et al., 2007] Li Wang, Ji Zhu, and Hui Zou. Hybrid huberized support vector machines for microarray classi- fication. In Proceedings of the 24th International Confer- ence on Machine Learning, pages 983-990, 2007.
  23. [Yang et al., 2006] Kun Yang, Zhipeng Cai, Jianzhong Li, and Guohui Lin. A stable gene selection in microarray data analysis. BMC Bioinformatics, 7(1):228, Apr 2006.
  24. [Zhao et al., 2009] Peng Zhao, Guilherme Rocha, and Bin Yu. The composite absolute penalties family for grouped and hierarchical variable selection. The Annals of Statis- tics, 37(6A):3468-3497, 2009.
  25. [Zhou et al., 2010] Yang Zhou, Rong Jin, and Steven C.H. Hoi. Exclusive lasso for multi-task feature selection. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 988-995, 2010.
  26. [Zhu et al., 2003] Ji Zhu, Saharon Rosset, Trevor Hastie, and Rob Tibshirani. 1-norm support vector machines. In Pro- ceedings of the 16th International Conference on Neural Information Processing Systems, pages 49-56, 2003.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards