Solving Separable Nonsmooth Problems Using Frank-Wolfe with Uniform Affine Approximations

Solving Separable Nonsmooth Problems Using Frank-Wolfe with Uniform Affine Approximations

Edward Cheung, Yuying Li

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 2035-2041. https://doi.org/10.24963/ijcai.2018/281

Frank-Wolfe methods (FW) have gained significant interest in the machine learning community due to their ability to efficiently solve large problems that admit a sparse structure (e.g. sparse vectors and low-rank matrices). However the performance of the existing FW method hinges on the quality of the linear approximation. This typically restricts FW to smooth functions for which the approximation quality, indicated by a global curvature measure, is reasonably good. In this paper, we propose a modified FW algorithm amenable to nonsmooth functions, subject to a separability assumption, by optimizing for approximation quality over all affine functions, given a neighborhood of interest. We analyze theoretical properties of the proposed algorithm and demonstrate that it overcomes many issues associated with existing methods in the context of nonsmooth low-rank matrix estimation.
Keywords:
Machine Learning: Structured Prediction
Machine Learning: Recommender Systems
Machine Learning: Feature Selection ; Learning Sparse Models
Machine Learning: Semi-Supervised Learning