Файл:A Method for Large-Scale L1-Regularized Logistic Regression L1 logistic reg aaai.pdf

Kwangmoo Koh and Seung-Jean Kim and Stephen Boyd Electrical Engineering Department Stanford University Stanford, CA 94305

Abstract

Logistic regression with ℓ1 regularization has been proposed as a promising method for feature selection in classification problems. Several specialized solution methods have been proposed for ℓ1-regularized logistic regression problems (LRPs). However, existing methods do not scale well to large problems that arise in many practical settings. In this paper we describe an efficient interior-point method for solving ℓ1-regularized LRPs. Small problems with up to a thousand or so features and examples can be solved in seconds on a PC. A variation on the basic method, that uses a preconditioned conjugate gradient method to compute the search step, can solve large sparse problems, with a million features and examples (e.g., the 20 Newsgroups data set), in a few tens of minutes, on a PC. Numerical experiments show that our method outperforms standard methods for solving convex optimization problems as well as other methods specifically designed for ℓ1regularized LRPs.