Antonio Coppola edited Introduction.tex  over 9 years ago

Commit id: a155646576d5b91155a513b179a516387c869b44

deletions | additions      

       

\section{Introduction}  In this vignette we demonstrate how to use the \textbf{lbfgs} R package. \footnote{We are extremely thankful to Dustin Tingley  for the his  advice and support of Dustin Tingley.} support.}  The original motivation for writing the package was the lack of a general R implementation of the Orthant-Wise Quasi-Newton Limited-Memory (OWL-QN) optimization algorithm \cite{Andrew_Gao_2007}, except in limited-scope contexts. The package uses the \href{http://www.chokkan.org/software/liblbfgs/}{libLBFGS} C++ library written by Naoaki Okazaki, which itself is a port of the Fortran method by Jorge Nocedal \cite{Nocedal_1980}. The linkage between R and C++ is achieved using Rcpp \cite{Eddelbuettel_2013}. \subsection{Notation} 

\subsection{The OWL-QN Algorithm}  The L-BFGS method cannot be applied to problems with an objective function of the form $r(x) = C \cdot \lvert \lvert x \lvert \lvert _1 = C \cdot \sum_i \lvert x_i \lvert$, such as LASSO regression or $L_1$-penalized log-linear models, given the non-differentiability of the objective function at any point where at least one of the parameters is zero. The OWL-QN algorithm exploits the fact that $L_1$-regularized objective functions will still be differential in any given orthant of the functional space. At each iteration, the algorithm chooses an orthant within which to evalute the function by estimating the sign of each of its parameters. The algorithm then constructs a quadratic approximation to the function in the given orthant using a regular L-BFGS procedure, and searches in the direction of the minimum of the approximation within the same orthant. For further details regarding OWL-QN, we refer the interested  reader to the original article \cite{Andrew_Gao_2007}.