Antonio Coppola edited Introduction.tex  over 9 years ago

Commit id: 75c470f19d772db4c6030a6daa7da20ca92025a8

deletions | additions      

       

In this vignette we demonstrate how to use the \textbf{lbfgs} R package. \footnote{We are extremely thankful for the advice and support of Dustin Tingley.} The original motivation for writing the package was the lack of a general R implementation of the Orthant-Wise Quasi-Newton Limited-Memory (OWL-QN) optimization algorithm \cite{Andrew_Gao_2007}, except in limited-scope contexts. The package uses the \href{http://www.chokkan.org/software/liblbfgs/}{libLBFGS} C++ library written by Naoaki Okazaki, which itself is a port of the Fortran method by Jorge Nocedal \cite{Nocedal_1980}.   \subsection{Notation}  Throughout this vignette, we will be using the following notation. Let $f: \mathbb{R}^\mathbb{n} \mapsto \mathbb{R}$ be the objective function to be minimized. We also let the $\lvert \lvert \cdot \lvert \lvert$ operator denote the $L_2$ norm of a vector, and $\lvert \lvert \cdot \lvert \lvert_1$ denote the $L_1$ norm.  \subsection{The L-BFGS Algorithm}  The Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm \cite{Liu_Nocedal_1989} is employed for solving high-dimensional minimization problems in scenarios where both the objective function and its gradient can be computed analytically. The L-BFGS algorithm belongs to the class of quasi-Newton optimization routines, which solve the given minimization problem by computing approximations to the Hessian matrix of the objective function. The algorith is specified as follows: