Antonio Coppola edited Introduction.tex  over 9 years ago

Commit id: 1762507d61525f2d7581872f65cd1fb090133f18

deletions | additions      

       

Q(x) = f(x^k) + (x - x^k)^T g^k + \frac{1}{2} (x - x^k)^T \mathbf{B_k} (x - x^k)  \]  A search direction can then be found by computing the vector $x^*$ that minimizes $Q(x)$. Assuming that Hessian is positive-definite, this is $x^* = x^k - \mathbf{H_k} g^k$. The next search point is then found along the ray defined by $ x^k - \alpha \mathbf{H_k} g^k$. The procedure is iterated until the gradient is zero, with some degree of convergence tolerance. The In order to optimize memory usage, the  L-BFGS algorithm avoids storing the sequential approximations of the Hessian matrix. Instead, L-BFGS stores curvature information from the last $m$ iterations of the algorithm, and uses them to find the new search direction. More specifically,  ... \subsection{The OWL-QN Algorithm}