Neither the paper nor the comments have mentioned this: https://en.wikipedia.org/wiki/Truncated_Newton_method https://en.wikipedia.org/wiki/Truncated_Newton_me

Neither the paper nor the comments have mentioned this: https://en.wikipedia.org/wiki/Truncated_Newton_method https://en.wikipedia.org/wiki/Truncated_Newton_method The Truncated Newton method uses an inner solver that only runs for a few iterations to approximate the Hessian. The approximate Hessian is used to approximately solve Newtons equation. I've implemented it and it works very well. When it gets close to the solution the convergence is very fast. I mention it because it sounds similar to what the paper discusses but you use conjugate gradients in the inner solver and Newton's equation in the outer solver.