"It (BFGS) is also more numerically stable, and has very effective “self-correcting properties” not shared by DFP, which may account for its superior performance in practice."
"In a sense, the line search and trust-region approaches differ in the order in which they choose the direction and distance of the move to the next iterate. Line search starts by fixing the direction pk and then identifying an appropriate distance, namely the step length αk . In trust region, we fir"
"This book focuses on smooth functions, by which we generally mean functions whose second derivatives exist and are continuous."
"Most of these bits (say t) are devoted to storing the fractional part, while the remainder encode the exponent e and other information, such as the sign of the number, or an indication of whether it is zero or "undefined.""
"This phenomenon is known as cancellation. It can also be explained (less formally) by noting that if both x and y are accurate to k digits, and if they agree in the frist k' digits, then their difference will contain only about k-k' significant digits–the first k' digits cancel each other out. This "