A description of how quasi Newton algorithms in general, and in special the BFGS algorithm work. Animations are made with the manimce library.
Sources:
* Nocedal & Wright: Numerical Optimization Ch. 6 (which also presents a SR1 trust region method)
* Dennis & More: Quasi-Newton Methods, Motivation, and Theory, SIAM Review, Vol. 19, No. 1, 1977 (describing the PSB method. The video is based mostly on the derivation in section 7 of this paper.)
The actual update formulas of BFGS are not included in the video. These can be found in both sources as well as e.g. Wikipedia.
19 май 2023
