Optimization Theory and Methods Nonlinear Programming [Paperback]

Optimization Theory and Methods can be used as a textbook for an optimization course for graduates and senior undergraduates. It is the result of the author's teaching and research over the past decade. It describes optimization theory and several powerful methods. For most methods, the book discusses an ideas motivation, studies the derivation, establishes the global and local convergence, describes algorithmic steps, and discusses the numerical performance.

Preface1 Introduction1.1 Introduction1.2 Mathematics Foundations1.2.1 Norm1.2.2 Inverse and Generalized Inverse of a Matrix1.2.3 Properties of Eigenvalues1.2.4 Rank-One Update1.2.5 Function and Differential1.3 Convex Sets and Convex Functions1.3.1 Convex Sets1.3.2 Convex Functions1.3.3 Separation and Support of Convex Sets1.4 Optimality Conditions for Unconstrained Case1.5 Structure of Optimization MethodsExercises2 Line Search2.1 Introduction2.2 Convergence Theory for Exact Line Search2.3 Section Methods2.3.1 The Golden Section Method2.3.2 The Fibonacci Method2.4 Interpolation Method2.4.1 Quadratic Interpolation Methods2.4.2 Cubic Interpolation Method2.5 Inexact Line Search Techniques2.5.1 Armijo and Goldstein Rule2.5.2 Wolfe-Powell Rule2.5.3 Goldstein Algorithm and Wolfe-Powell Algorithm2.5.4 Backtracking Line Search2.5.5 Convergence Theorems of Inexact Line SearchExercises3 Newtons Methods3.1 The Steepest Descent Method3.1.1 The Steepest Descent Method3.1.2 Convergence of the Steepest Descent Method3.1.3 Barzilai and Borwein Gradient Method3.1.4 Appendix: Kantorovich Inequality3.2 Newtons Method3.3 Modified Newtons Method3.4 Finite-Difference Newtons Method3.5 Negative Curvature Direction Method3.5.1 Gill-Murray Stable Newtons Method3.5.2 Fiacco-McCormick Method3.5.3 Fletcher-Freeman Method3.5.4 Second-Order Step Rules3.6 Inexact Newtons MethodExercises4 Conjugate Gradient Method4.1 Conjugate Direction Methods4.2 Conjugate Gradient Method4.2.1 Conjugate Gradient Method4.2.2 Beales lÛ