Introduction to Continuous Optimization

Book, 2013

This second edition introduces several areas and items that were not included in the first edition, as well as several corrections. A brief summary of these changes are given next. Chapter 1 includes a discussion on the diet problem, in addition to that on the staff planning problem, in order to very early on introduce linear programming. Figure 1.1 now has the terminating box ``Implementation'', whereas the original one had an infinite loop! Chapter 3 has been enriched by several new results on separating and supporting hyperplanes, and the associated theory of convex cones and their polar sets. Thanks to this study of separating hyperplanes,Theorem 5.17 on the necessity of the Fritz John conditions now has a complete proof. The end of Chapter 5 also includes a summary of the fascinating story of the development of the Karush--Kuhn--Tucker conditions. The sensitivity analysis in linear programming has been expanded with a discussion in Section 10.5.3 on the addition of a variable or a constraint, as well as an introduction to column generation based on the example of the minimum cost multi-commodity network flow problem (Section 10.6). Chapter 11 includes a brief discussion on Gauss--Newton methods for least-squares problems. Chapter 12 has changed its name from ``Optimization over convex sets'' to ``Feasible-direction methods,'' in order to reflect the fact that the scope is now wider---from essentially polyhedral sets to general closed sets (which, however, most often will be assumed to be convex). In particular, we have added new sections on algorithms defined by closed descent maps---an algorithm principle which was devised and analyzed mainly in the 1960s, and which is a quite elegant means to describe iterative methods. We also utilize this principle to contrast established convergent methods (such as the Frank--Wolfe method) and failed attempts (such as the algorithm of Zoutendijk). We have also added a brief discussion on reduced gradient methods, which are relatives to the simplex method; they are---in their original statement---not convergent, but a small adjustment results in a closed descent map and hence a convergent method. Exercises and their solutions are now placed at the end of the book, rather than at the end of each chapter. The first edition from 2005 has been used in teaching of several courses at Chalmers University of Technology and the University of Gothenburg. We wish to thank all the students who have given us remarks on the book. We would also like to thank Dr. Kin Cheong Sou for remarks and corrections on the first edition.