Scientific Computing Group
	Department of Computer Science
	University of Illinois at Urbana-Champaign

Numerical Analysis PhD Qualifying Examination

Reading List

The following reading list contains many books, but you aren't expected to read all of them! Typically, you will need to read only one or two books in any given category. The point of this list is to provide a wide variety of choices so that you can find sources you are comfortable with for learning or reviewing all of the material on the exam syllabus. Some items are highlighted in red because they offer especially concise coverage of requisite topics at about the right level expected for the exam, so these are a good place to start. You may also need to consult other references, however, to flesh out some topics in greater depth or detail, but you should be aware that many of these references go far beyond the minimum you are expected to know for the exam.

General Numerical Analysis (Comprehensive, Intermediate-Level)

A good general numerical analysis text will cover perhaps half to three-quarters of the material you need to know for the exam, so this is the obvious place to begin your reading. The book by Heath is comprehensive, concise, and up to date, and is the textbook used for CS 450, so it is a good candidate to read (or reread) for exam preparation. It contains no formal proofs, however, so you may want additionally to consult one or two of the more mathematical texts listed here.

K. Atkinson, An Introduction to Numerical Analysis, 2nd ed., Wiley, 1989.
G. Dahlquist and A. Bjorck, Numerical Methods in Scientific Computing, SIAM, 2008.
P. Deuflhard and A. Hohmann, Numerical Analysis in Modern Scientific Computing, 2nd ed., Springer, 2003.
W. Gautschi, Numerical Analysis: An Introduction, Birkhauser, 1997.
M. T. Heath, Scientific Computing: An Introductory Survey, 2nd ed., McGraw-Hill, 2002.
E. Isaacson and H. B. Keller, Analysis of Numerical Methods, Dover, 1994 (reprint of 1966 ed.).
D. Kincaid and W. Cheney, Numerical Analysis: Mathematics of Scientific Computing, 3rd ed., Brooks/Cole, 2002.
R. Kress, Numerical Analysis, Springer, 1998.
J. M. Ortega, Numerical Analysis: A Second Course, SIAM, 1990 (reprint of 1972 ed.).
A. Quarteroni, R. Sacco, and F. Saleri, Numerical Mathematics, Springer, 2000.
M. Schatzman, Numerical Analysis: A Mathematical Introduction, Oxford, 2002.
H. R. Schwarz, Numerical Analysis: A Comprehensive Introduction, Wiley, 1989.
J. Stoer and R. Bulirsch, Introduction to Numerical Analysis, 3rd ed., Springer, 2002.
E. Suli and D. Mayers, An Introduction to Numerical Analysis, Cambridge, 2003.

Numerical Computation (Error Analysis, Finite-Precision Arithmetic)

These topics are covered adequately in most general texts, including Heath, but the following references provide further details and examples on error analysis and floating-point arithmetic.

D. Goldberg, What every computer scientist should know about floating-point arithmetic, ACM Computing Surveys, 18(1):5-48, 1991.
N. J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed., SIAM, 2002.
M. L. Overton, Numerical Computing with IEEE Floating Point Arithmetic, SIAM, 2001.

Numerical Linear Algebra (Linear Systems, Least Squares, Eigenvalues)

There are many good books on this topic, from the relatively concise textbook by Trefethen and Bau to the encyclopedic treatise by Golub and Van Loan. Trefethen and Bau goes very little beyond the coverage in Heath, but offers a distinct perspective that will enrich your knowledge. It would also be a good idea to consult a book specifically on iterative methods for linear systems, several of which are listed here.

Z. Bai, et al., Templates for the Solution of Algebraic Eigenvalue Problems, SIAM, 2000.
R. Barrett, et al., Templates for the Solution of Linear Systems, SIAM, 1994.
A. Bjorck, Numerical Methods for Least Squares Problems, SIAM, 1996.
J. W. Demmel, Applied Numerical Linear Algebra, SIAM, 1997.
G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., Johns Hopkins, 1996.
A. Greenbaum, Iterative Methods for Solving Linear Systems, SIAM, 1997.
C. T. Kelley, Iterative Methods for Linear and Nonlinear Equations, SIAM, 1995.
B. N. Parlett, The Symmetric Eigenvalue Problem, SIAM, 1998 (reprint of 1980 ed.).
Y. Saad, Iterative Methods for Sparse Linear Systems, 2nd ed., SIAM, 2003.
G. W. Stewart, Matrix Algorithms, Vol. I: Basic Decompositions, SIAM, 1998; Vol. II: Eigensystems, SIAM, 2001
L. N. Trefethen and D. Bau, Numerical Linear Algebra, SIAM, 1997.
H. A. van der Vorst, Iterative Krylov Methods for Large Linear Systems, Cambridge, 2003
D. S. Watkins, Fundamentals of Matrix Computations, 2nd ed., Wiley, 2002

Nonlinear Equations and Optimization

The books by Kelley provide concise coverage, but with greater mathematical detail than most general NA texts, of everything you need to know in this category. The other references listed here are excellent, but contain either greater depth (e.g., Ortega and Rheinboldt) or more topics (e.g., constrained optimization) than required for the exam.

J. E. Dennis and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, SIAM, 1996 (reprint of 1983 ed.).
P. E. Gill, W. Murray and M. H. Wright, Practical Optimization, Academic, 1981.
C. T. Kelley, Iterative Methods for Linear and Nonlinear Equations, SIAM, 1995.
C. T. Kelley, Iterative Methods for Optimization, SIAM, 1999.
C. T. Kelley, Solving Nonlinear Equations with Newton's Method, SIAM, 2003.
J. Nocedal and S. J. Wright, Numerical Optimization, Springer, 1999.
J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic, 1970.

Interpolation and Approximation

These topics are adequately covered in many general NA texts, but it may nevertheless be a good idea to gain additional depth and details by consulting a more specialized book, such as Phillips. Note that Heath covers interpolation but not approximation of functions.

P. J. Davis, Interpolation and Approximation, Dover, 1975 (reprint of 1963 ed.).
C. de Boor, A Practical Guide to Splines, 2nd ed., Springer, 1984.
W. Gautschi, Orthogonal Polynomials: Computation and Approximation, Oxford, 2004
G. M. Phillips, Interpolation and Approximation by Polynomials, Springer, 2003.
T. J. Rivlin, An Introduction to the Approximation of Functions, Dover, 1981 (reprint of 1969 ed.).

Numerical Integration and Differentiation

Again, these topics are adequately covered in many general NA texts, but it may nevertheless be a good idea to gain additional depth and details by consulting a more specialized book, though all of the books listed go well beyond the minimum you will need.

P. J. Davis and P. Rabinowitz, Methods of Numerical Integration, 2nd ed., Academic, 1984.
G. Evans, Practical Numerical Integration, Wiley, 1993.
A. R. Krommer and C. W. Ueberhuber, Computational Integration, SIAM, 1998.

Numerical Solution of ODEs

Although this topic is covered adequately for purposes of the exam in many general NA texts, it is probably a good idea to go somewhat beyond that level by reading a more specialized source, such as Ascher and Petzold or Iserles, both of which are reasonably concise (but still go somewhat beyond the minimum you need).

U. M. Ascher and L. R. Petzold, Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations, SIAM, 1998
J. C. Butcher, Numerical Methods for Ordinary Differential Equations, 2nd ed., Wiley, 2003.
P. Deuflhard and F. Bornemann, Scientific Computing with Ordinary Differential Equations, Springer, 2002
J. R. Dormand, Numerical Methods for Differential Equations, CRC Press, 1996.
A. Iserles, A First Course in the Numerical Analysis of Differential Equations, Cambridge, 1996.
L. F. Shampine, Numerical Solution of Ordinary Differential Equations, Chapman & Hall, 1994.

Numerical Solution of PDEs

This is the topic on the exam syllabus that is by far the least well covered in general NA texts (indeed, some omit it entirely), so you will definitely need to do some additional reading here. Concise, minimally adequate sources are Morton and Mayers for finite difference methods and Johnson for finite element methods, both of which have been used as texts for CS 455. Note that Johnson's book has been superseded by the more recent book by Eriksson et al., but the latter is twice as long. Another attractive alternative is the book by Iserles, which covers both ODEs and PDEs (both finite difference and finite element methods) from a single, coherent perspective, and is relatively concise considering its breadth of coverage.

U. M. Ascher, Numerical Methods for Evolutionary Differential Equations, SIAM, 2008.
O. Axelsson and V. A. Barker, Finite Element Solution of Boundary Value Problems, SIAM, 2001 (reprint of 1984 ed.).
D. Braess, Finite Elements: Theory, Fast Solvers, and Applications in Solid Mechanics, 3rd ed., Cambridge, 2007.
K. Eriksson, D. Estep, P. Hansbo and C. Johnson, Computational Differential Equations, Cambridge, 1996.
M. S. Gockenbach, Partial Differential Equations: Analytical and Numerical Methods, SIAM, 2002.
M. S. Gockenbach, Understanding and Implementing the Finite Element Method, SIAM, 2006.
A. Iserles, A First Course in the Numerical Analysis of Differential Equations, Cambridge, 1996.
C. Johnson, Numerical Solution of Partial Differential Equations by the Finite Element Method, Cambridge, 1987.
H. P. Langtangen, Computational Partial Differential Equations, 2nd ed., Springer, 2003.
R. J. LeVeque, Finite Difference Methods for Ordinary and Partial Differential Equations, SIAM, 2007.
K. W. Morton and D. F. Mayers, Numerical Solution of Partial Differential Equations, 2nd ed., Cambridge, 2005.
A. Quarteroni and A. Valli, Numerical Approximation of Partial Differential Equations, 2nd ed., Springer 1997.
J. C. Strikwerda, Finite Difference Schemes and Partial Differential Equations, 2nd ed., SIAM, 2004.
J. W. Thomas, Numerical Partial Differential Equations, Vol. 1: Finite Difference Methods, Springer, 1995.
J. W. Thomas, Numerical Partial Differential Equations, Vol. 2: Conservation Laws and Elliptic Equations, Springer, 1999.

Mathematical Background

Research in numerical analysis generally relies on a substantial knowledge of mathematics and mathematical techniques. The relevant mathematical background material is typically learned through a combination of courses, books, and experience. Listed here are the major mathematical topics with which you should be reasonably familiar, along with (highly selective) suggested references for learning or reviewing them at an appropriate level for the exam. For convenience, mathematics courses at UIUC that cover each topic are also listed here. This does not mean that you are required to take these courses (or their equivalent elsewhere) or read these specific references in order to pass the exam. However, mathematical questions that are directly relevant to numerical methods (e.g., boundary conditions and well-posedness for PDEs, integration by parts, divergence theorem, etc.) may be asked on the exam, and you should prepare accordingly.

Linear Algebra (Math 415 or 418): vector spaces, linear independence, rank, bases, orthogonality, projectors
- G. Strang, Introduction to Linear Algebra, 3rd ed., Wellesley-Cambridge Press, 2003.
Matrix Theory (Math 415 or 418): eigenvalues, multiplicity, Schur form, Jordan form
- C. D. Meyer, Matrix Analysis and Applied Linear Algebra, SIAM, 2000.
- R. A. Horn and C. R. Johnson, Matrix Analysis, Cambridge, 1985.
Vector Analysis (Math 481): gradient, divergence, curl, divergence theorem
- P. C. Matthews, Vector Calculus, Springer, 1998.
Ordinary Differential Equations (Math 441 or 550): IVPs, BVPs, existence, uniqueness, stability
- R. M. M. Mattheij and J. Molenaar, Ordinary Differential Equations in Theory and Practice, SIAM, 2002 (reprint of 1996 ed.).
Partial Differential Equations (Math 442 or 553): classification, initial/boundary conditions, characteristics
- W. A. Strauss, Partial Differential Equations, An Introduction, Wiley, 1992.
- A. Tveito and R. Winther, Introduction to Partial Differential Equations: A Computational Approach, Springer, 1998.
Real Analysis (Math 444, 447, or 540/541): limits, continuity, differentiability, integrability
- S. Abbott, Understanding Analysis, Springer, 2001.
- D. Estep, Practical Analysis in One Variable, 2002.
Functional Analysis (Math 546): Banach spaces, Hilbert spaces, Sobolev spaces, linear operators, linear functionals
- K. Atkinson and W. Han, Theoretical Numerical Analysis: A Functional Analysis Framework, Springer, 2001.
- K. Saxe, Beginning Functional Analysis, Springer, 2002.
Applied Mathematics (Math 498, 556/557 or TAM 541/542):
- G. Strang, Introduction to Applied Mathematics, Wellesley-Cambridge Press, 1986.
- G. Strang, Computational Science and Engineering, Wellesley-Cambridge Press, 2007.

Scientific Computing Group, Department of Computer Science, University of Illinois at Urbana-Champaign, 201 N. Goodwin Ave., Urbana, IL 61801, USA.