UNIVERSITY OF NEW JERSEY
|1.||Course: CS402-01 Numerical Methods, 3 credits|
Carol Parken (Science Hall East 5021)
can be contacted
by telephone at (973)_720-2649 and by e-mail at ParkenC@wpunj.edu.
offered: Spring 2013
Time: Tuesday & Thursday 3:30PM - 4:45PM Location: Science Hall East 5019
Najarian, Prof. of Computer Science
Office: Science Hall East 5032, Phone: (973)-720-3383, E-mail: NajarianJ@wpunj.edu
Office Hours: Tuesday and Thursday 2:00PM - 3:15PM
and also other hours can be arranged by appointment.
1. Timothy Sauer. "Numerical Analysis".
2nd ed. Pearson. 2012.
(The Primary Textbook)
Sauer's Source code (MATLAB)
Suggested Readings (not required):
Joe D. Hoffman and Steven Frankel. "Numerical
Methods for Engineers and Scientists", Third
M.K. Jain, S. Iyengar, and R.K. Jain. "Numerical Methods for Scientific and Engineering Computation".
Brian Bradie, “Friendly Introduction to Numerical Analysis, A”, Prentice Hall, 2006.
Richard Burden and J.J. Faires. “Numerical Analysis”. Eighth Edition, Brooks/Cole, 2005.
Jeffery J. Leader, “Numerical Analysis and Scientific Computation”, Addison-Wesley, 2005.
Atkinson and Weimin Han. "Elementary Numerical Analysis", 3rd
Edition, Wiley, 2004.
Moler, Cleve. "Numerical Computing with MATLAB",
SIAM, 2004. Online version (free) at:
Ward Cheney and David Kincaid. “Numerical Mathematics and Computing”, Fifth Edition, Brooks/Cole Publishing Company, 2004.
“Introduction to Scientific Computing Using MATLAB”,
Prentice Hall, 2004.
Douglas Faires and Richard Burden, “Numerical
Methods”, Third Edition, Brooks/Cole, 2003.
Numerical Methods with MATLAB:Implementations and Applications",
“Scientific Computing”, Second Edition, McGraw Hill,
Katsman, J.J. "Numerical Methods". Tomsk: TPU Press, 2002.
Meyer, Carl. "Matrix Analysis and Applied Linear Algebra Book" (and its" Solutions Manual".
Shampine , R.C. Allen and S. Pruess, “Fundamentals
of Numerical Computing”, Wiley, 1997.
G. H. Golub and Van Loan, C , "Matrix Computations", Johns Hopkins Press, 1996.
Phillips and PJ
and Applications of Numerical Analysis”, Academic Press, 1996.
G.W. "Afternotes on Numerical Analysis", SIAM, 1996.
Asaithambi, “Numerical Analysis, Theory and Practice”, Saunders College
Terrence J. Akai, “Applied Numerical Methods for Engineers”, Wiley, 1994.
Nakamura , “Applied Numerical Analysis in C”, Prentice
Paul Hultquist, "Numerical Methods for Engineers and Computer Scientist”, Addison-Wesley, 1988.
Hamming, Numerical Methods for Scientists and
Engineers, Dover Reprint, 1987. (a classic)
B. Noble and J. Daniel, “Applied Linear Algebra”, Prentice Hall, 1977.
Germund Dahlquist and Ake Bjorck. "Numerical Methods", Prentice-Hall, 1974 (Dover reprint, 2003).
S. D. Conte, and Carl de Boor, “Elementary Numerical Analysis: An Algorithmic Approach”, Second Edition, McGraw Hill, 1972.
Eugene Isaacson and Herbert Keller, “Analysis of Numerical Methods”, John Wiley, 1966.
Kaiser Kunz, “Numerical Analysis”, McGraw Hill, 1957.
F. B. Hildebrand,
“Introduction to Numerical Analysis”, McGraw
Hill, 1956 (Dover Reprint 2nd ed, 1987).
Recommended Academic Numerical Methods Websites:
Outstanding notes on every 400-level course.
http://www.cse.uiuc.edu/heath/scicomp/ Heath's Scientific Computing (excellent)http://www.math.nyu.edu/faculty/goodman/teaching/Scientific_Computing/scientific_computing.html
(fine set of notes on Scientific Computing)
Moler, Cleve. "Numerical Computing with MATLAB", SIAM, 2004. Online version (free)
http://en.wikipedia.org/wiki/Numerical_analysis The Ubiquitous Wikipedia's corner
Of the next four links, the first most closely matches our syllabus, then the second, third...
Click here for the later topics in the textbook: Partial Differential Equations, Optimization
Some preliminary material from Strang is at: http://math.mit.edu/linearalgebra/ (Tops in LA)
Some material is available at: Sci-Prog-2011.
http://my.fit.edu/~jim/classes/ Useful problem-sets in several numerical methods courses.
Use both the Heath (Fall'05 semester) and Greenbaum (Fall'07 semester) models.
http://matrixanalysis.com/DownloadChapters.html Meyer's Matrix Analysis (magnificent)
http://meyer.math.ncsu.edu/ Course material on Meyer's foundations.
http://meyer.math.ncsu.edu/Meyer/Talks/Talks_Index.html Research esp. Google Rank ...
http://www.netlib.org/templates/templates.pdf Templates to solve of linear systems, 2nd_ed.
Iterative methods are illustrated in synoptic templates and constrasted in a concise manner.
http://web.eecs.utk.edu/~dongarra/etemplates/index.html Templates for eigenvalue algorithms.
http://dpl.ceegs.ohio-state.edu/courses/CV406/2012spring/ Applied to civil engineering,
good motivating slides and contextual perspective.
http://www.eecs.berkeley.edu/~wkahan/Math128/ Excellent articles on Numerical Analysis
http://www.cs.berkeley.edu/~demmel/ma221_Fall10/ Dummel's Num. Lin Algebra
http://www.eecs.berkeley.edu/~wkahan/MathH110/index.html Excellent articles on LA.
http://www.cs.cmu.edu/~jrs/jrspapers.html#cg Painless Conjugate Gradient paper
(also nice triangulation and mesh papers for Computer Graphics)
http://en.wikipedia.org/wiki/IEEE_754-2008 Floating Point
http://www1.maths.leeds.ac.uk/~kersale/2600/ Numerical Analysis at Leeds (concise)
http://www1.maths.leeds.ac.uk/~kersale/teaching.html Branching to PDE's (good synopsis)
http://www.ece.mcmaster.ca/~reilly/3sk4_01/outline.htm CAE perspective on Num. Meth.
http://www.public.asu.edu/~hhuang38/MAE384.html (Num. Meth. for Engineering)
http://www.as.ysu.edu/~faires/Numerical-Analysis/Programs/ Burden & Faires (9th ed) code
MATLAB References, Tutorials ...
http://www.mathworks.com/moler/chapters.html (Best book by MATLAB's creator)
Moler, Cleve. "Numerical Computing with MATLAB", SIAM, 2004.
http://www.mathworks.com/academia/ has numerous tutorials on MATLAB and its
many toolboxes. The site has a bountiful cornucopia of links for further inquiry & learning.
http://www.mathworks.com/academia/student_version/start.html 12 Introductory Videos
http://www.cs.berkeley.edu/~clancy/sp.study.guides/9A.sg.pdf MATLAB Study Guide
http://www.math.ucsd.edu/~bdriver/21d-s99/matlab-primer.html Fine Primer
http://www.che.utah.edu/~sutherland/wiki/index.php/Main_Page Great MATLAB Wiki
http://www.indiana.edu/~statmath/math/matlab/ (several tutorials)
Resources for Development (when we are not using C++ or general programming IDE's)
Numerical Methods packages and libraries with development environments which we will use
include (sorted by degree of use and expected preference):
1. MATLAB (in our labs), used in science and engineering industries, the commercial package
with powerful/fast routines, graphics, and toolboxes (http://www.mathworks.com).
2. Microsoft Excel for rapid proto-typing.
3. FREEMAT (free, like MATLAB), downloadable from http://freemat.sourceforge.net/,
4. Maxima (free Macsyma variant, the great original; http://maxima.sourceforge.net/ ),
5. GNU Octave (like MATLAB also) download from http://www.gnu.org/software/octave/,
6. SciLab (Scientific Software Package - open source) http://www.scilab.org/
Download from: http://www.scilab.org/products/scilab/download.
Tutorials at: http://hkumath.hku.hk/~nkt/Scilab/IntroToScilab.html and
0. The great treasure trove of public domain numerical algorithms is http://www.netlib.org/.
Classic libraries include IMSL and NAG; specialized ones are LA-, QUAD-, & EIS-PACK.
Popular alternatives to MATLAB you will encounter elsewhere are Mathematica
(in the Math lab, strong on symbolic processing, graphics, multi-lingual programmabilty,
with the enriching heritage of the Wolfram style and phlosophy :), MathCAD, and Maple.
For further reference on other numerical methods software, consult:
For ODE solving, we will contrast with:
Try out: http://www.berkeleymadonna.com/. Also read: http://www.jirka.org/diffyqs/.
Lastly, visit: http://www.synechism.org/wp/difference-equations-to-differential-equations/
Support Tools (for Lab and Home): (Free Visual Compilers for Windows (dot.Net))
Microsoft Visual Studio (including Visual C++) version 6 upto (dot.Net, 2010)
are available for FREE to all students taking CS courses at WPUNJ.
Just fill out form when distributed in class or go to:
MS will provide disk images to burn the full Visual Studio dot.Net / 2010 Professional.
Also available are several other MS software packages including Windows XP, Vista,
Windows 7, Virtual PC ... for student owned PC's.
Visual Studio 2008, 2010, & 2012RC Express Editions (has MS Visual C/C++)
(For general public free versions of Microsoft Visual C/C++, just click "Downloads" or DVD)
Helpful Visual C++ sites are:
Microsoft Visual C++.NET Tutorial
http://en.wikipedia.org/wiki/Microsoft_Visual_Studio (Wikipedia, the free encyclopedia)
Microsoft's own Visual Studio 2010 C++ Tutorials, Prog. Guide, & Ref. (indexed forward)
More Compilers and IDE's for C++: (Free
Compilers (Need IDE to implement ideas :))
Course Description:An introduction course in numerical methods, theory and application. Emphasizes building algorithms for solution of numerical problems, the sensitivity of these algorithms to numerical errors and efficiency of these algorithm. Topics include: solutions to non-linear equations; system of linear equations, interpolation, polynomial approximations, and quadrature solutions; numerical differentiations and integrations, eigenvalues and eigenvectors.
Besides hand-computation, we will be programming in C++/Java, MATLAB, and several other packages (cited above) as well as prototyping in Excel.
Upon completion of the course, students will be able to:
1. Acquire basic knowledge in theory and application of numerical methods used to solve physical problems. Measure: exams, surveys, and projects.
2. Gain advanced programming skills in the appropriate programming language(s). Measure: exams and projects.
3. Enhance their ability in analysis and construct of algorithms related to numerical methods. Measure: exams and projects.
4. Improve analytical and intellectual thought process in problem solving. Measure: exams and projects.
5. Strengthen their ability to present material both in oral and written form via homework and project participation. Measure: exams, homework and projects.
Measure: exams, homework and
|8.||Topical Outline of the
course content (tentatively each week = 1 chapter):
Review Syllabus, Algebraic Review, Floating Point & IEE standards, Errors, & Notation.
Disasters in seemingly trivial computation:
Case study #1 "Computers cannot store numbers."
Decimal 0.1 has no binary computer representation
Case study #2 "Infinite errors occur in simple sums."
Harmonic series (Sigma 1/k) is infinite, yet all computer sums "converge" to a finite answer.
Case study #3 "Even tiny errors cause disasters."
Linear equations with miniscule errors intersect erroneously
Conjuncting the above 3 premises, we conclude: "Numerical Methods is Hopeless!"
Not quite!!! Therefore, Let's begin :)
Appendix A: Matrix Algebra
A.1 Matrix fundamentals
A.2 Block multiplication
A.3 Eigenvalues and eigenvectors
A.4 Symmetric matrices
A.5 Vector calculus
Appendix B: Introduction to MATLAB (Hands-on lab session)
B.1 Starting MATLAB
B.2 MATLAB graphics
B.3 Programming in MATLAB
B.4 Flow control
B.6 Matrix operations
Chapter 0. Fundamentals
0.1 Evaluating a polynomial
0.2 Binary numbers (Decimal to binary and Binary to decimal)
0.3 Floating point representation of real numbers
0.3.1 Floating point formats
0.3.2 Machine representation
0.3.3 Addition of floating point numbers
0.4 Loss of significance
0.5 Review of calculus
Chapter 1. Solving Equations
1.1 The Bisection Method (Bracketing a root. How accurate and how fast?)
1.2 Fixed point iteration (Fixed points of a function, their geometry, convergence, & iteration)
1.3 Limits of accuracy (Forward & backward error, Wilkinson polynomial, sensitivity, error)
1.4 Newton’s Method (Quadratic versus linear convergence of Newton's method)
1.5 Root-finding without derivatives (Secant method and variants; Brent's Method)
REALITY CHECK 1: Kinematics of the Stewart platform
Chapter 2. Systems of Equations
2.1 Gaussian elimination (Naive Gaussian elimination, Operation counts)
2.2 The LU factorization (Backsolving with and Complexity of the LU factorization)
2.3 Sources of error (Error magnification and condition number, Swamping)
2.4 The PA=LU factorization (Partial pivoting, Permutation matrices, PA = LU factorization)
REALITY CHECK 2: The Euler-Bernoulli Beam
2.5 Iterative methods
2.5.1 Jacobi Method
2.5.2 Gauss-Seidel Method and SOR
2.5.3 Convergence of iterative methods
2.5.4 Sparse matrix computations
2.6 Methods for symmetric positive-definite matrices
2.6.1 Symmetric positive-definite matrices
2.6.2 Cholesky factorization
2.6.3 Conjugate Gradient Method
2.7 Nonlinear systems of equations
2.7.1 Multivariate Newton's method
2.7.2 Broyden's method
Chapter 3. Interpolation
3.1 Data and interpolating functions
3.1.1 Lagrange interpolation
3.1.2 Newton's divided differences
3.1.3 How many degree d polynomials pass through n points?
3.1.4 Code for interpolation
3.1.5 Representing functions by approximating polynomials
3.2 Interpolation error (Interpolation error formula, Newton form, Runge phenomenon)
3.3 Chebyshev interpolation (Chebyshev's Theorem, Chebyshev polynomials, Interval change)
3.4 Cubic splines (Properties of splines, Endpoint conditions)
3.5 Bézier curves
REALITY CHECK 3: Constructing fonts from Bézier splines
Chapter 4. Least Squares
4.1 Least squares and the normal equations
4.1.1 Inconsistent systems of equations
4.1.2 Fitting models to data
4.1.3 Linear and nonlinear models (Conditioning of least squares)
4.2 A survey of models (Periodic data, Data linearization)
4.3 QR factorization
4.3.1 Gram-Schmidt orthogonalization and least squares
4.3.2 Modified Gram-Schmidt orthogonalization
4.3.3 Householder reflectors
4.4 Generalized Minimum Residual (GMRES) Method
4.4.1 Krylov methods
4.4.2 Preconditioned GMRES
4.5 Nonlinear least squares
4.5.1 Gauss-Newton method
4.5.2 Models with nonlinear parameters
4.5.3 Levenberg-Marquardt method
REALITY CHECK 4: GPS, conditioning and nonlinear least squares
Quiz # 4
Chapter 5. Numerical Differentiation and Integration
5.1 Numerical differentiation
5.1.1 Finite difference formulas
5.1.2 Rounding error
5.1.4 Symbolic differentiation and integration
5.2 Newton-Cotes formulas for numerical integration
5.2.1 Trapezoid rule
5.2.2 Simpson's Rule
5.2.3 Composite Newton-Cotes Formulas
5.2.4 Open Newton-Cotes methods
5.3 Romberg integration
5.4 Adaptive quadrature
5.5 Gaussian quadrature
REALITY CHECK 5: Motion control in computer-aided modelling
Chapter 6. Ordinary Differential Equations
6.1 Initial value problems
6.1.1 Euler's method (skim next subchapter)
6.1.3 First-order linear equations
6.2 Analysis of IVP solvers
6.2.1 Local and global truncation error
6.2.2 The explicit trapezoid method
6.2.3 Taylor methods
6.3 Systems of ordinary differential equations
6.3.1 Higher order equations
6.3.2 Computer simulation: The pendulum
6.3.3 Computer simulation: Orbital mechanics
6.4 Runge-Kutta methods and applications
6.4.1 The Runge-Kutta family
6.4.2 Computer simulation: The Hodgkin-Huxley neuron
6.4.3 Computer simulation: The Lorenz equations
REALITY CHECK 6: The Tacoma Narrows Bridge
6.5 Variable step-size methods
6.5.1 Embedded Runge-Kutta pairs
6.5.2 Order 4/5 methods
6.6 Implicit methods and stiff equations
6.7 Multistep methods
6.7.1 Generating multistep methods
6.7.2 Explicit multistep methods
6.7.3 Implicit multistep methods
Chapter 7. Boundary Value Problems (Optional)
7.1 Shooting method
7.1.1 Solutions of boundary value problems
7.1.2 Shooting method implementation
REALITY CHECK 7: Buckling of a circular ring
7.2 Finite difference methods
7.2.1 Linear boundary value problems
7.2.2 Nonlinear boundary value problems
7.3 Collocation and the Finite Element Method
7.3.2 Finite elements and the Galerkin method
Chapter 8. Partial Differential Equations (Optional)
8.1 Parabolic equations (Forward difference, Backward difference, & Crank-Nicolson methods)
8.2 Hyperbolic equations (The wave equation, The CFL condition)
8.3 Elliptic equations ( Finite difference and finite element methods for elliptic equations)
REALITY CHECK 8: Heat distribution on a cooling fin
8.4 Nonlinear partial differential equations (Implicit Newton solver, Nonlinearity in 2D)
Chapter 9. Random Numbers and Applications
9.1 Random numbers (Pseudo-random numbers, Exponential and normal random numbers)
9.2 Monte-Carlo simulation (Power laws for Monte Carlo estimation, Quasi-random numbers)
9.3 Discrete and continuous Brownian motion (Random walks, Brownian motion)
9.4 Stochastic differential equations (Adding noise to diff. eq., Numerical methods for SDEs)
REALITY CHECK 9: The Black-Scholes formula
Chapter 10. Trigonometric Interpolation and the FFT
10.1 The Fourier Transform (Complex arithmetic, Discrete Fourier Transform, FFT)
10.2 Trigonometric interpolation (DFT Interpolation Theorem, fast trigonometric computation)
10.3 The FFT and signal processing
10.3.1 Orthogonality and interpolation
10.3.2 Least squares fitting with trigonometric functions
10.3.3 Sound, noise, and filtering
REALITY CHECK 10: The Wiener filter
Chapter 11. Compression
11.1 The Discrete Cosine Transform
11.1.1 One-dimensional DCT
11.1.2 The DCT and least squares approximation
11.2 Two-dimensional DCT and image compression
11.2.1 Two-dimensional DCT
11.2.2 Image compression
11.3 Huffman coding
11.3.1 Information theory and coding
11.3.2 Huffman coding for the JPEG format
11.4 Modified DCT and audio compression
11.4.1 Modified Discrete Cosine Transform
11.4.2 Bit quantization
REALITY CHECK 11: A simple audio codec using the MDCT
Chapter 12. Eigenvalues and Singular Values
12.1 Power iteration methods
12.1.1 Power iteration
12.1.2 Convergence of power iteration
12.1.3 Inverse power iteration
12.1.4 Rayleigh quotient iteration
12.2 QR algorithm
12.2.1 Simultaneous iteration
12.2.2 Real Schur form and QR
12.2.3 Upper Hessenberg form
REALITY CHECK 12: How search engines (like Google) rate page quality
12.3 Singular value decomposition
12.3.1 Finding the SVD in general
12.3.2 Special case: symmetric matrices
12.4 Applications of the SVD
12.4.1 Properties of the SVD
12.4.2 Dimension reduction
12.4.4 Calculating the SVD
Chapter 13. Optimization (Optional)
13.1 Unconstrained optimization without derivatives
13.1.1 Golden section search
13.1.2 Successive parabolic interpolation
13.1.3 Nelder-Mead search
13.2 Unconstrained optimization with derivatives
13.2.1 Newton's method
13.2.2 Steepest descent
13.2.3 Conjugate gradient search
13.2.4 Nonlinear least squares
REALITY CHECK 13: Molecular conformation and numerical optimization
(e.g., lecture, discussions, presentations,
a) Classroom lectures, discussions, and problem solving sessions
b) Homework reviews
c) Lab work programming with using compilers, IDE's, & profilers.
a. Reading Assignments
b. Tentative timeline for submission of written
e. Examinations (tentative dates, make-up policy, etc.)
All exams will be announced at least one full week in advance.
If you are absent on the day an exam is announced, you are responsible for finding out
about it from a fellow student or the professor. No make-up exams will be given except for
Item 8 (above) addresses the examination schedule issue.
Refer to: http://www.wpunj.edu/registrar/finalexamcalendars/final-exam-spring.dot
Final Exam. Period: Thursday 5/9/2013 at 2:00PM-4:30PM, Science Hall East 5019
f. Class participation
and other methods
for assessing student academic
|12.||Additional information (Calendar):
Last day for course withdrawal is 3/5/2013.
Classes will be in session from 1/14/2013 to 5/2/2013, followed by finals week 5/3/13-5/10/13.
(refer to: http://www.wpunj.edu/registrar/calendars/spring.dot ).
Holidays: 2/18/13 (Presidents Day) and 2/21/13 Thursday is a Monday schedule
3/17/13-3/23/13 (Spring Break) and 3/29/13 (Good Friday).