4.9 Newton’s Method

Newton's Method

Newton's Method is an iterative algorithm for finding roots (zeros) of equations that can't be solved algebraically. It starts with an initial guess and uses the derivative to refine that guess repeatedly, producing increasingly accurate approximations of the root.

This method is especially useful for nonlinear equations like $e^x - x^2 = 0$, where no algebraic technique will give you an exact answer. It shows up constantly in applied math, physics, and engineering.

Concept and Purpose

A root of a function $f(x)$ is a value of $x$ where $f(x) = 0$. For example, $x^2 - 4 = 0$ has roots at $x = \pm 2$.

Newton's Method finds roots by using linear approximation. The core idea: at your current guess $x_n$, draw the tangent line to $f(x)$. Where that tangent line crosses the x-axis becomes your next guess, $x_{n+1}$. Each tangent line gets you closer to the actual root, as long as conditions are reasonable.

This works because near a root, the tangent line is a good local approximation of the curve. So the tangent line's zero is close to the function's zero.

The Iterative Formula

The formula that drives Newton's Method:

$x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}$

• $x_n$ is your current approximation
• $f(x_n)$ is the function's value at that point
• $f'(x_n)$ is the slope of the tangent line at that point

The fraction $\frac{f(x_n)}{f'(x_n)}$ tells you how far to shift from your current guess. If $f(x_n)$ is large (you're far from zero), you take a bigger step. If the slope $f'(x_n)$ is steep, you take a smaller step.

Applying Newton's Method Step by Step

1. Choose an initial guess $x_0$ reasonably close to the expected root (a graph helps here).

2. Compute $f(x_0)$ and $f'(x_0)$.

3. Apply the formula to get $x_1 = x_0 - \frac{f(x_0)}{f'(x_0)}$.

4. Repeat using $x_1$ to find $x_2$, then $x_2$ to find $x_3$, and so on.

5. Stop when the difference $|x_{n+1} - x_n|$ is within your desired tolerance, or after a set number of iterations.

Quick example: Find a root of $f(x) = x^2 - 5$ (so you're approximating $\sqrt{5}$).

• $f'(x) = 2x$. Start with $x_0 = 2$.
• $x_1 = 2 - \frac{(4 - 5)}{2(2)} = 2 - \frac{-1}{4} = 2.25$
• $x_2 = 2.25 - \frac{(5.0625 - 5)}{2(2.25)} = 2.25 - \frac{0.0625}{4.5} \approx 2.23611$
• The actual value of $\sqrt{5} \approx 2.23607$. After just two iterations, you're accurate to four decimal places.

Convergence and Limitations

When it works well:

• Newton's Method converges quadratically when your initial guess is close enough to the root. Quadratic convergence means the number of correct digits roughly doubles with each iteration. That's fast.

When it can fail:

• Initial guess too far from the root. The tangent line may point you away from the root you want, or toward a different root entirely.
• Derivative near zero at some iterate. If $f'(x_n) \approx 0$, the formula sends $x_{n+1}$ flying off to a huge value. Geometrically, a nearly flat tangent line crosses the x-axis very far away.
• Repeated roots. At a repeated root like $x = 0$ for $f(x) = x^3$, convergence slows down significantly (it drops from quadratic to linear).
• Oscillation. For some functions and starting points, the method can bounce back and forth without settling down.
• Non-differentiable functions. The method requires $f'(x)$ to exist. You can't directly apply it to functions like $f(x) = |x|$.

Interpreting Results

• Each iteration produces a new approximation $x_n$. The sequence should converge toward the actual root.
• Check accuracy by evaluating $f(x_n)$. The closer it is to zero, the better your approximation.
• The difference $|x_{n+1} - x_n|$ gives a rough estimate of the error, though it's not a guarantee.
• If the method oscillates or the approximations diverge, your results aren't reliable. Try a different initial guess or verify the function meets the method's requirements.

Related Methods

• The secant method is a variation that replaces the derivative with a difference quotient, using two previous points instead. Useful when computing $f'(x)$ is difficult.
• Fixed-point iteration is another root-finding approach that can work when Newton's Method isn't suitable, though it typically converges more slowly.