In numerical analysis, the secant method is a root-finding algorithm that uses a succession of roots of secant lines to better approximate a root of a function f. The secant method can be thought of as a finite-difference approximation of Newton's method. However, the method was developed independently of Newton's method and predates it by over 3000 years.

Drawbacks of Newton's method

Newton's method is widely used in numerical analysis, but it has some drawbacks: it requires a derivative for it to work, and for some functions like f(x) = sin(cos(e^x)) are complicated, like in this one, f'(x) = -e^x sin(e^x) cos(cos(e^x)), where it is much longer than the original function. Therefore, if you don't want to use a derivative to approximate roots but still want to use a method that's faster than Bisection, that's when the secant method comes into play. This way, it uses a secant of two points instead of a tangent.

History

The secant method was created over 3000 years before Newton's method. A special case of this method was first called the rule of double false position in 18th century B.C. Babylonian clay tablets. It is now thought of as a primitive version because it is essentially the secant method applied to a linear equation. Because algebra hadn't been developed by that time, Babylonians had described their problems in a written form rather than symbolically. This method was developed as a result of common problems they encountered such as the sale and distribution of properties, inheritance, or for the purpose of portion control and the prediction of production. In 1545 the rule of double false position was first used as an iterative procedure by Cardano in his Artis Magnae (Papakonstantinou 2009). Thus, we have the secant method for a nonlinear equation.

The method

The secant method is defined by the recurrence relation xn = xn-1 - f(xn-1) * ((xn-1 - xn-2)/(f(xn-1) - f(xn-2))) = ((xn-2*f(xn-1) - xn-1*f(xn-2))/(f(xn-1) - f(xn-2))). As can be seen from the recurrence relation, the secant method requires two initial values, x0 and x1, which should ideally be chosen to lie close to the root.

Derivation of the method

Starting with initial values x0 and x1, we construct a line through the points (x0, f(x0)) and (x1, f(x1)), as shown in the picture above. In slope–intercept form, the equation of this line is y = ((f(x1) - f(x0))/(x1 - x0))*(x - x1) + f(x1). The root of this linear function, that is the value of x such that y = 0 is x = x1 - f(x1)*((x1 - x0)/(f(x1) - f(x0))). We then use this new value of x as x2 and repeat the process, using x1 and x2 instead of x0 and x1. We continue this process, solving for x3, x4, etc., until we reach a sufficiently high level of precision (a sufficiently small difference between xn and xn−1): x2 = x1 - f(x1)*((x1 - x0)/(f(x1) - f(x0))), x3 = x2 - f(x1)*((x2 - x1)/(f(x2) - f(x1))), ... , xn = xn-1 - f(xn-1)*((xn-1 - xn-2)/(f(xn-1) - f(xn-2))).

Convergence

The iterates xn of the secant method converge to a root of f if the initial values x0 and x1 are sufficiently close to the root. The order of convergence is φ, where φ = (1 + sqrt(5))/2 ≃ 1.618 is the golden ratio. In particular, the convergence is superlinear, but not quite quadratic (rather subquadratic).

This result only holds under some technical conditions, namely that f be twice continuously differentiable and the root in question be simple (i.e., with multiplicity 1).

If the initial values are not close enough to the root, then there is no guarantee that the secant method converges. There is no general definition of "close enough", but the criterion has to do with how "wiggly" the function is on the interval [x0, x1]. For example, if f is differentiable on that interval and there is a point where f' = 0 on the interval, then the algorithm may not converge.

Comparison with other root-finding methods

The secant method does not require that the root remain bracketed, like the bisection method does, and hence it does not always converge. The false position method (or regula falsi) uses the same formula as the secant method. However, it does not apply the formula on xn-1 and xn-2, like the secant method, but on xn-1 and on the last iterate xk such that f(xk) and f(xn-1) have a different sign. This means that the false position method always converges. The recurrence formula of the secant method can be derived from the formula for Newton's method xn = xn-1 - f(xn-1)/f'(xn-1) by using the finite-difference approximation f'(xn-1) ≃ (f(xn-1)-f(xn-2))/(xn-1-xn-2)). The secant method can be interpreted as a method in which the derivative is replaced by an approximation and is thus a quasi-Newton method. If we compare Newton's method with the secant method, we see that Newton's method converges faster (order 2 against φ ≈ 1.6). However, Newton's method requires the evaluation of both f and its derivative f' at every step, while the secant method only requires the evaluation of f. Therefore, the secant method may occasionally be faster in practice. For instance, if we assume that evaluating f takes as much time as evaluating its derivative and we neglect all other costs, we can do two steps of the secant method (decreasing the logarithm of the error by a factor φ^2 ≈ 2.6) for the same cost as one step of Newton's method (decreasing the logarithm of the error by a factor 2), so the secant method is faster. If, however, we consider parallel processing for the evaluation of the derivative, Newton's method proves its worth, being faster in time, though still spending more steps.

Generalizations

Broyden's method is a generalization of the secant method to more than one dimension.

The following graph shows the function f in red and the last secant line in bold blue. In the graph, the x intercept of the secant line seems to be a good approximation of the root of f.

Example of finding the root of the function represented by the red line (cos(x) − x^3) using the secant method, with the increasingly accurate approximations represented by the blue straight lines.

Fractals

Secant for f(z) = z^3 - 1

Secant for f(z) = cos(z)

The slowness of convergence of this method makes the fractals that emerge from it much more interesting than Newton or Halley. In addition, the secant method needs two initial guesses, which introduces another set of options that can drastically affect how the resulting fractal looks. As far as can be known, the first publication of the secant method for making fractals was in a FractInt formula file called SECANT.FRM by Mike Wareman. That file references Szyszkowicz M, "Computer art generated by the method of secants in the complex plane." COMPUTERS AND GRAPHICS, volume 14, number 3/4, p. 509 (1990). The file can be found in several collections of FractInt formula files, but it doesn't seem to be part of what you get when you download FractInt itself.

If you're an UltraFractal user, you can find the secant method realized in at least two places:

1. mt.ufm by Mark Townsend, dated 16 Jul 1999
2. pwc_convert.ufm by Paul Carlson, dated 19 Jul 1999.

Notes

• Can diverge or divide by zero - switching to bisection and inverse quadratic interpolation guarantees convergence (Brent-Dekker method)
• Superlinear and subquadratic convergence
• Generates more interesting fractals than Halley or Newton
• One of the oldest numerical methods known

Sources: Wikipedia, https://sites.google.com/site/knowyourrootsmaxima, and http://www.hpdz.net/TechInfo/Convergent.htm#SecantMethod