CHAPTER 03.04: NEWTON-RAPHSON METHOD: Derivation from Taylor Series of Newton-Raphson Method

In this segment, we're going to derive Newton-Raphson method of solving nonlinear equations by using Taylor series. Now, we know that Taylor series is all about that we can . . . I can give you the value of the function at a point, at some other point, which is h away from the point where you are if you can give me the value of the function at that particular point, you can give me the value of the first derivative of the function at that particular point, the value of the second derivative of the function at that particular point, the value of the third derivative of the function at that particular point, and so on and so forth.  So if somebody's able to give you the value of the function at some point, the value of its derivative at that particular point, the second derivative at that point, third derivative at that point, and then tells you, hey, I have another point which is h away from the point where you are giving me all this information, then I can give you the value of the function exactly at that particular point, provided you give me all the derivatives which I need, because I need infinite number of derivatives here, and also that the value of the function is defined and continuous between x and x plus h, and all its derivatives are also defined and continuous from that point.

Now, what does this have to do with Newton-Raphson method formula?  It's that we can derive Newton-Raphson method formula by using the Taylor series.  So let's suppose I'm going to put x equal to xi, and I'm going to say x-sub-i-plus-1 minus xi is equal to h.  What I mean by saying that is I'm here at xi and I'm going to go to x-sub-i-plus-1, and this distance here is h.  So in the formula which I have just written, this is x, and that's x plus h, I'm just using new terminology, that's all I'm doing, I'm going to say, hey, this is my xi and this is my x-sub-i-plus-1.  So if I rewrite the Taylor's Theorem here, now in terms of xi and x-sub-i-plus-1, that's . . . this is what I'm going to get, I'm going to get that, hey, I can give you the value of the function at x-sub-i-plus-1 if you can give me the value of the function at xi, plus f prime of xi, but h is . . . I'm going to, for h I'm going to substitute the difference between this point and this point, because that's what h is, the difference between the point where you know the information to the place where you want, to the point where you want to go, so I'm going to just put x-sub-i-plus-1 minus xi, and then I'll have plus f double-prime of xi divided by factorial 2, times x-sub-i-plus-1 minus xi, squared.  I'm just, again, substituting for h, I'm substituting the difference between the point where I'm going to from the point where I know the information, and so on and so forth, so I'm just not going to write the other terms, but there are infinite terms in this particular series. 

But what is Newton-Raphson method formula used for?  It is used for finding the root of the equation f of x equal to 0, so you're basically looking for that value of x where the value of the function becomes 0.  So you're basically looking for the point x-sub-i-plus-1 where this function becomes 0. So because you're looking for the point where this function is becoming 0, so I'm going to say, hey, I'm looking for the point where this becomes 0.  So 0 will be equal to f of xi, plus f prime of xi times x-sub-i-plus-1 minus xi, plus f double-prime of xi divided by factorial 2, times x-sub-i-plus-1 minus xi, squared, and so on and so forth, there are infinite number of terms there.  So if I take only the first two terms of the series here, if I only take the first two terms of the series here, and I say, hey, 0 is approximately proportional to f of xi plus f prime of xi times x-sub-i-plus-1 minus xi.  What I'm going to get is that I'm going to get the Newton-Raphson method formula from there, so if I have this, I'm taking only the first two terms of the Taylor series, I'm going to get this, x-sub-i-plus-1 is equal to xi minus the value of the function at xi divided by f prime of xi. And that's your Taylor series, sorry, that's your Newton-Raphson method formula being derived from Taylor series.  Now, why is it important to show it here? The reason why it's important to show you here is that first it tells you that, hey, these are the terms which you are forgetting about when you are deriving your Newton-Raphson method from Taylor series.  So, the kind of terms which you are forgetting about, or not accounting for are the terms which are quadratically . . . that the error is going to be quadratic, because this is the amount of error which is going to be caused, this is the term which is going to overwhelm the error quantity here, so that's why they say Newton-Raphson method converges quadratically, because the term, the first term which you are neglecting in deriving the Taylor series . . . deriving the Newton-Raphson method formula here is this term right here.  This also tells you that, hey, if the function which is in this equation f of x equal to 0, if it were a straight line, then you will have no error, because if . . . so let's look at that.  So if . . . so if you have f of x equal to 0, and somebody says, hey, if f of x is a linear polynomial, or a straight line, if it's a linear polynomial, then you know that, hey, the second derivative is 0, the third derivative is 0, and so on and so forth, all the derivatives above the second and . . . above the first derivative, which is second derivative, third derivative, and all the other derivatives are 0, in that case what you're going to find out is that the error in Newton-Raphson method is zero, because if you look at the Taylor series formula, or the Taylor's Theorem, you're finding out that the error has the term of the second derivative, third derivative, fourth derivative, and if they're all 0, because you chose f of x to be a linear polynomial, then you're going to get zero error in the Newton-Raphson method, which makes sense, because if you have a function like this, so if you have a straight line, which is same as a linear polynomial, so if your function in the equation f of x equal to 0 is a straight line, then if I start from an initial guess here, anywhere, it doesn't matter whether I'm close to the root or not, the tangent which I'm going to draw will exactly coincide with the function itself, and within the first iteration I will get my root of the equation f of x equal to 0.  So it makes sense from a graphical point of view also, but also from the Taylor's Theorem, you are able to judge that if your function f of x were a straight line in the equation f of x equal to 0, you need only one iteration to be able to get your root of the equation, and that's because all these derivatives, the second derivative, third derivative, fourth derivative, and all the other derivatives are 0. And that's the end of this segment.