CHAPTER 06.04: NONLINEAR REGRESSION: Exponential Model: Derivation: Part 2 of 2

So let's go ahead and take the derivative of the sum of the square of the residuals with respect to a, and what am I going to get?  I'm going to summation, i is equal to 1 to n, 2 times y i, minus a e to the power b xi, times then the derivative of this expression here with respect to a, which will be just e to the power b xi, and that I'm putting equal to 0.  And then I have the derivative with respect to be, which will be summation of, i is equal to 1 to n, again, 2 times yi, minus a e to the power b xi, and then I'll have to take the derivative of this quantity here with respect to be, and I'll get 0 here, but here I'll get minus a xi e to the power b xi equal to 0, because I'll have the a from here, and then the derivative of e to the power b xi with respect to b, not with respect to x, guys, but with respect to b is xi times e to the power b xi, and it's equal to 0.  So now I've got to do some expansion and things like that, so let me look at this first equation which I have. What am I going to get?  I'm going to get summation, 2, if I expand this, I'm going to get 2 summation, i is equal to 1 to n, yi e to the power b xi, that's what I'm going to get from there, because I'll be able to combine this term and this term, then -2, a is now a constant, because I'm trying to find out when does the . . . when do the derivatives, partial derivatives with respect to a and b become 0, so I can take the a to be outside, then summation, i is equal to 1 to n, and I have e to the power b xi here and e to the power b xi here, and that gives me e to the power b xi equal to 0, bases are same, powers are added, that's where I get that from. Now I take this second equation which I have here, so I get 2 times summation, i is equal to 1 to n, I get minus a here, so I can take minus a outside, so I get -2 a here, then I will have yi xi e to the power b xi, so that's what I'm going to get from here.  So -2 a is outside, and then yi xi e to the power b xi is there, then plus, what do I get?  Plus, this is minus, minus, it becomes plus, a is here, a is here, so that will be a squared, the outside, and then 2 is from here, and then what'll be inside will be xi e to the power 2 b xi equal to 0, because I'll have e to the power bxi here, e to the power b xi here, and xi here, so I'll get xi e to the power 2 bxi, again, bases are same, powers are added, that's what I'm going to get there.  So let's go ahead and rewrite this by simply saying, hey, this 2 and 2 cancels, or I can divide by 2 both sides, let's put it that way, I can divide 2 by both sides, so I don't need that.  I can also divided both sides by a.  So I can divide both sides by a like this here, and the reason why I can do that is because if . . . a has to be not equal to 0, because if a is equal to 0, then I have y equal to 0 as my regression curve, so I know that I cannot choose a equal to 0, so I can take out one of these as with one of these as right there.  So if I write it again, this is what I will get.  So the first equation which I will get will be summation, i is equal to 1 to n, yi e to the power b xi, minus a summation, i is equal to 1 to n, e to the power 2 b xi equal to 0.  And the second equation will turn out to be summation, i is equal to 1 to n, with a minus here, yi xi e to the power b xi, plus a summation, i is equal to 1 to n, xi e to the power 2 b xi equal to 0.  So this is after some simplification, this is what I am able to get.  Now the thing which you have to now appreciate is that you have two equations, two unknowns, all the ys and xs are known, n is known, because that's the number of data points.  The only thing which is not known here is the value of b and a, but you can very well see that you are basically getting two simultaneous nonlinear equations.  You're not getting two simultaneous linear equations just like we get in linear regression, but here we're getting two simultaneous nonlinear equations in terms of a an b.  So not only do we have two nonlinear equations, we also have to figure out a way to find now a solution for a and b, and also to find the right solution, because when you have two simultaneous nonlinear equations, there are several possibilities.  You may get no solution, you may get a unique solution, you might get a finite number of solutions, you might get infinite number of solutions.  So there's a lot riding on how you interpret, or how you numerically solve these two equations and two unknowns. Now, the only good news here is that if you look at this particular equation here, the first equation right here, what you will find out is that you can write down a explicitly in terms of b, because a is only here.  So since a is only here in the first equation, I can write down a explicitly in terms of b, and I can substitute then that value of a back in here, which will allow me to then solve only one equation and one unknown at a time.  So that's what I'm going to do, so if I take the first equation, I'll be able to say that a is summation, i is equal to 1 to n, yi e to the power b xi, divided by summation, i is equal to 1 to n, e to the power 2 b xi. So that's the expression coming from equation 1, where I'm able to write down a explicitly in terms of b. The only reason why I'm doing that is because then it allows me to now substitute now this value of a into this equation, and it allows me to then solve for b only.  So if I look at this equation now, so substitute a in equation 2, I get minus, so my first term does not change, or my first part does not change, it is this, and what I'm going to do is for a I'm going to substitute this formula here, so that's plus summation, i is equal to 1 to n, yi e to the power b xi, divided by summation, i is equal to 1 to n, e to the power 2 b xi, summation, xi e to the power 2 b xi equal to 0. So this is what you are getting as your expression for b now, because in this case, for this nonlinear regression model, you are able to write down a in terms of b, and since you are able to write down a in terms b, you are able to substitute in here, and now if you look at this, this is a single nonlinear equation now.  It might look complicated, but you've got to understand this is nothing but an equation which looks like this, function of b is equal to 0.  You are seeking a value of b so that this whole term on the left-hand side is 0, and once you have been able to do that, you'll be able to find out what b is.  Once you have been able to find b, you substitute it back into this particular equation, and you are able to find out what the value of a is.  So that's how this scheme is going to work when you want to find out what the values of a and b, the constants of the regression model are when you're going to solve a problem which has an exponential regression model. Now, somebody might say, hey, how do I go about solving a nonlinear equation like this? The only thing is that what you have to do is in order to be able to generate this function f, you may have to use certain loops to be able to calculate these summations, and you know that all the nonlinear equation numerical methods are based on putting in some value of b in order to calculate the value of the root of this equation here. So let's suppose if you are going to use bisection method, then you'll be using an upper and lower estimate of b, plugging it in here, and finding whether this function is changing sign, things like that.  So it is not very complicated so far as the implementation is concerned, it might look complicated so far as the formula is concerned, but it still is just simply a function of be, so you want to find out where does b become 0.  Once you have found out where b becomes 0 by a numerical method, you plug it back in here, and you get the value of a, and that's how you find out the constants of the exponential regression model.  And that's the end of this segment.