CHAPTER 06.04: NONLINEAR REGRESSION: Exponential Model: Derivation: Part 1 of 2

In this segment, we're going to talk about exponential model regression.  So somebody gives us data points, and wants us to do regression to an exponential model.  So we want to be able to see that how do we go about doing that.  So the problem statement is as follows, somebody says, hey, given n data points, like x1, y1, all the way up to xn, yn, and says best fit y is equal to a e to the power b x to the data.  So when we're talking about best fit, we're talking about in the least squares sense, that you want to best fit, in the least squares sense, this particular exponential model to this data . . . to these data points.  So, for example, if somebody gives you y as a function of x, so let's suppose y is given as a function of x, like this, and this is x1, comma, y1, let's suppose, and this is xn, comma, yn, keep in mind that these data points don't necessarily have to be given in and kind of ascending or descending order.  So then you are trying to draw an exponential curve which is going through those data points . . . not through those data points, but best fits those data points.  So y is equal to a e to the power b x is your exponential model.  So, again, the constants of the exponential model then are a and b, and that's what we want to be able to find.  So if we are able to find out what the value of a and b are, then we can predict the value of y at any value of x which you might be interested in.  So let's go ahead and see that how will we be able to go about finding the value of a and b, if we are trying to regress this particular exponential model to these data point which are given to us. So if we go back to the concept of the sum of . . . least squares, we have to first calculate what the residual is.  The residual is simply the difference between the observed value and the predicted value. So that's what the residual is, because if you are looking at a point xi, for example, the actual value, the experimental value, or the given value to you is yi, but when you draw this exponential model as the best fit for the data, then the residual between what is observed and what is predicted is as . . . is the difference between the two.  So this is your observed value, and this is your predicted value. So the difference between the two is the residual there, but the least squares basically say that, hey, go ahead and sum the square of all the errors, or the residuals.  So you take the residual at any point, i, for example, you square it, and then you add it based on how many data points are given to you, in this case being n, so that turns out to be the sum of the square of the residuals is equal to Ei squared. So if I write it down again, I get summation, i is equal to 1 to n, yi, minus a e to the power b xi, squared. So that's what I get as my sum of the square of the residuals. Now, in order to be able to find out what the constants of the model are, so if you look at this particular expression here, what we want to do is we want to make this sum of the square of the residuals to be as small as possible, we want to minimize it.  So we want to minimize it. So in order to be able to minimize it, we will need to find out when does this become as small as possible, and the only things which we can change are a and b.  a and b are the variables now for minimizing Sr, because yis and xis are fixed, they are given to us, they are experimental, or whatever data is given to us. So we can only minimize with respect to a and b, so that means that we'll have to take the partial derivatives, because we have two variables now, a and b, we'll have to take the partial derivative of the sum of the square of the residuals with respect to a and b, and put that equal to 0 to be able to possibly find out what the minimum of this function is, because just by taking the first derivative and put them equal to 0 tells you it's a max or a min.  So let's first go ahead and take the first derivative and put those equal to 0.  So I'm going to have . . . I'm going to take the derivative, partial derivative of the sum of the sum of the square of the residuals with respect to a, put that equal to 0, and I'm going to take the sum of the square of the residuals with respect to b and put that equal to 0.  And when I put these two equal to 0, I'm going to get two equations and two unknowns, for a and b, and now we'll be able to solve for a and b, and that's the whole point about finding out what are the values of a and b which makes this to be a minimum.  We're not going to go into the proof to show that it corresponds to a minimum, because by taking these derivatives equal to 0, and this derivative equal to 0, you're only finding the local max or min, it's a little bit more complicated of a proof to show that it corresponds to a minimum, but let's concentrate our attention on trying to find out the partial derivative Sr with respect to a, the partial derivative with respect to b, and put that equal to 0, and set up these two equations, two unknowns.