| Re: Question about CURVEFIT function [message #85858 is a reply to message #85740] |
Fri, 13 September 2013 22:57  |
Craig Markwardt
Messages: 1869
Registered: November 1996
|
Senior Member |
|
|
On Friday, August 30, 2013 4:17:59 AM UTC-4, fd_...@mail.com wrote:
>> "This method is the Gradient-expansion algorithm which combines the best features of the gradient search with the method of linearizing the fitting function."
>
>
>
> I still don't understand clearly how the CURVEFIT function works. The method of linearizing the fitting algorithm is the Levenberg-Marquart?
Maria wrote to me and asked this question privately. Here is what I wrote...
The CURVEFIT algorithm is copied from Bevington's "Data Reduction and Error Analysis for the Physical Sciences." Bevington is a very good book. Also, Numerical Recipes has a very good discussion of how this works.
The goal is to find the minimum of a function, in this case, chi-square. You can think of the function to be optimized as a valley in parameter space and the goal is to find the bottom of the valley (coordinates of lowest value of function). At a basic level, one might consider two ways to find a minimum of a function, steepest descent or Newton's method.
One way is to find the direction of steepest descent and follow that. You just calculate the gradient direction of the function and take your next step in that direction. This is good when the gradient is steep, but bad when you are near the bottom of the valley where the function is more rounded and less steep.
http://en.wikipedia.org/wiki/Gradient_descent
Another way is to use Newton's method. This method basically treats the shape of the valley as a paraboloid. You can know the shape of a paraboloid by measuring the derivatives at one position, and use those to solve for the position of the most extreme value (minimum function value). This type of method works well near the bottom of the valley where the shape of the function is more rounded but does not work well when the gradient is steep.
http://en.wikipedia.org/wiki/Newton%27s_method_in_optimizati on
Levenberg and Marquardt came up with a scheme to mix both kinds of solutions. When you are far from the bottom of the valley, you use the steepest gradient method. When you are close to the bottom, you use the Newton's method. This algorithm has the benefits of both techniques.
http://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_alg orithm
CURVEFIT uses Levenberg-Marquardt algorithm. It is concerned with calculating the gradients (derivatives) of the function, and then combining those to estimate the direction of steepest descent and Newton minimum. It forms a new trial solution, and starts a new iteration to try again. When it finds a minimum value to within the desired tolerance (CURVEFIT's TOL keyword), it reports that solution.
The MPFIT family of functions are also advanced Levenberg-Marquardt-style algorithms. The internals are also more robust to round-off errors and degenerate parameters.
Craig Markwardt
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
