comp.lang.idl-pvwave archive - RDF feed
type: channel
description: Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
items: Re: Simultaneous fitting in IDL

Re: Simultaneous fitting in IDL
type: item
description: On May 14, 6:54 am, Gianluca Li Causi <lica...@mporzio.astro.it> wrote: > Hi all, > I need to *simultaneously* fit two sets of data, Y1 and Y2, with two > functions, F1(X,P) and F2(X,P), having the same set of parameters P, > by using the routine LMFIT. > > In order to do this I join the two datasets in a uniq vector Y=[Y1,Y2] > and do the same for the model F=[F1,F2], as suggested also by Craig > Markwardt in a past thread in this group:http://groups.google.com/group/comp.lang.idl-pvwave/br owse_thread/thr... > > But my datasets contains a different number of points, N1 and N2, with > N2<<N1, so that the smaller one is almost neglected because its weight > in the uniq vector Y is proportional to N2 / N1. > > SO THAT: I've tried to multiply each error vector by the SQRT of its > fraction: > > Y1_err *= SQRT(N1 / (N1+N2)) > Y2_err *= SQRT(N2 / (N1+N2)) > > and now I get a nice fit for the two dataset simultaneously! > > BUT: when I compute the Reduced ChiSquare = ChiSquare / (N1 + N2 - > N_parameters) I get a completely wrong result (vey high, far from > 1.0), due to my modification of the true error vector ! > > THUS: what I can do ? > One idea is to force the Y1 and Y2 vectors to have the same number of > elements M, which must be the minimum multiple of both N1 and N2, but > it could be a very large number..... > > Could anybody help me ?! > Thanks a lot! > Gianluca Can't you just set the errors back to their old values before recomputing chi-squared? chris
creator: Chris[6]
date: 2009-05-15T08:28:01-00:00

Re: Simultaneous fitting in IDL
type: item
description: On May 15, 10:28 am, Chris <beaum...@ifa.hawaii.edu> wrote: > > Can't you just set the errors back to their old values before > recomputing chi-squared? > chris Yes, I can do this and re-compute the correct Reduced ChiSquare at the end by myself: BUT can I be sure that the fitting itself is not affected by my errors modification? And what about the final errors on the parameters, are they the correct ones or I must re-normalize them in some way? Thanks Gianluca
creator: Gianluca Li Causi
date: 2009-05-15T09:43:32-00:00

Re: Simultaneous fitting in IDL
type: item
description: Gianluca Li Causi wrote: > Hi all, > I need to *simultaneously* fit two sets of data, Y1 and Y2, with two > functions, F1(X,P) and F2(X,P), having the same set of parameters P, > by using the routine LMFIT. > > In order to do this I join the two datasets in a uniq vector Y=[Y1,Y2] > and do the same for the model F=[F1,F2], as suggested also by Craig > Markwardt in a past thread in this group: > http://groups.google.com/group/comp.lang.idl-pvwave/browse_t hread/thread/ba341b5169e7557f/498234b82c67279f?hl=en&lnk =gst&q=lmfit#498234b82c67279f > > But my datasets contains a different number of points, N1 and N2, with > N2<<N1, so that the smaller one is almost neglected because its weight > in the uniq vector Y is proportional to N2 / N1. > > SO THAT: I've tried to multiply each error vector by the SQRT of its > fraction: > > Y1_err *= SQRT(N1 / (N1+N2)) > Y2_err *= SQRT(N2 / (N1+N2)) > > and now I get a nice fit for the two dataset simultaneously! > > BUT: when I compute the Reduced ChiSquare = ChiSquare / (N1 + N2 - > N_parameters) I get a completely wrong result (vey high, far from > 1.0), due to my modification of the true error vector ! > > THUS: what I can do ? > One idea is to force the Y1 and Y2 vectors to have the same number of > elements M, which must be the minimum multiple of both N1 and N2, but > it could be a very large number..... > > > Could anybody help me ?! > Thanks a lot! > Gianluca > > > Gianluca, If you really believe your data and, more importantly, really believe the error bars associated with them then it's formally correct that all of your Y2 data points are essentially being ignored. Bet you're sorry you went to the trouble of measuring them now? :) The simple scenario when they wouldn't be ignored would be when one (or more) of the parameters only has a significant effect on the modelling of Y2 data. This doesn't seem to be the case here since when you mess with the errors you do get a change in your fit. Are the errors on your points (parti…
creator: Allan Whiteford
date: 2009-05-15T11:15:36-00:00

Re: Simultaneous fitting in IDL
type: item
description: On May 15, 1:15 pm, Allan Whiteford <allan.rem...@phys.remove.strath.ac.remove.uk> wrote: > Gianluca, > > If you really believe your data and, more importantly, really believe > the error bars associated with them then it's formally correct that all > of your Y2 data points are essentially being ignored. Bet you're sorry > you went to the trouble of measuring them now? :) Dear Allan, thank you much for your reply, but I do not fully agree qith you. A can assume that both my Y1 and Y2 data are random and not correlated, but they are not equally dependent on the parameters P= [P1,P2,P3]: in fact the F1 function is weakly dependent on the last parameter P3, while the F2 function is heavily dependent on it. So, if the Y2 data are essentially ingnored I get a large indetermination of P3, although the Y2 data are enough to constraint it very well! Instead if I modify the weight of Y2, i.e. its error, as I do, I get a nice result for any of my parameters, expecially the P3 given that the F2 function passes very well across the Y2 data points! At the limit where each function F_i only depends on a single parameter P_i, the simultaneous fitting of the joined Y_i vectors shoud give the same result of the independent one-parameter fitting of each Y_i dataset, shoudn't it? My procedure seems to satisfy this limit, which it seems to me is not satisfied by your suggestion to leave the errors as they are. > The simple scenario when they wouldn't be ignored would be when one (or > more) of the parameters only has a significant effect on the modelling > of Y2 data. This doesn't seem to be the case here since when you mess > with the errors you do get a change in your fit. > > Are the errors on your points (particularly your Y1 points) truly > random? Chances are your Y1 error estimates don't contain only random > errors but also have some form of systematic error - hence correlation > between the points which is a whole new can of worms. > > In the case of correlated errors, your we…
creator: Gianluca Li Causi
date: 2009-05-15T13:20:02-00:00

Re: Simultaneous fitting in IDL
type: item
description: Gianluca, Gianluca Li Causi wrote: > On May 15, 1:15 pm, Allan Whiteford > <allan.rem...@phys.remove.strath.ac.remove.uk> wrote: >> Gianluca, >> >> If you really believe your data and, more importantly, really believe >> the error bars associated with them then it's formally correct that all >> of your Y2 data points are essentially being ignored. Bet you're sorry >> you went to the trouble of measuring them now? :) > > > Dear Allan, > thank you much for your reply, but I do not fully agree qith you. > > A can assume that both my Y1 and Y2 data are random and not > correlated, I have no idea where your data are coming so you are the best person to say how valid that assumption is - it's not typical for that assumption to be valid (but it is typical for that assumption to be made). > but they are not equally dependent on the parameters P= > [P1,P2,P3]: in fact the F1 function is weakly dependent on the last > parameter P3, while the F2 function is heavily dependent on it. > The key thing here is lots of points with a weak dependence and a few points with a strong dependence. This is very different from zero dependence. > So, if the Y2 data are essentially ingnored I get a large > indetermination of P3, although the Y2 data are enough to constraint > it very well! > Instead if I modify the weight of Y2, i.e. its error, as I do, I get a > nice result for any of my parameters, expecially the P3 given that the > F2 function passes very well across the Y2 data points! > You have a pre-disposition to want the two curves to fit equally well. Your fitting algorithm has a pre-disposition to want every point to fit equally well (within its error bars). > At the limit where each function F_i only depends on a single > parameter P_i, the simultaneous fitting of the joined Y_i vectors > shoud give the same result of the independent one-parameter fitting of > each Y_i dataset, shoudn't it? Yes. > My procedure seems to satisfy this limit, which it seems to me is not > satisfied by …
creator: Allan Whiteford
date: 2009-05-18T16:27:16-00:00