comp.lang.idl-pvwave archive - RDF feed
type: channel
description: Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
items: match_2d
items: Re: match_2d

match_2d
type: item
description: Hello everyone, I am using match_2d from J.D.Smith's library (http:// tir.astro.utoledo.edu/idl/match_2d.pro) to match between two catalogues. Below is what i am doing : IDL> print,star_radec1 242.759 -29.4162 252.666 -31.6364 250.523 -30.5292 244.782 -20.2181 IDL> print,star_radec2 252.666 -31.6364 250.523 -30.5292 267.782 -22.9120 IDL> match=match_2d(star_radec1(0,*),star_radec1(1,*),star_radec2 (0,*),star_radec2(1,*),.02,MATCH_DISTANCE=md) IDL> print,match -1 -1 -1 -1 IDL> print,md 5.98943e-22 3.62562 2.23579e-17 2.62037e-22 Eventhough there are 3 objects matching, why is it that match_2d is not finding them?? I shall be very grateful for any pointers... Thanks and regards, Vino
creator: vino
date: 2009-04-20T13:40:19-00:00

Re: match_2d
type: item
description: Hi Jeremy! Thank you for ur reply... Unfortunately as you correctly guessed my array is too big even when i split it into chunks.. :( eg: RA1 FLOAT = Array[267679] DEC1 FLOAT = Array[267679] RA2 FLOAT = Array[235476] DEC2 FLOAT = Array[235476] So i guess any form of brute force array method is going to turn up memory issues!! :( Since all my stellar photometry data will be finally loaded into a data base, i just have to make sure i gather all the information for a particular object before further analysis...It is a round about way but saves a lot of initial processing time.... Thank you so much for your help... Regards Vino On Apr 24, 6:50 pm, Jeremy Bailin <astroco...@gmail.com> wrote: > On Apr 23, 8:10 am, vino <astrocr...@gmail.com> wrote: > > > >> Hi Jeremy!! > >> Thank you very much for helping me out....It works very well with my >> data set... >> For me to be able to use this routine is going to save me about a >> couple of weeks of runtime in my program!! > >> I have looked at WITHINSPHRAD but in that case, i still need to have >> a loop which is what i was trying to avoid!! > >> Thanks to J.D.Smith for giving us a boon with routines like this!! ( i >> will someday learn how to use histogram).. > >> Regards, > >> Vino > >> On Apr 22, 11:39 pm, JDS <jdtsmith.nos...@yahoo.com> wrote: > >>>> Aha... I've looked at it in gory detail, and it turns out that the >>>> routine implicitly assumes that the minimum value of both x2 and y2 >>>> are 0. So you can get it to work if you do the following: > >>> Aha! Thanks for the catch. That's what you get when you evaluate an >>> algorithm on artificial random coordinates ranging uniformly from >>> [0,1]. > >>> I've updated MATCH_2D at the address mentioned to handle this issue >>> explicitly, and also catch cases of matching points which fall just >>> slightly outside the bounding box of the search set. I've also added >>> a much-needed warning regarding using this Euclidean matching >>> algorithm for points on the sphere …
creator: vino
date: 2009-04-27T10:53:56-00:00

Re: match_2d
type: item
description: > I'm pretty sure there's a HIST_ND-based algorithm of doing this > similar to MATCH_2D but taking spherical trig into account, but I > don't have the patience to figure it out. That would be challenging for the whole sphere, since histogram can only evaluate monotonic coordinate fields. You can always first remap your coordinates using some projection which puts the ill-behaved parts (nominally, the poles) far away, and preserves distance locally. For example, if you have a small field (a degree or so) near the pole, this would be a nice way of solving the converging longitude lines issues. But generally? Sounds tough. JD
creator: JDS
date: 2009-04-27T19:06:37-00:00

Re: match_2d
type: item
description: On Apr 27, 3:06 pm, JDS <jdtsmith.nos...@yahoo.com> wrote: >> I'm pretty sure there's a HIST_ND-based algorithm of doing this >> similar toMATCH_2Dbut taking spherical trig into account, but I >> don't have the patience to figure it out. > > That would be challenging for the whole sphere, since histogram can > only evaluate monotonic coordinate fields. You can always first remap > your coordinates using some projection which puts the ill-behaved > parts (nominally, the poles) far away, and preserves distance > locally. For example, if you have a small field (a degree or so) near > the pole, this would be a nice way of solving the converging longitude > lines issues. But generally? Sounds tough. > > JD How about if it was done in 3D? Instead of 2D angular coordinates, use the 3D coordinates of the relevant points on the surface of a unit sphere, and then use HIST_ND to determine which 3D bin the points are in and build the algorithm analogously to MATCH_2D? The main problem I see is that, for small bin sizes (ie. small desired angular separations), there's a lot of wasted memory storing the histogram in locations that don't lie on the surface of the sphere and therefore are necessarily zero. But maybe there's a way of enumerating the bins that do contain part of the surface - if so, then you could use that enumeration to map the 3D positions into a simple number that you can run HISTOGRAM on. -Jeremy.
creator: Jeremy Bailin
date: 2009-04-29T02:44:09-00:00

Re: match_2d
type: item
description: On Apr 28, 10:44 pm, Jeremy Bailin <astroco...@gmail.com> wrote: > On Apr 27, 3:06 pm, JDS <jdtsmith.nos...@yahoo.com> wrote: > >>> I'm pretty sure there's a HIST_ND-based algorithm of doing this >>> similar toMATCH_2Dbut taking spherical trig into account, but I >>> don't have the patience to figure it out. > >> That would be challenging for the whole sphere, since histogram can >> only evaluate monotonic coordinate fields. You can always first remap >> your coordinates using some projection which puts the ill-behaved >> parts (nominally, the poles) far away, and preserves distance >> locally. For example, if you have a small field (a degree or so) near >> the pole, this would be a nice way of solving the converging longitude >> lines issues. But generally? Sounds tough. > >> JD > > How about if it was done in 3D? Instead of 2D angular coordinates, use > the 3D coordinates of the relevant points on the surface of a unit > sphere, and then use HIST_ND to determine which 3D bin the points are > in and build the algorithm analogously to MATCH_2D? > > The main problem I see is that, for small bin sizes (ie. small desired > angular separations), there's a lot of wasted memory storing the > histogram in locations that don't lie on the surface of the sphere and > therefore are necessarily zero. But maybe there's a way of enumerating > the bins that do contain part of the surface - if so, then you could > use that enumeration to map the 3D positions into a simple number that > you can run HISTOGRAM on. I thought of that and rejected it for the reason you mention. The vast majority of memory would be devoted to empty volume, and as the resolution grew, the fraction of wasted memory would grow as well. The mapping you describe to do away with the empty space is equivalent to spherical projection, for which there is no unique mapping for the whole sphere. One possibility would be to project iteratively, forming low distortion projections over the sphere to push the poles off …
creator: JDS
date: 2009-04-29T20:29:43-00:00

Re: match_2d
type: item
description: On Apr 29, 4:29 pm, JDS <jdtsmith.nos...@yahoo.com> wrote: > On Apr 28, 10:44 pm, Jeremy Bailin <astroco...@gmail.com> wrote: > > > >> On Apr 27, 3:06 pm, JDS <jdtsmith.nos...@yahoo.com> wrote: > >>>> I'm pretty sure there's a HIST_ND-based algorithm of doing this >>>> similar toMATCH_2Dbut taking spherical trig into account, but I >>>> don't have the patience to figure it out. > >>> That would be challenging for the whole sphere, since histogram can >>> only evaluate monotonic coordinate fields. You can always first remap >>> your coordinates using some projection which puts the ill-behaved >>> parts (nominally, the poles) far away, and preserves distance >>> locally. For example, if you have a small field (a degree or so) near >>> the pole, this would be a nice way of solving the converging longitude >>> lines issues. But generally? Sounds tough. > >>> JD > >> How about if it was done in 3D? Instead of 2D angular coordinates, use >> the 3D coordinates of the relevant points on the surface of a unit >> sphere, and then use HIST_ND to determine which 3D bin the points are >> in and build the algorithm analogously to MATCH_2D? > >> The main problem I see is that, for small bin sizes (ie. small desired >> angular separations), there's a lot of wasted memory storing the >> histogram in locations that don't lie on the surface of the sphere and >> therefore are necessarily zero. But maybe there's a way of enumerating >> the bins that do contain part of the surface - if so, then you could >> use that enumeration to map the 3D positions into a simple number that >> you can run HISTOGRAM on. > > I thought of that and rejected it for the reason you mention. The > vast majority of memory would be devoted to empty volume, and as the > resolution grew, the fraction of wasted memory would grow as well. > The mapping you describe to do away with the empty space is equivalent > to spherical projection, for which there is no unique mapping for the > whole sphere. One possibility wo…
creator: Jeremy Bailin
date: 2009-04-29T23:30:02-00:00

Re: match_2d
type: item
description: On Apr 29, 7:30 pm, Jeremy Bailin <astroco...@gmail.com> wrote: > On Apr 29, 4:29 pm, JDS <jdtsmith.nos...@yahoo.com> wrote: > > > >> On Apr 28, 10:44 pm, Jeremy Bailin <astroco...@gmail.com> wrote: > >>> On Apr 27, 3:06 pm, JDS <jdtsmith.nos...@yahoo.com> wrote: > >>>> > I'm pretty sure there's a HIST_ND-based algorithm of doing this >>>> > similar toMATCH_2Dbut taking spherical trig into account, but I >>>> > don't have the patience to figure it out. > >>>> That would be challenging for the whole sphere, since histogram can >>>> only evaluate monotonic coordinate fields. You can always first remap >>>> your coordinates using some projection which puts the ill-behaved >>>> parts (nominally, the poles) far away, and preserves distance >>>> locally. For example, if you have a small field (a degree or so) near >>>> the pole, this would be a nice way of solving the converging longitude >>>> lines issues. But generally? Sounds tough. > >>>> JD > >>> How about if it was done in 3D? Instead of 2D angular coordinates, use >>> the 3D coordinates of the relevant points on the surface of a unit >>> sphere, and then use HIST_ND to determine which 3D bin the points are >>> in and build the algorithm analogously to MATCH_2D? > >>> The main problem I see is that, for small bin sizes (ie. small desired >>> angular separations), there's a lot of wasted memory storing the >>> histogram in locations that don't lie on the surface of the sphere and >>> therefore are necessarily zero. But maybe there's a way of enumerating >>> the bins that do contain part of the surface - if so, then you could >>> use that enumeration to map the 3D positions into a simple number that >>> you can run HISTOGRAM on. > >> I thought of that and rejected it for the reason you mention. The >> vast majority of memory would be devoted to empty volume, and as the >> resolution grew, the fraction of wasted memory would grow as well. >> The mapping you describe to do away with the empty space is equivalent >> to s…
creator: Jeremy Bailin
date: 2009-04-30T00:00:30-00:00

Re: match_2d
type: item
description: On Apr 29, 8:00 pm, Jeremy Bailin <astroco...@gmail.com> wrote: > On Apr 29, 7:30 pm, Jeremy Bailin <astroco...@gmail.com> wrote: > > > >> On Apr 29, 4:29 pm, JDS <jdtsmith.nos...@yahoo.com> wrote: > >>> On Apr 28, 10:44 pm, Jeremy Bailin <astroco...@gmail.com> wrote: > >>>> On Apr 27, 3:06 pm, JDS <jdtsmith.nos...@yahoo.com> wrote: > >>>> > > I'm pretty sure there's a HIST_ND-based algorithm of doing this >>>> > > similar toMATCH_2Dbut taking spherical trig into account, but I >>>> > > don't have the patience to figure it out. > >>>> > That would be challenging for the whole sphere, since histogram can >>>> > only evaluate monotonic coordinate fields. You can always first remap >>>> > your coordinates using some projection which puts the ill-behaved >>>> > parts (nominally, the poles) far away, and preserves distance >>>> > locally. For example, if you have a small field (a degree or so) near >>>> > the pole, this would be a nice way of solving the converging longitude >>>> > lines issues. But generally? Sounds tough. > >>>> > JD > >>>> How about if it was done in 3D? Instead of 2D angular coordinates, use >>>> the 3D coordinates of the relevant points on the surface of a unit >>>> sphere, and then use HIST_ND to determine which 3D bin the points are >>>> in and build the algorithm analogously toMATCH_2D? > >>>> The main problem I see is that, for small bin sizes (ie. small desired >>>> angular separations), there's a lot of wasted memory storing the >>>> histogram in locations that don't lie on the surface of the sphere and >>>> therefore are necessarily zero. But maybe there's a way of enumerating >>>> the bins that do contain part of the surface - if so, then you could >>>> use that enumeration to map the 3D positions into a simple number that >>>> you can run HISTOGRAM on. > >>> I thought of that and rejected it for the reason you mention. The >>> vast majority of memory would be devoted to empty volume, and as the >>> resolution grew, the fraction of wasted me…
creator: Jeremy Bailin
date: 2009-05-09T05:02:40-00:00

Re: match_2d
type: item
description: Can you tell us what your error/problem when using this?? One thing i do is to use reform to make sure x1,y1,x2 and y2 is a 1D array. Regards, Vino On May 6, 9:29 am, Dave Poreh <d.po...@gmail.com> wrote: > Hello everyone, > I have a problem that I can’t fix. > I am using match_2d from J.D.Smith's library to match between two > catalogues. Below is what i am doing: > IDL> print, laser > 13.51132 78.08015 2.244529 > 13.5041 78.07921 2.416421 > 13.49686 78.07828 2.420123 > 13.48957 78.07736 2.668057 > 13.48225 78.07644 2.652986 > 13.4749 78.07554 2.604654 > 13.46749 78.07465 2.373039 > 13.46009 78.07375 2.699916 > ………………… > IDL> print, sat > 13.50834 78.08333 5.965842 > 13.50001 78.08333 3.459386 > 13.50001 78.075 3.092191 > 13.49168 78.075 3.444704 > 13.48334 78.075 3.364458 > 13.47501 78.075 3.108061 > 13.46668 78.075 3.121016 > 13.45834 78.075 3.017546 > 13.45001 78.075 3.15583 > …………. > What I want to do is find the closest data from array laser to each of > array sat and then average them (of course for some points of sat I do > not have close laser data). At the end I want to plot sat data and > averaged laser to comparison. > x1=laser[0,*] > y1=laser[1,*] > x2=sat[0,*] > y2=sat[1,*] > > match=match_2d(x1,y1,x2,y2,0.008,MATCH_DISTANCE=md) > Any help highly appreciated > Cheers > Dave
creator: vino
date: 2010-05-06T11:27:07-00:00

Re: match_2d
type: item
description: On May 6, 4:27 am, vino <astrocr...@gmail.com> wrote: > Can you tell us what your error/problem when using this?? > One thing i do is to use reform to make sure x1,y1,x2 and y2 is a 1D > array. > > Regards, > Vino > > On May 6, 9:29 am, Dave Poreh <d.po...@gmail.com> wrote: > > > >> Hello everyone, >> I have a problem that I can’t fix. >> I am using match_2d from J.D.Smith's library to match between two >> catalogues. Below is what i am doing: >> IDL> print, laser >> 13.51132 78.08015 2.244529 >> 13.5041 78.07921 2.416421 >> 13.49686 78.07828 2.420123 >> 13.48957 78.07736 2.668057 >> 13.48225 78.07644 2.652986 >> 13.4749 78.07554 2.604654 >> 13.46749 78.07465 2.373039 >> 13.46009 78.07375 2.699916 >> ………………… >> IDL> print, sat >> 13.50834 78.08333 5.965842 >> 13.50001 78.08333 3.459386 >> 13.50001 78.075 3.092191 >> 13.49168 78.075 3.444704 >> 13.48334 78.075 3.364458 >> 13.47501 78.075 3.108061 >> 13.46668 78.075 3.121016 >> 13.45834 78.075 3.017546 >> 13.45001 78.075 3.15583 >> …………. >> What I want to do is find the closest data from array laser to each of >> array sat and then average them (of course for some points of sat I do >> not have close laser data). At the end I want to plot sat data and >> averaged laser to comparison. >> x1=laser[0,*] >> y1=laser[1,*] >> x2=sat[0,*] >> y2=sat[1,*] > >> match=match_2d(x1,y1,x2,y2,0.008,MATCH_DISTANCE=md) >> Any help highly appreciated >> Cheers >> Dave Actually there is no error, but I can’t understand the procedure. At the end output is a 1D array (match). What I want is for each pixel of SAT array find the closest data from laser array (but I don’t know how to do that). Cheers Dave
creator: d.poreh
date: 2010-05-06T12:25:43-00:00

Re: match_2d
type: item
description: > Actually there is no error, but I can’t understand the procedure. At > the end output is a 1D array (match). What I want is for each pixel of > SAT array find the closest data from laser array (but I don’t know how > to do that). > Cheers > Dave In your example match=match_2d(x1,y1,x2,y2,0.008,MATCH_DISTANCE=md) 'match' gives the indices of x2,y2 which are the closest match to x1,y1. So the closest point to x1[0],y1[0] is x2[match[0]],y2[match[0]]. --Wayne
creator: wlandsman
date: 2010-05-06T18:11:23-00:00

Re: match_2d
type: item
description: On May 6, 11:11 am, wlandsman <wlands...@gmail.com> wrote: >> Actually there is no error, but I can’t understand the procedure. At >> the end output is a 1D array (match). What I want is for each pixel of >> SAT array find the closest data from laser array (but I don’t know how >> to do that). >> Cheers >> Dave > > In your example > > match=match_2d(x1,y1,x2,y2,0.008,MATCH_DISTANCE=md) > > 'match' gives the indices of x2,y2 which are the closest match to > x1,y1. So the closest point to x1[0],y1[0] is > x2[match[0]],y2[match[0]]. > > --Wayne Thank you. Another question: how could I find the other points of x2- y2 that surrounded in the search area (in this example 0.008) because maybe there is more than 1 point. Cheers Dave
creator: d.poreh
date: 2010-05-07T12:06:13-00:00

Re: match_2d
type: item
description: On May 7, 8:06 am, Dave Poreh <d.po...@gmail.com> wrote: > On May 6, 11:11 am, wlandsman <wlands...@gmail.com> wrote: > > > > > >>> Actually there is no error, but I can’t understand the procedure. At >>> the end output is a 1D array (match). What I want is for each pixel of >>> SAT array find the closest data from laser array (but I don’t know how >>> to do that). >>> Cheers >>> Dave > >> In your example > >> match=match_2d(x1,y1,x2,y2,0.008,MATCH_DISTANCE=md) > >> 'match' gives the indices of x2,y2 which are the closest match to >> x1,y1. So the closest point to x1[0],y1[0] is >> x2[match[0]],y2[match[0]]. > >> --Wayne > > Thank you. Another question: how could I find the other points of x2- > y2 that surrounded in the search area (in this example 0.008) because > maybe there is more than 1 point. > Cheers > Dave You use a routine that I coincidentally wrote yesterday: matchall_2d.pro, which is based heavily on JD's code (see below - I haven't yet put it up in JBIU but I'll do it soon). Note that if your coordinates are things like lat, lon and cover a larger area, 2D distances won't be correct and you should use withinsphrad_vec3d.pro... which is also apparently not yet in JBIU. Jeez, I'm slacking. If you need it, ask me and I'll send it... and it will hopefully be online soon too. ;+ ; NAME: ; MATCHALL_2D ; ; PURPOSE: ; Determines which of a set of 2D coordinates are a given distance from ; each of a vector of points. Based on JD's MATCH_2D and my WITHINSPHRAD_VEC3D ; (in fact, it's basically WITHINSPHRAD_VEC3D tuned back down to a ; Euclidean surface). ; ; CATEGORY: ; Astro ; ; CALLING SEQUENCE: ; Result = MATCHALL_2D(X1, Y1, X2, Y2, Distance, Nwithin) ; ; INPUTS: ; X1: Vector of X coordinates. ; ; Y1: Vector of Y coordinates. ; ; X2: Vector of X coordinates. ; ; Y2: Vector of Y coordinates. ; ; Distance: Maximum distance. ; ; OUTPUTS: ; The function returns the list of indices of X2, Y2 that lie within ; Sphrad of each point X1,Y1. The format of the returned a…
creator: Jeremy Bailin
date: 2010-05-07T12:30:04-00:00

Re: match_2d
type: item
description: On May 7, 5:30 am, Jeremy Bailin <astroco...@gmail.com> wrote: > On May 7, 8:06 am, Dave Poreh <d.po...@gmail.com> wrote: > > > > > >> On May 6, 11:11 am, wlandsman <wlands...@gmail.com> wrote: > >>>> Actually there is no error, but I can’t understand the procedure. At >>>> the end output is a 1D array (match). What I want is for each pixel of >>>> SAT array find the closest data from laser array (but I don’t know how >>>> to do that). >>>> Cheers >>>> Dave > >>> In your example > >>> match=match_2d(x1,y1,x2,y2,0.008,MATCH_DISTANCE=md) > >>> 'match' gives the indices of x2,y2 which are the closest match to >>> x1,y1. So the closest point to x1[0],y1[0] is >>> x2[match[0]],y2[match[0]]. > >>> --Wayne > >> Thank you. Another question: how could I find the other points of x2- >> y2 that surrounded in the search area (in this example 0.008) because >> maybe there is more than 1 point. >> Cheers >> Dave > > You use a routine that I coincidentally wrote yesterday: > matchall_2d.pro, which is based heavily on JD's code (see below - I > haven't yet put it up in JBIU but I'll do it soon). Note that if your > coordinates are things like lat, lon and cover a larger area, 2D > distances won't be correct and you should use > withinsphrad_vec3d.pro... which is also apparently not yet in JBIU. > Jeez, I'm slacking. If you need it, ask me and I'll send it... and it > will hopefully be online soon too. > > ;+ > ; NAME: > ; MATCHALL_2D > ; > ; PURPOSE: > ; Determines which of a set of 2D coordinates are a given distance > from > ; each of a vector of points. Based on JD's MATCH_2D and my > WITHINSPHRAD_VEC3D > ; (in fact, it's basically WITHINSPHRAD_VEC3D tuned back down to a > ; Euclidean surface). > ; > ; CATEGORY: > ; Astro > ; > ; CALLING SEQUENCE: > ; Result = MATCHALL_2D(X1, Y1, X2, Y2, Distance, Nwithin) > ; > ; INPUTS: > ; X1: Vector of X coordinates. > ; > ; Y1: Vector of Y coordinates. > ; > ; X2: Vector of X coordinates. > ; > ; Y2: Vector of Y coordinates. > ; > ; Distanc…
creator: d.poreh
date: 2010-05-07T12:55:06-00:00