Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
Remove the repeated lines in my array...
Dear Folk, Hi, I have an array (24*300,000) and it is lon-lat with some specification (24 characteristics). Some of these lines are repeated numbers. I mean same lon-lat and same specifications. I just want to know how could i remove the repeated lines in my array, Thanks for any kind of helps in advance, Cheers, Dave
Re: Remove the repeated lines in my array...
Am 13.12.2016 um 16:37 schrieb dave poreh: > I have an array (24*300,000) and it is lon-lat with some > specification (24 characteristics). Some of these lines are repeated > numbers. I mean same lon-lat and same specifications. I just want to > know how could i remove the repeated lines in my array, the following functions are untested function ord, array ;+ ; Calculates the ordinal of each value of an array in terms of the ; sorted values. ; inspired by jbiu's ord & uniqify, written by Markus Schmassmann ; IDL> result=ord(array) ;- compile_opt defint32, strictarr return, reform( value_locate(array[uniq(array,sort(array))],array), $ size(array,/dimensions),/overwrite ) end function uniq_multi, array ords=ulonarr(24,300000) for i=0,23 do ords[i,*]=ord(array[i,0]) maxO=max(ords,dim=2) idx=ulon64arr(300000) for i=0,23 do idx=idx*maxO[i]+ords[*,i] return, array[*,uniq(idx,sort(idx))] end if you run into an overflow for idx, then you could print each of the lines into an element of an array of strings and then use array[*,uniq(idx,sort(stringarray))] I hope one of these approaches works or can be made to work, Good luck, Markus
On Tuesday, December 13, 2016 at 7:07:28 PM UTC+3:30, dave poreh wrote: > Dear Folk, > Hi, > I have an array (24*300,000) and it is lon-lat with some specification (24 characteristics). Some of these lines are repeated numbers. I mean same lon-lat and same specifications. I just want to know how could i remove the repeated lines in my array, > Thanks for any kind of helps in advance, > Cheers, > Dave Thanks Markus, I have tried this code but it gives me some error!, If u give me your email i could send u one text file you test it yourself,if you do not mind, Thanks for any kind of help before, Cheers, Dave
On 12/13/2016 06:22 PM, Markus Schmassmann wrote: > Am 13.12.2016 um 16:37 schrieb dave poreh: >> I have an array (24*300,000) and it is lon-lat with some >> specification (24 characteristics). Some of these lines are repeated >> numbers. I mean same lon-lat and same specifications. I just want to >> know how could i remove the repeated lines in my array, > the following functions are untested > > function ord, array > ;+ > ; Calculates the ordinal of each value of an array in terms of the > ; sorted values. > ; inspired by jbiu's ord & uniqify, written by Markus Schmassmann > ; IDL> result=ord(array) > ;- > compile_opt defint32, strictarr > return, reform( value_locate(array[uniq(array,sort(array))],array), $ > size(array,/dimensions),/overwrite ) > end > > function uniq_multi, array > ords=ulonarr(24,300000) > for i=0,23 do ords[i,*]=ord(array[i,0]) > maxO=max(ords,dim=2) > idx=ulon64arr(300000) > for i=0,23 do idx=idx*maxO[i]+ords[*,i] > return, array[*,uniq(idx,sort(idx))] > end > > > if you run into an overflow for idx, then you could print each of the > lines into an element of an array of strings and then use > > array[*,uniq(idx,sort(stringarray))] > > I hope one of these approaches works or can be made to work, here the corrected code should anyone else care. Seasonal greetings to everyone, Markus function uniq_multi, array ;+ ; same as uniq, but considers multiple columns of an array ;- compile_opt defint32, strictarr sz=size(array,/dim) ncol =sz[0] nlines=sz[1] ords=ulonarr(ncol,nlines) for i=0,ncol-1 do ords[i,*]=ord(array[i,*]) maxO=max(ords,dim=2) idx=ulon64arr(1,nlines) for i=0,ncol-1 do idx=idx*maxO[i]+ords[i,*] return, array[*,uniq(idx,sort(idx))] end