| Remove the repeated lines in my array... [message #93983] |
Tue, 13 December 2016 07:37  |
d.poreh
Messages: 406
Registered: October 2007
|
Senior Member |
|
|
Dear Folk,
Hi,
I have an array (24*300,000) and it is lon-lat with some specification (24 characteristics). Some of these lines are repeated numbers. I mean same lon-lat and same specifications. I just want to know how could i remove the repeated lines in my array,
Thanks for any kind of helps in advance,
Cheers,
Dave
|
|
|
|
| Re: Remove the repeated lines in my array... [message #93986 is a reply to message #93983] |
Tue, 13 December 2016 09:22  |
Markus Schmassmann
Messages: 129
Registered: April 2016
|
Senior Member |
|
|
Am 13.12.2016 um 16:37 schrieb dave poreh:
> I have an array (24*300,000) and it is lon-lat with some
> specification (24 characteristics). Some of these lines are repeated
> numbers. I mean same lon-lat and same specifications. I just want to
> know how could i remove the repeated lines in my array,
the following functions are untested
function ord, array
;+
; Calculates the ordinal of each value of an array in terms of the
; sorted values.
; inspired by jbiu's ord & uniqify, written by Markus Schmassmann
; IDL> result=ord(array)
;-
compile_opt defint32, strictarr
return, reform( value_locate(array[uniq(array,sort(array))],array), $
size(array,/dimensions),/overwrite )
end
function uniq_multi, array
ords=ulonarr(24,300000)
for i=0,23 do ords[i,*]=ord(array[i,0])
maxO=max(ords,dim=2)
idx=ulon64arr(300000)
for i=0,23 do idx=idx*maxO[i]+ords[*,i]
return, array[*,uniq(idx,sort(idx))]
end
if you run into an overflow for idx, then you could print each of the
lines into an element of an array of strings and then use
array[*,uniq(idx,sort(stringarray))]
I hope one of these approaches works or can be made to work,
Good luck,
Markus
|
|
|
|
| Re: Remove the repeated lines in my array... [message #94002 is a reply to message #93983] |
Mon, 19 December 2016 07:33 |
d.poreh
Messages: 406
Registered: October 2007
|
Senior Member |
|
|
On Tuesday, December 13, 2016 at 7:07:28 PM UTC+3:30, dave poreh wrote:
> Dear Folk,
> Hi,
> I have an array (24*300,000) and it is lon-lat with some specification (24 characteristics). Some of these lines are repeated numbers. I mean same lon-lat and same specifications. I just want to know how could i remove the repeated lines in my array,
> Thanks for any kind of helps in advance,
> Cheers,
> Dave
Thanks Markus,
I have tried this code but it gives me some error!,
If u give me your email i could send u one text file you test it yourself,if you do not mind,
Thanks for any kind of help before,
Cheers,
Dave
|
|
|
|
| Re: Remove the repeated lines in my array... [message #94021 is a reply to message #93986] |
Thu, 22 December 2016 05:30 |
Markus Schmassmann
Messages: 129
Registered: April 2016
|
Senior Member |
|
|
On 12/13/2016 06:22 PM, Markus Schmassmann wrote:
> Am 13.12.2016 um 16:37 schrieb dave poreh:
>> I have an array (24*300,000) and it is lon-lat with some
>> specification (24 characteristics). Some of these lines are repeated
>> numbers. I mean same lon-lat and same specifications. I just want to
>> know how could i remove the repeated lines in my array,
> the following functions are untested
>
> function ord, array
> ;+
> ; Calculates the ordinal of each value of an array in terms of the
> ; sorted values.
> ; inspired by jbiu's ord & uniqify, written by Markus Schmassmann
> ; IDL> result=ord(array)
> ;-
> compile_opt defint32, strictarr
> return, reform( value_locate(array[uniq(array,sort(array))],array), $
> size(array,/dimensions),/overwrite )
> end
>
> function uniq_multi, array
> ords=ulonarr(24,300000)
> for i=0,23 do ords[i,*]=ord(array[i,0])
> maxO=max(ords,dim=2)
> idx=ulon64arr(300000)
> for i=0,23 do idx=idx*maxO[i]+ords[*,i]
> return, array[*,uniq(idx,sort(idx))]
> end
>
>
> if you run into an overflow for idx, then you could print each of the
> lines into an element of an array of strings and then use
>
> array[*,uniq(idx,sort(stringarray))]
>
> I hope one of these approaches works or can be made to work,
here the corrected code should anyone else care.
Seasonal greetings to everyone, Markus
function uniq_multi, array
;+
; same as uniq, but considers multiple columns of an array
;-
compile_opt defint32, strictarr
sz=size(array,/dim)
ncol =sz[0]
nlines=sz[1]
ords=ulonarr(ncol,nlines)
for i=0,ncol-1 do ords[i,*]=ord(array[i,*])
maxO=max(ords,dim=2)
idx=ulon64arr(1,nlines)
for i=0,ncol-1 do idx=idx*maxO[i]+ords[i,*]
return, array[*,uniq(idx,sort(idx))]
end
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
