| Re: Where vs Histogram vs ?? [message #32668 is a reply to message #32570] |
Tue, 22 October 2002 20:53   |
Andrew Cool
Messages: 219
Registered: January 1996
|
Senior Member |
|
|
Hi Stein,
The multistage method with the structure gives about a 13-15%
improvement, or from about 10 minutes down to about 8.5 minutes
for a run through the entire 15,425,228 records.
Cheers,
Andrew
Andrew Cool wrote:
>
>> Given the above, could you perhaps try a multi-stage selection, e.g.,
>>
>> wrf_ok = !database.WRF EQ WRF
>> wrf_freq_ok = !database.FREQ EQ FREQ AND temporary(wrf_ok)
>> ;; By now you should have 1/12th of the data left!
>> ;; Don't know how many bad_data_values you expect, the next one might
>> ;; not gain much:
>> wrf_freq_good =!database.parameter(nominated_parameter) NE bad_data_value $
>> AND temporary(wrf_freq_ok)
>>
>> index1 = where(wrf_freq_good)
>>
>> ;; Build a new database on this subset (smaller than 1/12th),
>> ;; continue with the rest of your searches...
>>
>
> Hello Stein,
>
> I think your multistage selection using "AND Temporary(prev_stage)"
> is the way for me. I'm rather enamoured with the use of the structure
> in this database, and reluctant to give it up without a fight. It just
> makes it so easy to query the database from the command line as well as
> programatically.
>
> Although arrays would probably be faster, I'll settle for a V2 rather
> than
> a Saturn V if it means I can keep the structures.
>
> Thanks to everyone for their suggestions!
>
> Andrew
>
> ------------------------------------------------------------ -----------------
> Andrew D. Cool
> Electromagnetics & Propagation Group
> Intelligence, Surveillance & Reconnaissance Division
> Defence Science & Technology Organisation
> PO Box 1500, Edinburgh
> South Australia 5111
>
> Phone : 061 8 8259 5740 Fax : 061 8 8259 6673
> Email : andrew.cool@dsto.defence.gov.au
> ------------------------------------------------------------ -----------------
--
------------------------------------------------------------ -----------------
Andrew D. Cool .->-.
Electromagnetics & Propagation Group '-<-'
Intelligence, Surveillance & Reconnaissance Division Transmitted on
Defence Science & Technology Organisation 100% recycled
PO Box 1500, Edinburgh electrons
South Australia 5111
Phone : 061 8 8259 5740 Fax : 061 8 8259 6673
Email : andrew.cool@no-spam.dsto.defence.gov.au
Please remove the no-spam from my email address to reply ;-)
------------------------------------------------------------ -----------------
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
