| Re: Short array memory allocation [message #61222] |
Thu, 10 July 2008 13:56  |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article
<0b404667-7c2f-41d9-a507-ede3ba1df3fc@56g2000hsm.googlegroups.com>,
ray.laymon064@gmail.com wrote:
> Hi Everyone,
>
> I searched the posted messages but couldn't find the answer I am
> looking for. Sorry if I repeat the same question.
>
> I am dealing with some data in "grib" format. I perform a simple
> operation on each file (open/extract single pixel). The array size of
> each file is small (450*250), and I don't store any of the variables
> at each loop (overwrite so no memory allocation is needed). However,
> there are lots of files (~600,000).
>
> I have a 2.5 Ghz dual-core 4GB memory laptop. When I run my script it,
> the task manager tells me IDL uses ~ 12% CPU & ~32MB memory (total PC
> CPU usage is ~ 16% and total PC memory usage is ~ 450MB)...And the
> whole script takes ~1,5 hours.
>
> I know, there are lots of small files, and naturally it takes time but
> is there a way I can speed up the processing time?
> Thanks,
>
> Ray
Try opening and closing the files without doing any work. It will probably
take about the same amount of time. That is, most of it is probably
system overhead.
Ken Bowman
|
|
|
|
| Re: Short array memory allocation [message #61224 is a reply to message #61222] |
Thu, 10 July 2008 13:55  |
pgrigis
Messages: 436
Registered: September 2007
|
Senior Member |
|
|
buy a 7500 rpm speed harddisk for your laptop?
laptop harddisks are slower than workstation ones,
so it may just be that you are hitting the limit of the hardware...
Ciao,
Paolo
ray.laymon...@gmail.com wrote:
> Hi Everyone,
>
> I searched the posted messages but couldn't find the answer I am
> looking for. Sorry if I repeat the same question.
>
> I am dealing with some data in "grib" format. I perform a simple
> operation on each file (open/extract single pixel). The array size of
> each file is small (450*250), and I don't store any of the variables
> at each loop (overwrite so no memory allocation is needed). However,
> there are lots of files (~600,000).
>
> I have a 2.5 Ghz dual-core 4GB memory laptop. When I run my script it,
> the task manager tells me IDL uses ~ 12% CPU & ~32MB memory (total PC
> CPU usage is ~ 16% and total PC memory usage is ~ 450MB)...And the
> whole script takes ~1,5 hours.
>
> I know, there are lots of small files, and naturally it takes time but
> is there a way I can speed up the processing time?
> Thanks,
>
> Ray
|
|
|
|
| Re: Short array memory allocation [message #61309 is a reply to message #61222] |
Fri, 11 July 2008 06:01 |
ray.laymon064
Messages: 8
Registered: July 2008
|
Junior Member |
|
|
That was the answer I was afraid to get..
I tried opening and closing the files only and it took the ~same
time...
Still, is there any IDL setting that I can change and make my IDL use
more sources?
In a way IDL to open/read the file faster.
Ray,
p.s. I have 7200 rpm SATA disk, and unfortunately I can't modify..
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
