comp.lang.idl-pvwave archive: archive » reading and writing very slow

Home » Public Forums » archive » reading and writing very slow

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

reading and writing very slow [message #75052]

Fri, 11 February 2011 05:12

oxfordenergyservices is currently offline

oxfordenergyservices
Messages: 56
Registered: January 2009

Member

Hi

I have some 1-2 GB text files (lots of them!), each containing weather
data for many thousands of stations for 1 year (per file). I need to
get the data out of the year files and into files which have all the
data for 1 weather station. It's easy but slow.

I am reading each year file line by line and appending that line to
the filename of the station (which is one of the fields on the line).
(openw.../append, close). Does opening and closing files so many
times have such an overhead? Is there a quicker way?

thanks

Report message to a moderator

Previous Topic:	idl or envi?
Next Topic:	Array searching efficiency

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 17:37:24 PDT 2025

Total time taken to generate the page: 0.00417 seconds