comp.lang.idl-pvwave archive: archive » Reading netcdf/hdf5 files slows down

Home » Public Forums » archive » Reading netcdf/hdf5 files slows down

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Reading netcdf/hdf5 files slows down [message #92023]

Thu, 01 October 2015 12:12

khyde
Messages: 21
Registered: November 2013

Junior Member

Hello,

I have several thousand netcdf and hdf5 files that I need to open and extract data from. As the program loops through the files, it progressively takes longer and longer to read each file. Over the course of 24 hours, it went from taking ~6 seconds to completely read the file to more than a minute. Since seconds add up when reading 50,000+ files, I was hoping someone might know why the program would be slowing down and if there is anything (other than restarting IDL) I can do to fix it.

For the netcdf files I'm using the following procedures:
NCDF_OPEN
NCDF_INQUIRE
NCDF_ATTNAME
NCDF_ATTGET
NCDF_CLOSE

And for the HDF5:
H5F_OPEN
H5G_OPEN
H5D_OPEN
H5D_READ
H5D_CLOSE
H5G_CLOSE
H5F_CLOSE

Lastly, I am being sure to close every file once I am done with it and removing any unneeded variables from memory.

Any suggestions would be greatly appreciated.

Thank you,
Kim

Report message to a moderator

[Message index]

		Reading netcdf/hdf5 files slows down By: khyde on Thu, 01 October 2015 12:12
		Re: Reading netcdf/hdf5 files slows down By: Paul Van Delst[1] on Thu, 01 October 2015 13:45
		Re: Reading netcdf/hdf5 files slows down By: khyde on Fri, 02 October 2015 05:49

Previous Topic:	Attempt to call undefined procedure: 'DEFAULT' error
Next Topic:	How do you using the IDL to add a straight line of 45 degrees inclination to scatter plot of the square?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 13:36:10 PDT 2025

Total time taken to generate the page: 0.00512 seconds