comp.lang.idl-pvwave archive: archive » h5

Home » Public Forums » archive » h5_parse() in the profiler

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

h5_parse() in the profiler [message #93414]

Fri, 08 July 2016 09:50

MarioIncandenza
Messages: 231
Registered: February 2005

Senior Member

Hi IDL Wizards,

I'm working on an application requiring chunking through a huge quantity of HDF5 files. For (EXTREME) ease of coding, my code does
IDL> H5DATA = H5_PARSE(HDF5_file,/READ_DATA)
, and then operates on H5DATA. So so easy to code, but that call to H5_PARSE() is very time-consuming. I ran the IDL Profiler (as elegantly described here: http://www.idlcoyote.com/code_tips/whyslow.html), and found that all the time was being spent in two routines:
Routine Calls Only Total
CREATE_STRUCT (S) 1320 61.130619 0.046311 61.130619 0.046311
H5D_READ (S) 92 53.353344 0.579928 53.353344 0.579928

The 'H5D_READ' I understand, that is the low-level I/O and it is constrained by the system. But the 'CREATE_STRUCT' surprised me.

I guess CREATE_STRUCT() is where the memory allocation is occurring, but does it seem right that this takes more time than the actual disk I/O?

Any insights are welcome. I could rewrite the code to pull specific data out of the HDF5 file by hand, but that would be hundreds of lines of code, and I'd really rather not...

--Edward H.

Report message to a moderator

[Message index]

		h5_parse() in the profiler By: MarioIncandenza on Fri, 08 July 2016 09:50
		Re: h5_parse() in the profiler By: Markus Schmassmann on Fri, 08 July 2016 10:46
		Re: h5_parse() in the profiler By: MarioIncandenza on Fri, 08 July 2016 21:38
		Re: h5_parse() in the profiler By: Jim Pendleton on Sat, 09 July 2016 07:05

Previous Topic:	keep track of all libraries used
Next Topic:	IDL asynchronous execution fails at >50 child processes

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 13:38:57 PDT 2025

Total time taken to generate the page: 0.00419 seconds