| IDL & HDF-5 files: Reading attributes at (unknown) list elements [message #67883] |
Tue, 08 September 2009 13:32 |
Luds
Messages: 7
Registered: September 2009
|
Junior Member |
|
|
Hello,
I'm new to hdf5 file formats, but am comfortable with IDL. I have a
question about reading hdf5 files which I can't seem to solve, and can
find no useful help on any available resource.
Here it is....
I'm working with N-body simulation outputs in hdf5 format, and have
run
the same simulation twice; once at very low resolution (i.e. small
file
size for testing), and once at high resolution (i.e. ~34 Gb file size,
for
production). The files contain a header with a bunch of basic info on
the
run, and attributes which are the particle data: (3D) position, (3D)
velocities, and (1D int) array if particle "ID"'s.
Associated with each of these files I have another file which lists
(apart
from the header) only a set of particle IDs; a (non-contiguous) subset
of
IDs from the simulation output above (these IDs are used to tag
particles
in the simulation file that happen to cluster together during the run
-
and the ultimate goal is to analyze these "particle clusters"). Let's
call
this one "ID_file", since it's only full of IDs.
With the small simulation file there are no problems. I just read the
desired
attribute for all particles using something like this:
group_id = h5g_open(file_id,gName)
dataset_id = h5d_open(group_id,desired_particle_attr)
particle_Attr = h5d_read(dataset_id)
I can then simply match the IDs found in ID_file with
those in the simulation file to access the list elements of the
positions and velocities of
particles belonging to the "particle clusters".
The problem is with the big simulation... I can't read the entire file
into memory, not even one attribute at a time. So what I need to have
is a
quick way to read from the simulation file only certain elements of
the
attributes: those corresponding to the IDs found in ID_file. Remember,
all
IDs in ID_file are in the simulation file also, but are not
necessarily
ordered in the same way.
So, is there a way I can read from the simulation file only the
elements
of attribute_1 (pos, or vel) that match a pre-defined set of
attribute_2
(IDs from ID_file)? It seems I should be able to use something like
H5S_SELECT_ELEMENTS, but I can't figure that procedure out for my
life.
If this is impossible.... I also know the positions of the center for
each "particle cluster"
so perhaps I could somehow read just a cube of data surrounding the
center position.
I can't think of any way to work-around my problem, and IDL doesn't
have the best hdf5
documentation.
Thanks for any advice anyone may have!
Luds
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
