comp.lang.idl-pvwave archive: archive » Appending data to a dataset in an HDF5 file

Home » Public Forums » archive » Appending data to a dataset in an HDF5 file

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Appending data to a dataset in an HDF5 file [message #52421 is a reply to message #52356]

Thu, 01 February 2007 08:51

eddie haskell
Messages: 29
Registered: September 1998

Junior Member

> The title says it all, really: I'm trying to create a dataset with an
> unlimited dimension, so that I can append data later, preferably
> without reading the previous data.

There are several possible issues as to why it is failing for you:

When you create your dataset you must set the chunk_dimensions keyword
to be able to extend the size of a dataset.

You use h5d_extend to actually extend the dataset

If you want to write new data in the extended space without touching the
original data you need to use a hyperslab

Example:

;; create file
f = h5f_create('test.h5')
;; create datatype
dt = h5t_idl_create(1b)
;; create dataspace with unlimited size
ds = h5s_create_simple(10, max_dim=[-1])
;; create dataset: must set chunk dimensions
d = h5d_create(f, 'test', dt, ds, chunk=10)
;; write original data
h5d_write, d, bindgen(10)

;; extend dataset by two elements
h5d_extend, d, 12
;; get updated matching dataspace
ds2 = h5d_get_space(d)
;; mark dataspace to write to the last two elements only
h5s_select_hyperslab, ds2, 10, 2, /reset
;; create data to write
data = bytarr(12)
;; put new data in the last two elements
data[10:11] = [100b, 200b]
;; write only the new elements to the extended dataset
h5d_write, d, data, file_space=ds2, memory_space=ds2

;; verify new dataset
print, h5d_read(d)

;; close everything
h5f_close, f

Hope this helps.
Cheers,
eddie

Report message to a moderator

[Message index]

		Appending data to a dataset in an HDF5 file By: Maarten[1] on Wed, 31 January 2007 08:40
		Re: Appending data to a dataset in an HDF5 file By: eddie haskell on Fri, 02 February 2007 07:54
		Re: Appending data to a dataset in an HDF5 file By: Maarten[1] on Mon, 05 February 2007 02:35
		Re: Appending data to a dataset in an HDF5 file By: Maarten[1] on Fri, 02 February 2007 00:50
		Re: Appending data to a dataset in an HDF5 file By: eddie haskell on Thu, 01 February 2007 08:51
		Re: Appending data to a dataset in an HDF5 file By: David Fanning on Mon, 05 February 2007 05:22

Previous Topic:	IDL job opportunities
Next Topic:	image mapping

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 22:54:46 PDT 2025

Total time taken to generate the page: 1.04047 seconds