comp.lang.idl-pvwave archive: archive » Reading various HDF files

Home » Public Forums » archive » Reading various HDF files

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Reading various HDF files [message #46204]

Wed, 09 November 2005 02:02

Nicola
Messages: 21
Registered: November 2005

Junior Member

Hi all
I'm facing this problem which deal I think to memory problem.
I've the ouput of numerical simulations which are saved in HDF format.
I've outputs every, lets say 10 time step, which means that I've around
1000 HDF file each of them containig 8 fields which have dimension
(64,128,8, dimension on x,y,z direction. It's a cubic domain
simulation).
Now I want to create a movie of the evolution of one field, let's say.
I choose one of the field of HDF file let's call it field and I choose
one particular plane on z, let's say I want the evolution of
field[*,*,8] as a function of time showing it as a movie.
Is there a good way to do this whitout a for loop for reading all the
files and than saving the only field which I need?
I hope I've been sufficiently clear.
Thanks for all the answer
n

Report message to a moderator

Re: Reading various HDF files [message #46272 is a reply to message #46204]

Fri, 11 November 2005 23:56

R.Bauer
Messages: 1424
Registered: November 1998

Senior Member

Rick Towler wrote:

> I don't want to complicate things but forget about MPEG. MPEG is most
> often a lousy format for scientific animations. I would suggest writing
> all of your frames to disk as individual .png files then using another
> program to create the animation.
>
> For 8-bit animations, it's tough to beat the FLI/FLC format. Rich
> Signell has a page with all you need to know to get started with FLI/FLC
> format at: http://woodshole.er.usgs.gov/operations/modeling/flc.html

Hi Rick

thanks for this link. I still have searched a while ago for something like
this. This is perfect for us because I have decided in the past to use
color_quan for our idl anim files to translate 24bit to 8bit.

cheers
Reimar

>
> For 24-bit animations it is a bit more complicated as there are a number
> of encoder/decoders out there. I would be happy to advise further if
> you want to go this route.
>
> If you just want to stick with MPEG, understand that most MPEG players
> only support certain frame sizes and bit rates so to ensure
> compatibility you need to stick to them. For frames sizes, I would
> stick with D1 (704x480 or 720x480), half D1 (352x480), and quarter D1
> (352x240). MPEG-2 will give you better overall quality vs file size but
> not all media players play MPEG-2 content (for example, Windows Media
> Player requires a special plug-in). MPEG-1 is widely compatible but you
> are limited in maximum bit-rate (1856 kbps) and quality vs file size
> isn't as good as MPEG-2.
>
> -Rick
>
> Nicola wrote:
>> Actually for the moment I've tried only to read each sds and then to
>> extract the let's call 0-zplane. I did not tried for the moment start
>> and count which will be me reading just one plane and so I can divided
>> for 8 the computational time. I will try..and also I will try mpeg_put
>> (although I was not succesfully in creating mpeg file with IDL up to
>> now :-( )
>> thank's a lot
>> n
>>

Report message to a moderator

Re: Reading various HDF files [message #46274 is a reply to message #46204]

Fri, 11 November 2005 03:35

Nicola
Messages: 21
Registered: November 2005

Junior Member

Thank you rick, maybe this will be the faster way avoiding mpeg
problems on IDL. Could you suggest me which encoder can I use
considering the fact that I would like to have a 24-bit animation.
thanks a lot to all of you

Report message to a moderator

Re: Reading various HDF files [message #46278 is a reply to message #46204]

Thu, 10 November 2005 09:31

Nicola
Messages: 21
Registered: November 2005

Junior Member

Maybe the problem is in my idl_startup which in some sense make me
crash windowing during batch file. Indeed in my startup file I've
something like this

if (!version.os_family eq 'unix') then device,true_color=24
window,/free,/pixmap,colors=-10
wdelete,!d.window
device,retain=2,decompose=0
device,get_visual_depth=depth

maybe this is the cause of failure during non-interactive session (with
no-interactive I mean with no X-windowing or with idl programs launched
in batch with at command )
sorry for boring you guy and thank you for your help

Report message to a moderator

Re: Reading various HDF files [message #46279 is a reply to message #46204]

Thu, 10 November 2005 08:39

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

Nicola writes:

> Here I am again, hoping not to be too boring. I've tried tvim and it
> worked fine for me. The only question that I'd like to now if is there
> any possibility to make the same think not interactively but through a
> batch file. I do not want to occupy the cpu of my server with this kind
> of stuff but just want to make it not interactively. I generally to
> this by creating an IDL programs and then use on linux the at command
> with low priority. The problem is that in this way no interactive
> xwindowing and consequently tv or tvim fails....
> maybe some of you have some ideas

Why don't you use the IMAGE keyword to MPEG_PUT instead
of the WINDOW keyword. The former loads the frame directly
with an image, whereas the latter takes a copy of of the
current graphics window as the image it loads. Be sure
you BYTSCL your image as you put it into the MPEG object.
That way you have no need of a window or a TV substitute
command (TVIMAGE, IMGDISP, PLOTIMAGE, etc.)

Cheers,

David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/

Report message to a moderator

Re: Reading various HDF files [message #46281 is a reply to message #46204]

Thu, 10 November 2005 07:58

peter.albert@gmx.de is currently offline

peter.albert@gmx.de
Messages: 108
Registered: July 2005

Senior Member

I am not sure what you mean with interactive or not interactive
respectively. I don't see where user interaction is necessary in the
above steps. If your problem has to do with X-forwarding or crashing
WINDOW commands, then you could use e.g. an ssh connection with
X-forwarding, e.g. with ssh -X ... If you then use the PIXMAP keyword
to WINDOW, you won't even get bothered by the windows appearing on yout
screen. Another option might be using the Z-Buffer, but I just heard
vague rumours and never used it myself.

Cheers,

Peter

Report message to a moderator

Re: Reading various HDF files [message #46282 is a reply to message #46204]

Thu, 10 November 2005 06:02

Nicola
Messages: 21
Registered: November 2005

Junior Member

Here I am again, hoping not to be too boring. I've tried tvim and it
worked fine for me. The only question that I'd like to now if is there
any possibility to make the same think not interactively but through a
batch file. I do not want to occupy the cpu of my server with this kind
of stuff but just want to make it not interactively. I generally to
this by creating an IDL programs and then use on linux the at command
with low priority. The problem is that in this way no interactive
xwindowing and consequently tv or tvim fails....
maybe some of you have some ideas

Report message to a moderator

Re: Reading various HDF files [message #46283 is a reply to message #46204]

Thu, 10 November 2005 03:51

peter.albert@gmx.de
Messages: 108
Registered: July 2005

Senior Member

Hi Nicola,

did you ever try the XSIZE and YSIZE keywords with WINDOW?

And if a window of size 64, 128 seems to be too small, you might also
want to use CONGRID. Or you give TVIM a try.
( http://www.astro.washington.edu/deutsch-bin/getpro/library28 .html?TVIM)

Cheers,

Peter

Nicola schrieb:

> Still some problem hopefully you can help me.
> Each of the field has dimension of the kind (64,128) or (128,256) or
> (256,512) or something like this. Using the suggestion that Peter gave
> me the image is not display in the entire window but only occupy a part
> of the window itself (obviously the window is larger than the image). I
> tried to work with position, xsize and ysize keyword of tv (or better
> of tvscl) but I did not manage to let the image occupy all the graphic
> window. Any suggestion?
> thank a lot

Report message to a moderator

Re: Reading various HDF files [message #46284 is a reply to message #46204]

Thu, 10 November 2005 03:07

Nicola
Messages: 21
Registered: November 2005

Junior Member

Still some problem hopefully you can help me.
Each of the field has dimension of the kind (64,128) or (128,256) or
(256,512) or something like this. Using the suggestion that Peter gave
me the image is not display in the entire window but only occupy a part
of the window itself (obviously the window is larger than the image). I
tried to work with position, xsize and ysize keyword of tv (or better
of tvscl) but I did not manage to let the image occupy all the graphic
window. Any suggestion?
thank a lot

Report message to a moderator

Re: Reading various HDF files [message #46292 is a reply to message #46204]

Wed, 09 November 2005 06:41

Nicola
Messages: 21
Registered: November 2005

Junior Member

I really thank you Peter. I will try all the suggestions

Report message to a moderator

Re: Reading various HDF files [message #46309 is a reply to message #46204]

Fri, 18 November 2005 13:03

R.Bauer
Messages: 1424
Registered: November 1998

Senior Member

Hi Rick,

perhaps you like to know I have added this format for sequences
to the MoinMoin wiki engine http://moinmoin.wikiwikiweb.de

to my Gallery2 parser
http://moinmoin.wikiwikiweb.de/ParserMarket/Gallery2

It's not idl but we often use this to show idl images to the world.
And by now we could show them in fli format animations too.

cheers
Reimar

Rick Towler wrote:

>
> Nicola wrote:
>> Thank you rick, maybe this will be the faster way avoiding mpeg
>> problems on IDL. Could you suggest me which encoder can I use
>> considering the fact that I would like to have a 24-bit animation.
>> thanks a lot to all of you
>
> Well, I don't know about faster :( You'll inevitably spend time
> learning new software and experimenting with encoding. But in the end
> you should have higher quality and smaller animations :)
>
>
> If you only require Mac and windows support I would recommend:
>
> VP3 encoder - free, info at http://www.vp3.com/ The binaries hosted at
> xiph.org seem to be offline. Don't know the status of the project.
> This codec works with videomach.
>
> Divx - $20 from www.divx.com. A popular encoder for video bootleggers,
> I have had quite good luck with it using it for scientific animations.
> This codec works with videomach. The decoder is also available for
> linux, but the version lags a bit behind the windows and mac ports.
>
> Sorenson codecs (v4 and v5) - These codecs ship with Quicktime Pro
> (www.quicktime.com ~$20 for the windows version). This codec *doesn't*
> work with videomach. You can use videomach to create an uncompressed
> .avi which you can then encode using quicktime pro. The upside is that
> many people have quicktime installed on their PCs.
>
> Windows Media - The windows media *encoder* is available free:
> http://www.microsoft.com/windows/windowsmedia/9series/encode r/default.aspx
> Like the sorenson codecs, this encoder doesn't provide a Video for
> windows (VFW) interface and thus it can't be used directly with
> videomach. You would need to create uncompressed .avi files and encode
> using the WME software. The upside with WM is that anyone with windows
> media player 9+ installed can view the files. Further, if they have
> windows media 7 installed and an internet connection the WM9 decoder
> will be downloaded automatically.
>
>
> Remember that you need the codec installed on any machine you want to
> view the animation on. This includes laptops at conferences. Make sure
> you test before you stand up at the podium!
>
>
> I'll also recommend aviIDL from Ronn Kling's website
> (http://www.kilvarock.com/) which provides an interface to the video for
> windows API from IDL. Using this package you can use any VFW codec and
> go straight from IDL to your animation. No videomach required.
>
> Good luck!
>
> -Rick

Report message to a moderator

Re: Reading various HDF files [message #46315 is a reply to message #46204]

Fri, 18 November 2005 02:25

R.Bauer
Messages: 1424
Registered: November 1998

Senior Member

I see it does depend on the kind of photos I tried

Reimar Bauer wrote:
> Dear Rick
>
> there is newer version here and I tested it with digital photos and it
> works!
>
> http://vento.pi.tu-berlin.de/fli.html
>
>
> It would need some more work on utf-8 file names but principle it works
> with true color images too.
>
> cheers
> Reimar
>
> Rick Towler wrote:
>
>> Nicola wrote:
>>
>>
>>> Thank you rick, maybe this will be the faster way avoiding mpeg
>>> problems on IDL. Could you suggest me which encoder can I use
>>> considering the fact that I would like to have a 24-bit animation.
>>> thanks a lot to all of you
>>
>>
>> Well, I don't know about faster :( You'll inevitably spend time
>> learning new software and experimenting with encoding. But in the end
>> you should have higher quality and smaller animations :)
>>
>>
>> If you only require Mac and windows support I would recommend:
>>
>> VP3 encoder - free, info at http://www.vp3.com/ The binaries hosted at
>> xiph.org seem to be offline. Don't know the status of the project. This
>> codec works with videomach.
>>
>> Divx - $20 from www.divx.com. A popular encoder for video bootleggers,
>> I have had quite good luck with it using it for scientific animations.
>> This codec works with videomach. The decoder is also available for
>> linux, but the version lags a bit behind the windows and mac ports.
>>
>> Sorenson codecs (v4 and v5) - These codecs ship with Quicktime Pro
>> (www.quicktime.com ~$20 for the windows version). This codec *doesn't*
>> work with videomach. You can use videomach to create an uncompressed
>> .avi which you can then encode using quicktime pro. The upside is that
>> many people have quicktime installed on their PCs.
>>
>> Windows Media - The windows media *encoder* is available free:
>> http://www.microsoft.com/windows/windowsmedia/9series/encode r/default.aspx
>> Like the sorenson codecs, this encoder doesn't provide a Video for
>> windows (VFW) interface and thus it can't be used directly with
>> videomach. You would need to create uncompressed .avi files and encode
>> using the WME software. The upside with WM is that anyone with windows
>> media player 9+ installed can view the files. Further, if they have
>> windows media 7 installed and an internet connection the WM9 decoder
>> will be downloaded automatically.
>>
>>
>> Remember that you need the codec installed on any machine you want to
>> view the animation on. This includes laptops at conferences. Make sure
>> you test before you stand up at the podium!
>>
>>
>> I'll also recommend aviIDL from Ronn Kling's website
>> (http://www.kilvarock.com/) which provides an interface to the video for
>> windows API from IDL. Using this package you can use any VFW codec and
>> go straight from IDL to your animation. No videomach required.
>>
>> Good luck!
>>
>> -Rick
>
>
>

--
Reimar Bauer

Institut fuer Stratosphaerische Chemie (ICG-I)
Forschungszentrum Juelich
email: R.Bauer@fz-juelich.de
------------------------------------------------------------ -------
a IDL library at ForschungsZentrum Juelich
http://www.fz-juelich.de/icg/icg-i/idl_icglib/idl_lib_intro. html
============================================================ =======

Report message to a moderator

Re: Reading various HDF files [message #46316 is a reply to message #46204]

Fri, 18 November 2005 02:19

R.Bauer
Messages: 1424
Registered: November 1998

Senior Member

Dear Rick

there is newer version here and I tested it with digital photos and it
works!

http://vento.pi.tu-berlin.de/fli.html

It would need some more work on utf-8 file names but principle it works
with true color images too.

cheers
Reimar

Rick Towler wrote:
>
> Nicola wrote:
>
>> Thank you rick, maybe this will be the faster way avoiding mpeg
>> problems on IDL. Could you suggest me which encoder can I use
>> considering the fact that I would like to have a 24-bit animation.
>> thanks a lot to all of you
>
>
> Well, I don't know about faster :( You'll inevitably spend time
> learning new software and experimenting with encoding. But in the end
> you should have higher quality and smaller animations :)
>
>
> If you only require Mac and windows support I would recommend:
>
> VP3 encoder - free, info at http://www.vp3.com/ The binaries hosted at
> xiph.org seem to be offline. Don't know the status of the project. This
> codec works with videomach.
>
> Divx - $20 from www.divx.com. A popular encoder for video bootleggers,
> I have had quite good luck with it using it for scientific animations.
> This codec works with videomach. The decoder is also available for
> linux, but the version lags a bit behind the windows and mac ports.
>
> Sorenson codecs (v4 and v5) - These codecs ship with Quicktime Pro
> (www.quicktime.com ~$20 for the windows version). This codec *doesn't*
> work with videomach. You can use videomach to create an uncompressed
> .avi which you can then encode using quicktime pro. The upside is that
> many people have quicktime installed on their PCs.
>
> Windows Media - The windows media *encoder* is available free:
> http://www.microsoft.com/windows/windowsmedia/9series/encode r/default.aspx
> Like the sorenson codecs, this encoder doesn't provide a Video for
> windows (VFW) interface and thus it can't be used directly with
> videomach. You would need to create uncompressed .avi files and encode
> using the WME software. The upside with WM is that anyone with windows
> media player 9+ installed can view the files. Further, if they have
> windows media 7 installed and an internet connection the WM9 decoder
> will be downloaded automatically.
>
>
> Remember that you need the codec installed on any machine you want to
> view the animation on. This includes laptops at conferences. Make sure
> you test before you stand up at the podium!
>
>
> I'll also recommend aviIDL from Ronn Kling's website
> (http://www.kilvarock.com/) which provides an interface to the video for
> windows API from IDL. Using this package you can use any VFW codec and
> go straight from IDL to your animation. No videomach required.
>
> Good luck!
>
> -Rick

--
Reimar Bauer

Institut fuer Stratosphaerische Chemie (ICG-I)
Forschungszentrum Juelich
email: R.Bauer@fz-juelich.de
------------------------------------------------------------ -------
a IDL library at ForschungsZentrum Juelich
http://www.fz-juelich.de/icg/icg-i/idl_icglib/idl_lib_intro. html
============================================================ =======

Report message to a moderator

Re: Reading various HDF files [message #46389 is a reply to message #46274]

Mon, 14 November 2005 15:42

Rick Towler
Messages: 821
Registered: August 1998

Senior Member

Nicola wrote:
> Thank you rick, maybe this will be the faster way avoiding mpeg
> problems on IDL. Could you suggest me which encoder can I use
> considering the fact that I would like to have a 24-bit animation.
> thanks a lot to all of you

Well, I don't know about faster :( You'll inevitably spend time
learning new software and experimenting with encoding. But in the end
you should have higher quality and smaller animations :)

If you only require Mac and windows support I would recommend:

VP3 encoder - free, info at http://www.vp3.com/ The binaries hosted at
xiph.org seem to be offline. Don't know the status of the project.
This codec works with videomach.

Divx - $20 from www.divx.com. A popular encoder for video bootleggers,
I have had quite good luck with it using it for scientific animations.
This codec works with videomach. The decoder is also available for
linux, but the version lags a bit behind the windows and mac ports.

Sorenson codecs (v4 and v5) - These codecs ship with Quicktime Pro
(www.quicktime.com ~$20 for the windows version). This codec *doesn't*
work with videomach. You can use videomach to create an uncompressed
.avi which you can then encode using quicktime pro. The upside is that
many people have quicktime installed on their PCs.

Windows Media - The windows media *encoder* is available free:
http://www.microsoft.com/windows/windowsmedia/9series/encode r/default.aspx
Like the sorenson codecs, this encoder doesn't provide a Video for
windows (VFW) interface and thus it can't be used directly with
videomach. You would need to create uncompressed .avi files and encode
using the WME software. The upside with WM is that anyone with windows
media player 9+ installed can view the files. Further, if they have
windows media 7 installed and an internet connection the WM9 decoder
will be downloaded automatically.

Remember that you need the codec installed on any machine you want to
view the animation on. This includes laptops at conferences. Make sure
you test before you stand up at the podium!

I'll also recommend aviIDL from Ronn Kling's website
(http://www.kilvarock.com/) which provides an interface to the video for
windows API from IDL. Using this package you can use any VFW codec and
go straight from IDL to your animation. No videomach required.

Good luck!

-Rick

Report message to a moderator

Previous Topic:	Re: struct tag definitions
Next Topic:	Re: Plotting 3D spheres

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 11:53:59 PDT 2025

Total time taken to generate the page: 0.00541 seconds