comp.lang.idl-pvwave archive
Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado

Home » Public Forums » archive » Re: Read Zipped files
Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend 
Switch to threaded view of this topic Create a new topic Submit Reply
Re: Read Zipped files [message #49895] Wed, 30 August 2006 14:36 Go to next message
loknath is currently offline  loknath
Messages: 11
Registered: May 2006
Junior Member
I am running out of hard drive space and didnt want to unzip the files
but now I feel it is easier if I unzip the files in my hard drive.
Thanks everyone
Loknath

Mike Wallace wrote:
>> I don't think any of this would work in Windows but probably in OS X. Didn't
>> someone have a Windows DLM that supported zipped files?
>
> You're right. Windows doesn't have the same concept of named pipes that
> *nix does, and due to OS X being rooted in BSD, it should support named
> pipes.
>
> As for the original question, if you're using gzip to compress the
> files, IDL can read them by using the /COMPRESS flag to the OPENR, OPENU
> or OPENW commands.
>
> If you're files are zip files, the only option I know of is to unzip the
> file before you use it. If you are using *nix, you can use the named
> pipe approach as the file would be unzipped into memory rather than to
> the hard drive. If you're using Windows, the only option I can think of
> is to write a small program in another language that could handle the
> unzip and stream the results out to shared memory. You'd then have to
> make your IDL program read from shared memory instead of the file. It's
> not the easiest thing in the world if you've never worked with shared
> memory before.
>
> When you said that it would "be costly for my hard drive," what do you
> mean? Are you referring to all the files that would have to be written
> back out to disk all the time or are you concerned with running out of
> disk space?
>
> -Mike
Re: Read Zipped files [message #49915 is a reply to message #49895] Wed, 30 August 2006 04:51 Go to previous messageGo to next message
Haje Korth is currently offline  Haje Korth
Messages: 651
Registered: May 1997
Senior Member
Hi,
Randall Franks DLM is available at
http://kilvarock.com/freesoftware/dlms/randallfrank.htm. It thas
built-in gzip compression.

Cheers,
Haje



loknath wrote:
> Hi,
> I have huge satellite data stored in zipped files and unzipping would
> be costly for my hard drive. Is there a way IDL can unzip these files
> and read or read these files without unzipping in the hard drive.
> Thanks
Re: Read Zipped files [message #49917 is a reply to message #49915] Tue, 29 August 2006 21:05 Go to previous messageGo to next message
Mike Wallace is currently offline  Mike Wallace
Messages: 25
Registered: May 2006
Junior Member
> I don't think any of this would work in Windows but probably in OS X. Didn't
> someone have a Windows DLM that supported zipped files?

You're right. Windows doesn't have the same concept of named pipes that
*nix does, and due to OS X being rooted in BSD, it should support named
pipes.

As for the original question, if you're using gzip to compress the
files, IDL can read them by using the /COMPRESS flag to the OPENR, OPENU
or OPENW commands.

If you're files are zip files, the only option I know of is to unzip the
file before you use it. If you are using *nix, you can use the named
pipe approach as the file would be unzipped into memory rather than to
the hard drive. If you're using Windows, the only option I can think of
is to write a small program in another language that could handle the
unzip and stream the results out to shared memory. You'd then have to
make your IDL program read from shared memory instead of the file. It's
not the easiest thing in the world if you've never worked with shared
memory before.

When you said that it would "be costly for my hard drive," what do you
mean? Are you referring to all the files that would have to be written
back out to disk all the time or are you concerned with running out of
disk space?

-Mike
Re: Read Zipped files [message #49918 is a reply to message #49917] Tue, 29 August 2006 13:52 Go to previous messageGo to next message
mchinand is currently offline  mchinand
Messages: 66
Registered: September 1996
Member
In article <1156870907.629435.73050@i3g2000cwc.googlegroups.com>,
loknath <ladhikari@gmail.com> wrote:
> Hi,
> I have huge satellite data stored in zipped files and unzipping would
> be costly for my hard drive. Is there a way IDL can unzip these files
> and read or read these files without unzipping in the hard drive.
> Thanks
>

I'm not sure if this is the best way to do this, but on Linux (and other Unix
flavors?) you could use named pipes:

from the shell prompt (only has to be done once):

> mkfifo pipe_filename

then in IDL for each file you want to unzip and read (filename is a string
variable with containing the filename of zipped file you want to read):

spawn,'unzip -p '+filename+' >pipe_filename&'

<insert read commands here, using pipe_filename as if it were a regular file
containing your unzipped data>

If your files used gzip compression instead, you could just use the /COMPRESS
option for OPENR.

I don't think any of this would work in Windows but probably in OS X. Didn't
someone have a Windows DLM that supported zipped files?

Good luck,

--Mike



--
Michael Chinander, PhD
m-chinander@uchicago.edu
Department of Radiology
University of Chicago
Re: Read Zipped files [message #50089 is a reply to message #49895] Thu, 31 August 2006 05:13 Go to previous message
Haje Korth is currently offline  Haje Korth
Messages: 651
Registered: May 1997
Senior Member
Use a temp directory than you clear out after processing each file.
this is essentially what you would do with unzipping to RAM as well.

Of course you are free to look beyond the IDL prompt usage to join the
group of DLM programmers. :-)

loknath wrote:
> I am running out of hard drive space and didnt want to unzip the files
> but now I feel it is easier if I unzip the files in my hard drive.
> Thanks everyone
> Loknath
>
> Mike Wallace wrote:
>>> I don't think any of this would work in Windows but probably in OS X. Didn't
>>> someone have a Windows DLM that supported zipped files?
>>
>> You're right. Windows doesn't have the same concept of named pipes that
>> *nix does, and due to OS X being rooted in BSD, it should support named
>> pipes.
>>
>> As for the original question, if you're using gzip to compress the
>> files, IDL can read them by using the /COMPRESS flag to the OPENR, OPENU
>> or OPENW commands.
>>
>> If you're files are zip files, the only option I know of is to unzip the
>> file before you use it. If you are using *nix, you can use the named
>> pipe approach as the file would be unzipped into memory rather than to
>> the hard drive. If you're using Windows, the only option I can think of
>> is to write a small program in another language that could handle the
>> unzip and stream the results out to shared memory. You'd then have to
>> make your IDL program read from shared memory instead of the file. It's
>> not the easiest thing in the world if you've never worked with shared
>> memory before.
>>
>> When you said that it would "be costly for my hard drive," what do you
>> mean? Are you referring to all the files that would have to be written
>> back out to disk all the time or are you concerned with running out of
>> disk space?
>>
>> -Mike
  Switch to threaded view of this topic Create a new topic Submit Reply
Previous Topic: Weird Map Projection
Next Topic: drawing a ROI on a zoomed image

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ] [ PDF ]

Current Time: Fri Oct 10 16:24:15 PDT 2025

Total time taken to generate the page: 0.55868 seconds