comp.lang.idl-pvwave archive: archive » ZLIB

Home » Public Forums » archive » ZLIB_UNCOMPRESS limited to 4GB?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

ZLIB_UNCOMPRESS limited to 4GB? [message #89979]

Mon, 12 January 2015 12:03

markb77
Messages: 217
Registered: July 2006

Senior Member

hi,

I am trying to access data that is stored in a zlib compressed format in a set of existing files. The data size in some of the files is > 4GB.

For the smaller files, I can use ZLIB_UNCOMPRESS to access the data successfully. However, when I try ZLIB_UNCOMPRESS on the larger datasets, IDL crashes hard (the program just closes).

I've tested ZLIB_COMPRESS with large datasets, and this throws an error when the input data is > 4GB.

So, are both of these routines limited to 4GB data? If so, why?

IDL Version 8.3, Microsoft Windows (Win32 x86_64 m64). (c) 2013, Exelis Visual Information Solutions, Inc.

My machine has 32GB of RAM... and I have also tested this on IDL 8.4

Any ideas on how to decompress this data in IDL?

thanks
Mark

Report message to a moderator

Re: ZLIB_UNCOMPRESS limited to 4GB? [message #90678 is a reply to message #89979]

Wed, 25 March 2015 07:20

markb77
Messages: 217
Registered: July 2006

Senior Member

hi,

The 4GB limit is inherent to the ZLIB compress() and uncompress() routines. Instead, in order to access data this large I would need to use ZLIB inflate().

Does anyone have IDL code or a DLM which uses ZLIB inflate() ?

thanks
Mark

Report message to a moderator

Re: ZLIB_UNCOMPRESS limited to 4GB? [message #90679 is a reply to message #90678]

Wed, 25 March 2015 09:52

Heinz Stege
Messages: 189
Registered: January 2003

Senior Member

On Wed, 25 Mar 2015 07:20:42 -0700 (PDT), superchromix wrote:

> The 4GB limit is inherent to the ZLIB compress() and uncompress() routines. Instead, in order to access data this large I would need to use ZLIB inflate().
>
> Does anyone have IDL code or a DLM which uses ZLIB inflate() ?
>
I don't know, if this helps: 7z from http://www.7-zip.org/ has a
command line version that can be used together with IDLs spawn
procedure. The 7z extract command can extract files from a zip-file
greater than 4 GB.

I did a test with a 0.7 GB file in a 6.6 GB zip file on my Windows PC.

Cheers, Heinz

Report message to a moderator

Re: ZLIB_UNCOMPRESS limited to 4GB? [message #90682 is a reply to message #90679]

Thu, 26 March 2015 05:07

markb77
Messages: 217
Registered: July 2006

Senior Member

Thanks, but unfortunately this is not a ZIP file - it is compressed data stored within another file format.
Mark

Report message to a moderator

Previous Topic:	Information about scroll action - new position/offset information?
Next Topic:	Draw a circle enclosed to the other

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 09:10:53 PDT 2025

Total time taken to generate the page: 0.00503 seconds