| Re: error reading a large number of binary files [message #53735 is a reply to message #53600] |
Mon, 23 April 2007 07:24  |
![Paul Van Delst[1] is currently offline](theme/default/images/xoffline.png.pagespeed.ic.XRkd1fkXye.png "Paul Van Delst[1] is currently offline")
Paul Van Delst[1]
Messages: 1157
Registered: April 2002
|
Senior Member |
|
|
mark@atmos.colostate.edu wrote:
> Is there any particular reason why IDL would struggle reading a really
> large number of binary files? I'm trying to compute a number of
> statistics from the ECWMF ERA40 reanalysis 4*daily data, which is a
> HUGE amount of data. The data was originally in Grib format, which I
> converted to ieee binary with the wgrib program. Each file contains a
> month's worth of data, which has 23 pressure levels. In trying to
> track down the cause of the error I'm getting, I've pared the program
> down to what you can see below (just compute the long-term average
> zonal wind). FWIW, I'm running this on a Mac PPC Quad G5 with 8GB of
> RAM, and it's IDL 6.3.
>
> What happens is that I get an error message like this:
>
> % READU: Corrupted f77 unformatted file detected. Unit: 100, File:
> daily_4/u196710.bin
> % Execution halted at: READUDAILY 19 /Users/mark/Datasets/
> era40/readudaily.pro
> % $MAIN$
>
> And on repeated trials, the error always occurs in a different file.
> And I know that none of the files are actually corrupted.
> Interestingly, I wrote a similar program in fortran and using the
> NAGWare compiler it has similar problems, but it worked successfully
> using the xlf fortran compiler. However, I'd prefer to have it work
> in IDL, but I cannot for the life of me figure out what I'm doing
> wrong.
>
> Any ideas?
Are the original files little or big endian?
On what machine did you create the "new" files? A little or big endian machine? (Your mac
is big endian, right?)
On what machines did you test the fortran program? All your NAG and xlf compilers on you mac?
What happens when you use the /SWAP_ENDIAN keyword in your OPENR?
What does the binary keyword do?
> openr, lun, infiles[i], /get_lun, /binary, /f77_unformatted
I can't find a reference to it in the docs.
cheers,
paulv
--
Paul van Delst Ride lots.
CIMSS @ NOAA/NCEP/EMC Eddy Merckx
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
