Reading Columnar Data Files

QUESTION: On the IDL newsgroup recently, Jason Li from the NASA Goddard Space Flight Center asked this question:

   >I have an ASCII text file that contains data in a nice tabular form,
   >
   >     0   28660    1827.1    72.7705  -158.8828   3388.0   22.3846   10.8545
   >     1   28661    1827.7    72.7701  -158.8752   3391.0   21.1213   10.6029
   >     2   28662    1828.3    72.7698  -158.8677   3394.0   19.8743   10.3546
   >     .
   >     .
   >     .
   >
   >I want to read them all and save into an array: data[8, numberOfLines]. But
   >I don't know numberOfLines in the file before hand. What is the most efficient
   >way to find that out?

   >On UNIX, I can pass number of lines information back from "wc" command. Of
   >course the same code won't work a on a Mac or Windows machine. Please help.

ANSWER: Jason got all kinds of good answers to his question. I've compiled and edited a few of the responses below. I caution you that I haven't tested these programs extensively myself, so if you have problems or questions with them I would encourage you to contact the authors directly.

The first response was from Paul Krummel (paul.krummel@dar.csiro.au) with the CSIRO Atmospheric Research Division in Australia. He writes:

Below are two functions, File_Line and File_Size that I use regularly. I have used them on both Windows and UNIX and they work fine. They should be platform independent and I find them to be efficient! They were originally written by R. Bauer and were modified by me.

There was actually a third file, named Type, that was also required. I've placed all three files in a file named file_line.pro, which you can download here.

The next response was from Helge Rebhan (Helge.Rebhan@gmx.net), who wrote from Germany. Helge offered a program named ReadTab, which he claims "reads any regular formatted table in a 2D floating point array". I've saved the file as readtab.pro for you to download here. Helge cautions and apologizes for the comments, which are written in German. :-)

Finally, Liam Gumley (liamg@ssec.wisc.edu) and Paul van Delst (paul.vandelst@ssec.wisc.edu) wrote in to recommend a program named DDRead, which they swear by and which was written by Fred Knight (knight@ll.mit.edu) at the MIT Lincoln Laboratory. You can download Fred's program from the User Contributed Library at Research System's anonymous FTP server:

   ftp://ftp.rsinc.com/pub/user_contrib/knight/ddread.pro

Paul cautions that DDRead uses another of Fred's programs named NLines in which the total number of lines in the file is initially set to zero with a command like this:

   n_lines = 0

If you have a great many lines in your file, you may want to make this a long integer or you will have problems:

   n_lines = 0L

The DDRead program uses other routines that are also found in Fred's library at the FTP address above. Read the documentation to DDRead to find out which files you need.

In IDL 5.6, RSI introduced a new function named File_Lines that can also be used to return the number of lines in an ASCII text file. This is probably the best way to get this information currently. Note, however, that the file does have to be read to compute this value.