comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » readcol procedure

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: readcol procedure [message #32150 is a reply to message #32020]

Fri, 13 September 2002 10:43

R.Bauer
Messages: 1424
Registered: November 1998

Senior Member

Pavel A. Romashkin wrote:
> Reimar Bauer wrote:
>
>> if you use the eof method you have to read line by line. As you know idl
>> is an array orientated language so reading in an array is much faster.
>> It's really fast. If you have only 10 lines it doesn't matter but
>> sometimes we got datafiles of nearly 100.000 lines. In this case it is
>> very important.
>
>
> I am sorry to disagree.
> I routinely read large (60k-200k rows) ASCII files with unknown number
> of lines. I always use large arrays to read into and never ever use EOF
> with line by line reading.
> All I have to do is to catch I/O error in case my buffer array is too
> big as my reading approaches the end of file, then look up what size it
> should have actually been, resize the buffer, then read the last portion
> of file only. Reading a file with 80k lines using this method takes
> about 0.1 s.
> Take a look:
> http://www.ainaco.com/idl/idl_library/read_ascii_columns.pro
> Cheers,
> Pavel

I don't understand where you are disagree.
I will try a comparison with the usb device and no file cache or
how should comparisons be done?

In principle you are using eof by an on_ioError condition.
This is nearly the same. Or not ?
You are reading portions and if you know the number of rows of the file
you can read this in one portions without an error.
Then it is nearly the same to my routine. The difference is only to my
routine that's it only need the filename and no column input and header
lines could be more or less then one line.

is this right ?

Reimar

--
Reimar Bauer

Institut fuer Stratosphaerische Chemie (ICG-I)
Forschungszentrum Juelich
email: R.Bauer@fz-juelich.de
------------------------------------------------------------ -------
a IDL library at ForschungsZentrum Juelich
http://www.fz-juelich.de/icg/icg-i/idl_icglib/idl_lib_intro. html
============================================================ =======

Report message to a moderator

[Message index]

		readcol procedure By: shih I Chun on Tue, 10 September 2002 10:15
		Re: readcol procedure By: Liam E. Gumley on Thu, 12 September 2002 08:43
		Re: readcol procedure By: R.Bauer on Thu, 12 September 2002 11:39
		Re: readcol procedure By: David Fanning on Thu, 12 September 2002 08:59
		Re: readcol procedure By: R.Bauer on Wed, 11 September 2002 23:42
		Re: readcol procedure By: David Fanning on Wed, 11 September 2002 14:12
		Re: readcol procedure By: Wayne Landsman on Wed, 11 September 2002 07:54
		Re: readcol procedure By: R.Bauer on Tue, 10 September 2002 23:33
		Re: readcol procedure By: Don J Lindler on Tue, 10 September 2002 13:30
		Re: readcol procedure By: R.Bauer on Sun, 15 September 2002 08:16
		Re: readcol procedure By: Pavel A. Romashkin on Fri, 13 September 2002 12:30
		Re: readcol procedure By: R.Bauer on Fri, 13 September 2002 10:43
		Re: readcol procedure By: Pavel A. Romashkin on Fri, 13 September 2002 09:36
		Re: readcol procedure By: David Fanning on Fri, 13 September 2002 08:50
		Re: readcol procedure By: Pavel A. Romashkin on Fri, 13 September 2002 08:39
		Re: readcol procedure By: R.Bauer on Thu, 12 September 2002 14:05
		Re: readcol procedure By: Liam E. Gumley on Thu, 12 September 2002 12:11

Previous Topic:	duplicate .pro filenames
Next Topic:	Re: duplicate .pro filenames

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 02:09:45 PDT 2025

Total time taken to generate the page: 0.80527 seconds