comp.lang.idl-pvwave archive: archive » Re: slow file-handling

Home » Public Forums » archive » Re: slow file-handling

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: slow file-handling [message #6918 is a reply to message #6913]

Tue, 03 September 1996 00:00

James Tappin
Messages: 54
Registered: December 1995

Member

Jorn Helbert wrote:
>
> Hi,
> I have a problem reading in a large data-file
> the file consist of 57422 rows with the following format
> t foo foo foo bx by bz
>
> all double floating and I need the foo-values not. if i just read it
> line by line it takes nearly ten minutes on a sparc twenty. my read
> block is the following
> b_temp = double([0.,0.,0.])
> t_temp = [0.0]
> WHILE NOT eof(u1) DO BEGIN
> readf,u1,t_temp,foo,foo,foo,b_temp
> t_u = [t_u,t_temp]
> b_u = [b_u,b_temp]
> END
>
> I don't know before how long the file is. so I have to use the temporary
> values.
> Any ideas how to speed this up?
>
> cheers
> jorn
>
> --
> Joern Helbert
>
> Space and atmospheric physics group
> Imperial College - London
> phone: 0171 594-7764

One possibility on a unix box is to spawn "wc" to see how many lines there are
(if you need VMS portability, then select that or FSTAT according to the
operating system [!version.os] -- I don't know about PC's & macs)

So you'd have something like:

spawn, 'wc '+filename, res
nrecs=fix(res) ; Number of lines is first field in WC output

inarr =dblarr(7,nrecs)

openr, iu, /get, filename
readf, iu, inarr

t_u = transpose(float(inarr(0,*))) ; Assuming you really need to go
; back to single precision, transpose
; Otherwise it will be a (1,n) array
b_u = inarr(4:6, *)

--
+------------------------+---------------------------------- --+---------+
| James Tappin, | School of Physics & Space Research | O__ |
| sjt@star.sr.bham.ac.uk | University of Birmingham | -- \/` |
| Ph: 0121-414-6462. Fax: 0121-414-3722 | |
+----------------------------------------------------------- --+---------+

Report message to a moderator

[Message index]

		Re: slow file-handling By: Marty Ryba on Tue, 03 September 1996 00:00
		Re: slow file-handling By: Marty Ryba on Tue, 03 September 1996 00:00
		Re: slow file-handling By: jlaw on Tue, 03 September 1996 00:00
		Re: slow file-handling By: Hermann Mannstein on Tue, 03 September 1996 00:00
		Re: slow file-handling By: James Tappin on Tue, 03 September 1996 00:00
		Re: slow file-handling By: David Theil on Mon, 02 September 1996 00:00
		Re: slow file-handling By: brian.jackel on Wed, 04 September 1996 00:00

Previous Topic:	passing parameters indirectly
Next Topic:	IDL Triangulate/TriGrid >>> Wave 6.0 ?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sun Oct 12 19:19:40 PDT 2025

Total time taken to generate the page: 2.40173 seconds