comp.lang.idl-pvwave archive: archive » operating on tab-/comma-delimited files

Home » Public Forums » archive » operating on tab-/comma-delimited files

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

operating on tab-/comma-delimited files [message #85698]

Wed, 28 August 2013 09:56

Seb
Messages: 15
Registered: January 2006

Junior Member

Hi,

I am in the process of updating code I've inherited as I learn IDL, and
noticed that it uses the following workflow for fixing errors in
tab-/comma-delimited files:

1. Use file_search to generate a list of files to work on

2. Use FOR loop to iterate through each file

2.1 Create a float array with the same dimensions as input file
2.2 Load the data into a string array with 'openr' -> 'strarr' ->
'readf' -> 'close'
2.3 Loop through each line doing various tests/operations
2.4 Copy the fixed data into the float array in 2.1

3. Write the float array with 'openw' -> 'printf' -> 'close'

Based on what I'm reading, it seems this should be more efficient by
using 'read_ascii' and then fixing the errors with array indexing and
then writing the data back. Is this worth pursuing, or are there any
recommendations for a module I should know about?

Thanks,

--
Seb

Report message to a moderator

Re: operating on tab-/comma-delimited files [message #85699 is a reply to message #85698]

Wed, 28 August 2013 10:11

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

Seb writes:

>
> Hi,
>
> I am in the process of updating code I've inherited as I learn IDL, and
> noticed that it uses the following workflow for fixing errors in
> tab-/comma-delimited files:
>
> 1. Use file_search to generate a list of files to work on
>
> 2. Use FOR loop to iterate through each file
>
> 2.1 Create a float array with the same dimensions as input file
> 2.2 Load the data into a string array with 'openr' -> 'strarr' ->
> 'readf' -> 'close'
> 2.3 Loop through each line doing various tests/operations
> 2.4 Copy the fixed data into the float array in 2.1
>
> 3. Write the float array with 'openw' -> 'printf' -> 'close'
>
> Based on what I'm reading, it seems this should be more efficient by
> using 'read_ascii' and then fixing the errors with array indexing and
> then writing the data back. Is this worth pursuing, or are there any
> recommendations for a module I should know about?

Normally you step through as file like this reading a line at a time if
the problems with the file are more difficult to deal with than the
simple problems READ_ASCII can handle. We can't say without seeing the
file itself.

Cheers,

David

--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thue. ("Perhaps thou speakest truth.")

Report message to a moderator

Re: operating on tab-/comma-delimited files [message #85700 is a reply to message #85698]

Wed, 28 August 2013 10:21

wlandsman
Messages: 743
Registered: June 2000

Senior Member

I'd say the code is already efficient in that it reads and writes each file with a single I/O operation. Any place for improvement would be in step 2.3.

On Wednesday, August 28, 2013 12:56:27 PM UTC-4, Seb wrote:
> Hi,
>
>
>
> I am in the process of updating code I've inherited as I learn IDL, and
>
> noticed that it uses the following workflow for fixing errors in
>
> tab-/comma-delimited files:
>
>
>
> 1. Use file_search to generate a list of files to work on
>
>
>
> 2. Use FOR loop to iterate through each file
>
>
>
> 2.1 Create a float array with the same dimensions as input file
>
> 2.2 Load the data into a string array with 'openr' -> 'strarr' ->
>
> 'readf' -> 'close'
>
> 2.3 Loop through each line doing various tests/operations
>
> 2.4 Copy the fixed data into the float array in 2.1
>
>
>
> 3. Write the float array with 'openw' -> 'printf' -> 'close'
>
>
>
> Based on what I'm reading, it seems this should be more efficient by
>
> using 'read_ascii' and then fixing the errors with array indexing and
>
> then writing the data back. Is this worth pursuing, or are there any
>
> recommendations for a module I should know about?
>
>
>
> Thanks,
>
>
>
> --
>
> Seb

Report message to a moderator

Previous Topic:	delete all windows at once
Next Topic:	What subprogram? What parameters and keywords?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 15:07:56 PDT 2025

Total time taken to generate the page: 0.00560 seconds