| Re: Very slow IDL vs Matlab (ascii file reading) [message #36893 is a reply to message #36891] |
Tue, 04 November 2003 23:11   |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
Marcin Jakubowski wrote:
> Hi all,
> I've tried to read an ASCII file, which is composed of one very long row
> (660*496) double precision numbers, each of them delimited with
> tabulator. In Matlab 6.5 I am using small program
>
> =============================================
> fid = fopen('Mydata.dat');
> data = fscanf(fid,'%g',[660,496]);
> fclose(fid)
> =============================================
>
> and it takes about one second to read the file. If I try to do similar
> in IDL 6.0
>
> =============================================
> data = FltArr(660, 496)
> OpenR, lun, 'Mydata.dat', /Get_Lun
> ReadF, lun, data
> Free_Lun, lun
> =============================================
>
> then it takes about 20 minutes (!!!) to read the same file. What causes
> the problem? Unfortunately I need to use the IDL as it is a part of huge
> code written in IDL. Is it any chance to shorten that time?
>
>
> Many thanks in advance,
> Marcin
>
> P.s. I've performed checks on PC and Linux machines and the outcomes are
> similar.
My previous examples does not have all data in one line!!!
(It's always easier to check if a complete example is provided to us)
Here is an example to create the data file.
x=make_array(/double,value=999D,660L*496L)
w=17l*660L*496L
openw,lun,/get_l,'Mydata.dat',width=w
printf,lun,x
free_lun,lun
By reading this data I got the same problem as described above.
This could be a bug in width. Any ideas?
I would suggest to use a shell command to split the lines after 1000 numbers
in new lines. On linux this could be done by sed.
Reimar
--
Forschungszentrum Juelich
email: R.Bauer@fz-juelich.de
http://www.fz-juelich.de/icg/icg-i/
============================================================ ======
a IDL library at ForschungsZentrum Juelich
http://www.fz-juelich.de/icg/icg-i/idl_icglib/idl_lib_intro. html
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
