| Re: Read large ascii file quickly [message #74290 is a reply to message #74289] |
Fri, 07 January 2011 04:01   |
Paul Magdon
Messages: 8
Registered: January 2011
|
Junior Member |
|
|
Dear All,
I tried some of your ideas:
1.) @ pp data=reform((read_csv('file.dat')).(5),5000,5000)
works fine but takes the same time as with the loop. (David was expecting that :))
2.)@ David ReadF, lun, data, Format='(44x,I0)'
does not work as the dataset also includes negative values where 44x is not correct any longer
3.) @ MC Also, I think you can read the data in one go into a single big array -so no loop needed as EOF stops the read with an error message -you can then
catch the EOF error generated
How to do that?
So, still I didn't have a faster solution.
X.) Yeah, people who put big data sets in columns should be shot
I totally agree but since I need to process so called arff files there is no other way to do it. Our does anyone knows an implementation of RandomForest classifiers in IDL?
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
