| Curious FindFile behavior [message #13589] |
Wed, 25 November 1998 00:00  |
lesht
Messages: 1
Registered: November 1998
|
Junior Member |
|
|
I'm running IDL5.1 on an SGI (IRIX6.2). I'm using FindFile to create
an array of data filenames that I can pass to some processing code.
There are (as it turns out) 515 files in the directory I'm searching.
The filenames are all of the form xxxxxxxxx.xx.yymmdd.hhmmss.cdf. I
get the following from FindFile-
IDL> files=FindFile('*.*', COUNT=nf)
IDL> PRINT, nf
0
IDL> files=FindFile('*.cdf', COUNT=nf)
IDL> PRINT, nf
0
IDL> files=FindFile('*98*.cdf', COUNT=nf)
IDL> PRINT, nf
0
IDL> files=FindFile('*980*.cdf', COUNT=nf)
IDL> PRINT, nf
425
IDL> files=FindFile('*981*.cdf', COUNT=nf)
IDL> PRINT, nf
90
I don't see why cases 2 and (especially) 3 didn't result in nf=515.
Can anyone explain this? Thanks.
Barry Lesht
bmlesht@anl.gov
|
|
|
|
| Re: Curious FindFile behavior [message #13767 is a reply to message #13589] |
Fri, 27 November 1998 00:00  |
Martin Schultz
Messages: 515
Registered: August 1997
|
Senior Member |
|
|
Barry Lesht wrote:
> I'm running IDL5.1 on an SGI (IRIX6.2). I'm using FindFile to create
> an array of data filenames that I can pass to some processing code.
> There are (as it turns out) 515 files in the directory I'm searching.
> The filenames are all of the form xxxxxxxxx.xx.yymmdd.hhmmss.cdf. I
> get the following from FindFile-
>
> IDL> files=FindFile('*.*', COUNT=nf)
> IDL> PRINT, nf
> 0
> [...]
>
> I don't see why cases 2 and (especially) 3 didn't result in nf=515.
> Can anyone explain this? Thanks.
>
> Barry Lesht
> bmlesht@anl.gov
Yup,
this seems to be an SGI related problem. I have never heard of other
machines with this behaviour. I had once contacted RSI about a similar
problem, and they replied it may have to do with the shell expansion (in
my case I was getting into trouble using '~' identifiers even in small
directories). It would probbaly help if you could forward this bug
report to RSI -- maybe they will fix it at some point...
Please find below a third variant of a routine that has been designed to
overcome the problem you posted,
Regards,
Martin.
--
------------------------------------------------------------ -------
Dr. Martin Schultz
Department for Engineering&Applied Sciences, Harvard University
109 Pierce Hall, 29 Oxford St., Cambridge, MA-02138, USA
phone: (617)-496-8318
fax : (617)-495-4551
e-mail: mgs@io.harvard.edu
Internet-homepage: http://www-as.harvard.edu/people/staff/mgs/
------------------------------------------------------------ -------
; $Id: mfindfile.pro,v 1.1 1998/10/09 19:53:32 mgs Exp $
;----------------------------------------------------------- --
;+
; NAME:
; MFINDFILE
;
; PURPOSE:
; find all the files that match a given specification.
; On our system, the IDL findfile function does not
; work correctly!!
;
; CATEGORY:
; System routines
;
; CALLING SEQUENCE:
; listing = MFINDFILE(filemask)
;
; INPUTS:
; FILEMASK -> a path and filename specification to look
; for.
;
; KEYWORD PARAMETERS:
; none
;
; OUTPUTS:
; A string list containing all the files that match the
; specification.
;
; SUBROUTINES:
;
; REQUIREMENTS:
;
; NOTES:
; Spawns a unix ls -1 command !
;
; EXAMPLE:
; list = mfindfile('~mgs/terra/chem1d/code/*.f')
;
; ; returns all fortran files in Martin's chem1d directory.
;
; MODIFICATION HISTORY:
; mgs, 14 Sep 1998: VERSION 1.00
;
;-
; Copyright (C) 1998, Martin Schultz, Harvard University
; This software is provided as is without any warranty
; whatsoever. It may be freely used, copied or distributed
; for non-commercial purposes. This copyright notice must be
; kept with any copy of this software. If this software shall
; be used commercially or sold as part of a larger package,
; please contact the author to arrange payment.
; Bugs and comments should be directed to mgs@io.harvard.edu
; with subject "IDL routine mfindfile"
;----------------------------------------------------------- --
function mfindfile,mask
if (!version.os_family eq 'unix') then begin
; make my own findfile
path = extract_path(mask,filename=fname)
path = expand_path(path)
newpath = path+fname
; print,'fname:',fname,' path:',path,' newpath:',newpath
command = 'ls -1'
cstr = command+' '+newpath
spawn,cstr,listing
return,listing
endif else begin ; other OS - use IDL's original
return,findfile(mask)
endelse
end
|
|
|
|
| Re: Curious FindFile behavior [message #13774 is a reply to message #13589] |
Thu, 26 November 1998 00:00 |
mallors
Messages: 76
Registered: November 1997
|
Member |
|
|
In article <73holl$fts@milo.mcs.anl.gov>,
lesht@ANLER.ER.ANL.GOV (Barry Lesht) writes:
> I'm running IDL5.1 on an SGI (IRIX6.2). I'm using FindFile to create
> an array of data filenames that I can pass to some processing code.
> There are (as it turns out) 515 files in the directory I'm searching.
> The filenames are all of the form xxxxxxxxx.xx.yymmdd.hhmmss.cdf. I
> get the following from FindFile-
>
> IDL> files=FindFile('*.*', COUNT=nf)
> IDL> PRINT, nf
> 0
> IDL> files=FindFile('*.cdf', COUNT=nf)
> IDL> PRINT, nf
> 0
> IDL> files=FindFile('*98*.cdf', COUNT=nf)
> IDL> PRINT, nf
> 0
> IDL> files=FindFile('*980*.cdf', COUNT=nf)
> IDL> PRINT, nf
> 425
> IDL> files=FindFile('*981*.cdf', COUNT=nf)
> IDL> PRINT, nf
> 90
>
> I don't see why cases 2 and (especially) 3 didn't result in nf=515.
> Can anyone explain this? Thanks.
This problem should really be addressed by RSI. What is
happening, I believe, is that FINDFILE is spawning a shell to
get the file list via something like "ls". However, the shell
has limits on the byte length of command line arguments - for
long lists, the command will fail. RSI should really be
using another UNIX tool meant for this job - that is the
"find" command. See the UNIX man pages exec(2), find(1),
xargs(1) for more info.
I have a program FINDFILES on my web page that should work
for you, and also "fixes" searching for files recursively.
See http://cspar.uah.edu/~mallozzir/idl/idl.html
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~
Robert S. Mallozzi 256-544-0887
Mail Code ES 84
Work: http://www.batse.msfc.nasa.gov/ Marshall Space Flight Center
Play: http://cspar.uah.edu/~mallozzir/ Huntsville, AL 35812
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~
|
|
|
|
| Re: Curious FindFile behavior [message #13776 is a reply to message #13589] |
Thu, 26 November 1998 00:00 |
Nigel Wade
Messages: 286
Registered: March 1998
|
Senior Member |
|
|
Barry Lesht wrote:
:> I'm running IDL5.1 on an SGI (IRIX6.2). I'm using FindFile to create
:> an array of data filenames that I can pass to some processing code.
:> There are (as it turns out) 515 files in the directory I'm searching.
:> The filenames are all of the form xxxxxxxxx.xx.yymmdd.hhmmss.cdf. I
:> get the following from FindFile-
:>
:> IDL> files=FindFile('*.*', COUNT=nf)
:> IDL> PRINT, nf
:> 0
:> IDL> files=FindFile('*.cdf', COUNT=nf)
:> IDL> PRINT, nf
:> 0
:> IDL> files=FindFile('*98*.cdf', COUNT=nf)
:> IDL> PRINT, nf
:> 0
:> IDL> files=FindFile('*980*.cdf', COUNT=nf)
;> IDL> PRINT, nf
:> 425
:> IDL> files=FindFile('*981*.cdf', COUNT=nf)
:> IDL> PRINT, nf
:> 90
:>
:> I don't see why cases 2 and (especially) 3 didn't result in nf=515.
:> Can anyone explain this? Thanks.
:>
:> Barry Lesht
:> bmlesht@anl.gov
We had a problem on our system when the filesystem holding /tmp
was short of room.
I discovered that findfile was creating a temporary file in /tmp
containing the matched filenames. When this list was large and file
filesystem containg /tmp didn't have room to hold it a truncated list
of filenames was returned. There was no error indication.
The number of matched files also varied with how much space was
available in /tmp.
Have a look and see how much space is left in /tmp.
Or was it /var/tmp? Now I'm not sure which it was...
Check both, it was one of the two.
--
-----------------------------------------------------------
Nigel Wade, System Administrator, Space Plasma Physics Group,
University of Leicester, Leicester, LE1 7RH, UK
E-mail : nmw@ion.le.ac.uk
Phone : +44 (0)116 2523568, Fax : +44 (0)116 2523555
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
