| Re: Multiple journal files on linux [message #31204 is a reply to message #31108] |
Wed, 19 June 2002 09:21   |
![Paul Van Delst[1] is currently offline](theme/default/images/xoffline.png.pagespeed.ic.XRkd1fkXye.png "Paul Van Delst[1] is currently offline")
Paul Van Delst[1]
Messages: 1157
Registered: April 2002
|
Senior Member |
|
|
Ken Mankoff wrote:
>
> On 19 Jun 2002, Mark McGrath wrote:
>> This routine makes an `almost' unique file. An example:
>> Running IDL on a linux cluster, as we do here, if the machine clocks
>> are a litle out of sync it would be possible to over write the file
>> that had been opened already, no? (Not much info lost there though,
>> but the more serious case of the clocks being a lot out of sync then
>> large amounts of info in the journal file could be lost.)
>>
>> Any ideas, anyone?
>
> I ran into the problem of making a unique directory on a
> multiprocessor machine, where a timestamp may not suffice...
>
> Use the unique process ID. Each IDL session will have a unique PID, no
> matter how many processors and what the time is on each machine... I
> use a combination, the time (in msec), then a "_", then the PID.
>
> I believe you can still have multiple processes assigned the same time
> and PID if you are using networked computers (as opposed to 1 computer
> with multiple processors), but the chances, especially when using msec
> and not sec, drop to VERY low.
I do the following for multiple job submissions on a multi CPU machine in a shell script.
I didn't want the PID to be involved and just use a simple counter suffix. I've only got
access to 8 CPUS at once so I've never exceeded the (arbitrary) max value of 10
directories/files created at the same time.
Works quite well.
# ---------------------------
# Get the start date and time
# ---------------------------
LBLRUN_DATE=`date '+%Y%m%d'`
LBLRUN_TIME=`date '+%H%M%S%Z'`
# ----------------------------------------------------
# Create a root definition for the LBLRUN_TAG variable
# ----------------------------------------------------
ROOT_LBLRUN_TAG='_'`whoami`'_'${LBLRUN_DATE}'_'${LBLRUN_TIME }
# ------------------------------------------------------------ ----
# Make sure the directory doesn't already exist by suffixing
# the name with an _ and an integer identifier.
# Need to check this just in case the time and dates were assigned
# at the same time.
# ------------------------------------------------------------ ----
LBLRUN_SUFFIX=0
while :
do
# -- Suffix the directory with a number
LBLRUN_SUFFIX=`expr ${LBLRUN_SUFFIX} + 1`
LBLRUN_TAG=${ROOT_LBLRUN_TAG}'_'${LBLRUN_SUFFIX}
LBL_WORK=${LBL_RUN_ROOT}'/.lblrtm'${LBLRUN_TAG}
# -- Create the work directory
mkdir ${LBL_WORK}
# -- If successful, exit this loop
if [ $? -eq 0 ]; then
break
fi
# -- Otherwise keep trying to create a work directory (assuming
# -- the creation failed since it already exists).
# -- If we have reached a nesting of 10, maybe something is wrong?
if [ ${LBLRUN_SUFFIX} -gt 10 ]; then
echo "Error occurred creating work directory (based on .lblrtm${ROOT_LBLRUN_TAG})"
date
echo "${LBL_RUN_ROOT} directory listing follows:"
ls -laF ${LBL_RUN_ROOT}
exit 2
fi
done
--
Paul van Delst
CIMSS @ NOAA/NCEP/EMC Beer is good.
Ph: (301)763-8000 x7274 My wife.
Fax:(301)763-8545
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
