| Re: Running IDL from cron [message #23826] |
Fri, 23 February 2001 15:53  |
thompson
Messages: 584
Registered: August 1991
|
Senior Member |
|
|
Vapuser <vapuser@catspaw.jpl.nasa.gov> writes:
> Cron uses a crippled environment in the Bourne shell, it defines
> only a limited number of environmental variables.
I've been able to use a fully functional C shell in my cron jobs, by the simple
expedient of explicitly telling it what shell to run. This is done by putting
#!/bin/csh
as the first line of the script. For example, the extremely simple script
#!/bin/csh
printenv
when run as a cron job, demonstrates that my .cshrc file is being automatically
executed. On the other hand, my .login file is not. To do that, I would have
to use
#!/bin/csh
source .login
printenv
which works, although I do get a "not a terminal" error. For myself, it's not
necessary to run .login--all setup is done in .cshrc.
A valid C-shell script to run IDL would be
#!/bin/csh
idl << done
print,'Hello world'
done
I do have a cron job which uses IDL in just this fashion, which nightly creates
some images for a web page. As somebody has already pointed out before me, the
way to do this in a cron job is to use the Z buffer.
William Thompson
|
|
|
|
|
|
| Re: Running IDL from cron [message #23832 is a reply to message #23831] |
Fri, 23 February 2001 13:06  |
Vapuser
Messages: 63
Registered: November 1998
|
Member |
|
|
Gwyn Fireman <Gwyn.Fireman@gsfc.nasa.gov> writes:
> Hi, everyone -
>
> With Deja gone and Google barely functional I have to ask here what I'm
> sure has been addressed before:
>
> How do you run IDL routines from cron - while maintaining IDL_PATH and
> other environment variables?
>
> I have had no trouble running my idl routines using batch or at, when
> starting in the same directory as the main-level routine. In that case
> my environment is copied to the spawned process.
>
> Now I have the need to run these from cron. I have been able to write
> scripts to cd to the appropriate directory and to run my .cshrc (where
> my IDL_PATH and other environment variables are defined), but keep
> running into problems. The latest is than .cshrc bombs on a
> system-level env that apparently isn't defined when running from cron.
>
> So can anyone point me in the right direction?
> I'm running { alpha OSF unix 5.3 Nov 11 1999}.
>
> Many thanks in advance,
> Gwyn Fireman
Cron uses a crippled environment in the Bourne shell, it defines
only a limited number of environmental variables.
from the crontab manpage
The shell is invoked from your $HOME directory with an arg0 of sh. Users
who desire to have their .profile executed must explicitly do so in the
crontab file. Cron supplies a default environment for every shell,
defining HOME, LOGNAME, USER, SHELL(=/bin/sh),
PATH(=/usr/sbin:/usr/bsd/:/usr/bin:/bin/:/etc:/usr/etc), and TZ.
You have to explicitly source whatever files you need to define the
correct env. Usually, one does this by sourcing their .profile as
the first part of a multi-part command, but you can source any file
if you want to restrict the cronjob environment in some way. To
execute two or more commands in the same shell, you put them inside
parantheses separated by a semi-colon. When run the first command
ensures the correct environment for the second.
Like this.
0 * * * * (. /path/to/rc/file; /path/to/script) > some-file 2>&1;
(Note the `.' there!)
If your .profile sets up your environment and you don't think you
need to restrict the cron job in anyway, simply source it as the
first part of each crontab entry. e.g.
0 * * * * (. $HOME/.profile ; /path/to/script) > some-file 2>&1;
(`some-file' can be /dev/null if you desire. If nothing is there and
there is any output to stdout/stderr, crontab mails it to you)
If you're using the csh, you can convert all the commands to their
Bourne shell equivalents. I think you'll find this easier in the
long run than trying to figure out how to correctly configure a Bourne
shell using Csh configuration files.
This does *not* mean, by the way, that you have to convert *all* the
scripts you call to any particular shell! All my cron jobs call perl
scripts as the second command but they could just as easily run csh
scripts. All that's being done with the first script is to configure
the environment.
If you need help with the differences between Bourne and CSH
configuration syntax, I'm sure there's someone around you who can
help.
All of my automated processing for my web page
(http://haifung.jpl.nasa.gov) is done by cronjobs with a combination
of perl scripts calling IDL using temporary files constructed on the
fly by the perl scripts. I just source $HOME/.profile, which then
sources other .rc files, as the first thing in each crontab
entry. By the way, as my native shell, I run the tcsh, so I'm
sympathetic to csh users.
The only other problem is the matter of the display, which you can
get around, as Liam(?) pointed out, by using the Z buffer. No problem
if you're doing gifs or grayscale jpegs. But it's an inconvenient
but not insurmountable difficulty if you're doing true-color
graphics. The solution is to operate on the indivudual color planes
individually in the Z buffer and then assemble the image at the end
and write it out. It's harder to do, but it isn't impossible. This
is precisely how I make the truecolor jpegs on haifung.
In fact, were it not for a problem caused by an SGI utility, which
is necessary to my processing, I wouldn't even need to be logged
into the console for it to run. As it is, I have to be or the
animation processing will timeout waiting for a connection to the X
server when this SGI utility (dmconvert) goes to convert the 60 .gif
files into a quicktime movie. The words I spoke when I found out
that despite spending a week ridding my IDL code of all connections
to the X server only to discover that it was the blackbox of the SGI
utility that was the fatal stumbling block to my goal of complete
'X' independence are not reproducible in polite company.
whd
--
William Daffer: 818-354-0161: William.Daffer@jpl.nasa.gov
|
|
|
|
|
|
| Re: Running IDL from cron [message #23857 is a reply to message #23856] |
Thu, 22 February 2001 23:34 |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
"Liam E. Gumley" wrote:
Dear Liam,
write_png instead of write_gif ...
cheers
Reimar
>
> Gwyn Fireman wrote:
>> How do you run IDL routines from cron - while maintaining IDL_PATH and
>> other environment variables?
>>
>> I have had no trouble running my idl routines using batch or at, when
>> starting in the same directory as the main-level routine. In that case
>> my environment is copied to the spawned process.
>>
>> Now I have the need to run these from cron. I have been able to write
>> scripts to cd to the appropriate directory and to run my .cshrc (where
>> my IDL_PATH and other environment variables are defined), but keep
>> running into problems. The latest is than .cshrc bombs on a
>> system-level env that apparently isn't defined when running from cron.
>
> Craig has given you a good start in a previous post. The other item
> which often causes problems when running IDL via cron is that you can't
> use graphics windows (unless you are prepared to leave a login session
> running all the time).
>
> The solution is to use the Z-buffer, which allows you to emulate the
> functionality of an 8-bit graphics window. Let's say you want to call
> this procedure (myplot.pro):
>
> ;---
> PRO MYPLOT, X
> plot, x
> END
> ;---
>
> You'll need an IDL script which sets up the Z-buffer, calls the
> procedure, and saves the resulting image (script.pro):
>
> ;---
> ;- Set up Z buffer for 8-bit graphics
> set_plot, 'Z'
> device, z_buffering=0, set_resolution=[640, 480], $
> set_colors=256, set_character_size=[10, 12]
>
> ;- Run my plotting procedure
> myplot, findgen(10)
>
> ;- Save the result to a GIF file
> tvlct, r, g, b, /get
> write_gif, 'myplot.gif', tvrd(), r, g, b
write_png, 'myplot.png', tvrd(), r, g, b
>
> ;- Exit IDL
> exit
> ;---
>
> Finally, you need a shell script to configure and run IDL (script.csh):
>
> #---
> #!/bin/csh
> cd $HOME/myplot
> source /usr/local/rsi/idl/bin/idl_setup
> setenv IDL_PATH ${IDL_PATH}:+\${IDL_DIR}/user_contrib/jhu_apl
> setenv IDL_STARTUP
> idl script.pro >& script.out
> #---
>
> Make sure the shell script has execute permission, and you're done. Test
> the script to make sure it works:
>
> % ./script.csh
>
> and you should get the following output in script.out:
>
> IDL Version 5.3 (IRIX mipseb). (c) 1999, Research Systems, Inc.
> Installation number: 13221.
> Licensed for use by: Space Science & Engineering Cntr, Univ of WI
>
> % Compiled module: MYPLOT.
> % Compiled module: WRITE_GIF.
> % Loaded DLM: GIF.
>
> Cheers,
> Liam.
> http://cimss.ssec.wisc.edu/~gumley/
--
Reimar Bauer
Institut fuer Stratosphaerische Chemie (ICG-1)
Forschungszentrum Juelich
email: R.Bauer@fz-juelich.de
http://www.fz-juelich.de/icg/icg1/
=============================================
a IDL library at ForschungsZentrum J�lich
http://www.fz-juelich.de/icg/icg1/idl_icglib/idl_lib_intro.h tml
http://www.fz-juelich.de/zb/text/publikation/juel3786.html
|
|
|
|
| Re: Running IDL from cron [message #23859 is a reply to message #23857] |
Thu, 22 February 2001 11:57 |
Liam E. Gumley
Messages: 378
Registered: January 2000
|
Senior Member |
|
|
Gwyn Fireman wrote:
> How do you run IDL routines from cron - while maintaining IDL_PATH and
> other environment variables?
>
> I have had no trouble running my idl routines using batch or at, when
> starting in the same directory as the main-level routine. In that case
> my environment is copied to the spawned process.
>
> Now I have the need to run these from cron. I have been able to write
> scripts to cd to the appropriate directory and to run my .cshrc (where
> my IDL_PATH and other environment variables are defined), but keep
> running into problems. The latest is than .cshrc bombs on a
> system-level env that apparently isn't defined when running from cron.
Craig has given you a good start in a previous post. The other item
which often causes problems when running IDL via cron is that you can't
use graphics windows (unless you are prepared to leave a login session
running all the time).
The solution is to use the Z-buffer, which allows you to emulate the
functionality of an 8-bit graphics window. Let's say you want to call
this procedure (myplot.pro):
;---
PRO MYPLOT, X
plot, x
END
;---
You'll need an IDL script which sets up the Z-buffer, calls the
procedure, and saves the resulting image (script.pro):
;---
;- Set up Z buffer for 8-bit graphics
set_plot, 'Z'
device, z_buffering=0, set_resolution=[640, 480], $
set_colors=256, set_character_size=[10, 12]
;- Run my plotting procedure
myplot, findgen(10)
;- Save the result to a GIF file
tvlct, r, g, b, /get
write_gif, 'myplot.gif', tvrd(), r, g, b
;- Exit IDL
exit
;---
Finally, you need a shell script to configure and run IDL (script.csh):
#---
#!/bin/csh
cd $HOME/myplot
source /usr/local/rsi/idl/bin/idl_setup
setenv IDL_PATH ${IDL_PATH}:+\${IDL_DIR}/user_contrib/jhu_apl
setenv IDL_STARTUP
idl script.pro >& script.out
#---
Make sure the shell script has execute permission, and you're done. Test
the script to make sure it works:
% ./script.csh
and you should get the following output in script.out:
IDL Version 5.3 (IRIX mipseb). (c) 1999, Research Systems, Inc.
Installation number: 13221.
Licensed for use by: Space Science & Engineering Cntr, Univ of WI
% Compiled module: MYPLOT.
% Compiled module: WRITE_GIF.
% Loaded DLM: GIF.
Cheers,
Liam.
http://cimss.ssec.wisc.edu/~gumley/
|
|
|
|
|
|
| Re: Running IDL from cron [message #23862 is a reply to message #23860] |
Thu, 22 February 2001 10:44 |
Craig Markwardt
Messages: 1869
Registered: November 1996
|
Senior Member |
|
|
Gwyn Fireman <Gwyn.Fireman@gsfc.nasa.gov> writes:
> Hi, everyone -
>
> With Deja gone and Google barely functional I have to ask here what I'm
> sure has been addressed before:
Heh, I tried this in my newsgroup archive and nothing found. Hmm.
The main problems I've found with running cron jobs are:
1. environment variables not set, and
2. programs won't run because path is not set right (consequence of #1)
The answer to number 2 is to specify all program paths explicitly. As
for number 1, I would say that you should to write your shell script
to set those variables explicitly as well, and bypass your .cshrc.
Something like this:
#!/bin/sh
HOME=/home/fireman
$HOME/bin/setup_dirs
IDL_PATH=+$HOME/lib/idl:+/usr/local/rsi/idl/lib
EXPERIMENT_DIR=/data/fireman
/usr/local/bin/idl $HOME/scripts/run_job.pro
This way everything is explicit and there isn't a chance for cron to
monkey with it.
Good luck,
Craig
--
------------------------------------------------------------ --------------
Craig B. Markwardt, Ph.D. EMAIL: craigmnet@cow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
------------------------------------------------------------ --------------
|
|
|
|
| Re: Running IDL from cron [message #23863 is a reply to message #23862] |
Thu, 22 February 2001 09:31 |
Kenneth Mankoff
Messages: 42
Registered: August 1999
|
Member |
|
|
This is a pretty ugly hack, but one i use with at and cron.
create a save file of your IDL session, with the following two commands:
IDL> save, /var, file='variables.sav'
IDL> save, /all, file='procedures.sav'
Then, i have my at job restore those two save files.
I usually have my at jobs run a third file that is full of .com commands
to compile any other procedures i'm interested in using...
wait for some other people to respond to this post, i'm sure there is a
better way... :)
-k.
On Thu, 22 Feb 2001, Gwyn Fireman wrote:
> Hi, everyone -
>
> With Deja gone and Google barely functional I have to ask here what I'm
> sure has been addressed before:
>
> How do you run IDL routines from cron - while maintaining IDL_PATH and
> other environment variables?
>
> I have had no trouble running my idl routines using batch or at, when
> starting in the same directory as the main-level routine. In that case
> my environment is copied to the spawned process.
>
> Now I have the need to run these from cron. I have been able to write
> scripts to cd to the appropriate directory and to run my .cshrc (where
> my IDL_PATH and other environment variables are defined), but keep
> running into problems. The latest is than .cshrc bombs on a
> system-level env that apparently isn't defined when running from cron.
>
> So can anyone point me in the right direction?
> I'm running { alpha OSF unix 5.3 Nov 11 1999}.
>
> Many thanks in advance,
> Gwyn Fireman
>
>
|
|
|
|
| Re: Running IDL from cron [message #23890 is a reply to message #23831] |
Mon, 26 February 2001 15:52 |
Ben Tupper
Messages: 186
Registered: August 1999
|
Senior Member |
|
|
My dear mother-in-law will let wind a big, 'Oh, fiddlesticks!' when pressed.
I prefer to leave the rough stuff to Pavel, where is he?
David Fanning wrote:
> Vapuser (vapuser@catspaw.jpl.nasa.gov) writes:
>
>> The words I spoke when I found out
>> that despite spending a week ridding my IDL code of all connections
>> to the X server only to discover that it was the blackbox of the SGI
>> utility that was the fatal stumbling block to my goal of complete
>> 'X' independence are not reproducible in polite company.
>
> I don't think you will find any polite company here.
> Oh, maybe Ben Tupper never says any bad words. But
> surely he is the only one. :-)
>
> Cheers,
>
> David
>
> --
> David Fanning, Ph.D.
> Fanning Software Consulting
> Phone: 970-221-0438 E-Mail: davidf@dfanning.com
> Coyote's Guide to IDL Programming: http://www.dfanning.com/
> Toll-Free IDL Book Orders: 1-888-461-0155
--
Ben Tupper
248 Lower Round Pond Road
POB 106
Bristol, ME 04539
Tel: (207) 563-1048
Email: PemaquidRiver@tidewater.net
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
