comp.lang.idl-pvwave archive: archive » Simplest way to spawn multiple IDL processes from within IDL?

Home » Public Forums » archive » Simplest way to spawn multiple IDL processes from within IDL?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Simplest way to spawn multiple IDL processes from within IDL? [message #88704]

Fri, 06 June 2014 05:27

rjp23
Messages: 97
Registered: June 2010

Member

I have a piece of code that I want to run on multiple data files with the only difference in the input being the date within each filename.

Previously for a small number of files I've just done it in a loop something like this:

dates=['200901', '200902', '200903']

FOR i=0, n_elements(dates)-1 do begin
this_date=dates[i]
processing_code, this_date
ENDFOR

The processing_code routine itself can take a while to run and now that the number of dates I'm processing is getting larger I wondered if there was a straightforward wait to spawn multiple instances of IDL from within the main IDL procedure. I don't need any input back into the main procedure from the subroutine that does the processing so I was hoping that there was a straightforward way.

I've looked at http://slugidl.pbworks.com/w/page/37642590/split_for%20tutor ial and while it does what I want to some extent, it'd be nicer if I could explicitly control the number of splits (so as not to swamp our server).

I wondered whether a simple spawn command that echoed the commands back into an IDL process would work but I can't seem to get it to.

Any other thoughts?

Cheers

Rob

Report message to a moderator

Re: Simplest way to spawn multiple IDL processes from within IDL? [message #88705 is a reply to message #88704]

Fri, 06 June 2014 09:10

chris_torrence@NOSPAM is currently offline

chris_torrence@NOSPAM
Messages: 528
Registered: March 2007

Senior Member

Could you just use the IDL_IDLBridge? Here's the code for a single process:

obj = IDL_IDLBridge()
obj.Execute,"processing_code," + this_date, /NOWAIT
while (obj.Status() ne 0) do void=1

You could easily extend this by doing multiple bridge objects. I would keep it under about 8-10 at a time - otherwise you start to run into OS limits.

-Chris

Report message to a moderator

Re: Simplest way to spawn multiple IDL processes from within IDL? [message #88706 is a reply to message #88704]

Fri, 06 June 2014 09:43

Phillip Bitzer
Messages: 223
Registered: June 2006

Senior Member

On Friday, June 6, 2014 7:27:02 AM UTC-5, rj...@le.ac.uk wrote:
> I have a piece of code that I want to run on multiple data files with the only difference in the input being the date within each filename.
>
>
>
> The processing_code routine itself can take a while to run and now that the number of dates I'm processing is getting larger I wondered if there was a straightforward wait to spawn multiple instances of IDL from within the main IDL procedure. I don't need any input back into the main procedure from the subroutine that does the processing so I was hoping that there was a straightforward way.
>
>
>
> Any other thoughts?
>
>
>
> Cheers
>
>
>
> Rob

You may want to check out the discussion in this thread: https://groups.google.com/forum/#!searchin/comp.lang.idl-pvw ave/parallel$20idl%7Csort:date/comp.lang.idl-pvwave/LynFzO-7 rbU/4u8fGTTf9qMJ

I mentioned then I've had some success with the Parallel IDL routines. It uses the IDLBridges Chris mentions.

I haven't checked out Mike's TaskDL (yet), but it might be just what you need.

Report message to a moderator

Previous Topic:	gridded interpolation
Next Topic:	RSI ENVI listserv

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 11:35:06 PDT 2025

Total time taken to generate the page: 0.00506 seconds