| IDL_IDLBridge limited to 4? [message #73403] |
Thu, 04 November 2010 09:14  |
don.woodraska
Messages: 13
Registered: October 2005
|
Junior Member |
|
|
I've been using the IDL_IDLBridge to run tasks in parallel in the
background. However, when I increase the number of IDL_IDLBridge
objects beyond 4, they don't seem to do anything. Although they are
given tasks, the linux top command doesn't show any increase in CPU
usage. I've cranked it up from 4 to as many as 14 IDL_IDLBridge
objects, but the CPU usage doesn't go beyond 400%.
My box has dual quad cores with hyperthreading, so it looks like 16
CPU cores are available. IDL recognizes all 16 in !CPU, but I can't
seem to use them.
IDL> help,!cpu,/str
** Structure !CPU, 6 tags, length=32, data length=32:
HW_VECTOR LONG 0
VECTOR_ENABLE LONG 0
HW_NCPU LONG 16
TPOOL_NTHREADS LONG 16
TPOOL_MIN_ELTS LONG64 100000
TPOOL_MAX_ELTS LONG64 0
I've split my job into 4 pieces and run 4 separate IDL instances with
4 bridge objects each on 4 separate terminals. There has to be a
better way.
Is there some kind of an internal limit in IDL preventing me from
using more than 4 bridge objects at the same time? Is there a startup
parameter I need to change? I'm running version 7.0.8 (no DE) on
Redhat 5.
Thanks!
|
|
|
|
| Re: IDL_IDLBridge limited to 4? [message #73501 is a reply to message #73403] |
Wed, 10 November 2010 11:13  |
d.rowenhorst@gmail.co
Messages: 10
Registered: August 2008
|
Junior Member |
|
|
On Nov 9, 1:26 pm, Donald Woodraska <don.woodra...@gmail.com> wrote:
>> Are you using the NOWAIT keyword for asynchronous operation?
>
> Yes. I first create an array of objects using a call to
> OBJ_NEW('IDL_IDLBridge') for each index in the array.
>
> I create an integer variable called threadcnt (unimaginative) to cycle
> through all of the bridge objects to check if their status() is not
> busy
> if oBridgeArr[threadcnt]->Status() eq 0 then begin
> ...
> oBridgeArr[threadcnt]->execute, cmdlist[i], /NOWAIT
>
> If the /NOWAIT keyword was not set then I couldn't run more than 1 at
> a time. By limiting the number of objects in the array I can see that
> for one object I get 100% CPU usage, for 2 I get 200%, for 3 I get
> 300%, and for 4 I get 400%. For anything more than 4 I always get
> 400%. There is no difference from 4 all the way up to 14.
>
> However, by running 4 separate IDL command-line sessions, I can use
> all 1600% available (nearly).
>
> I saw another post where someone tried using one IDL_IDLBridge object
> to create a bunch of other IDL_IDLBridge objects. I haven't tried
> that, but even if that works, it's just a workaround.
>
> Has anyone else tried to use more than 4 IDL_IDLBridges simultaneously?
When using more that 4 bridges, I have the problem where I can not
destroy/cleanup the bridges, and the program gets stuck waiting for
the bridge to die.
|
|
|
|
| Re: IDL_IDLBridge limited to 4? [message #73559 is a reply to message #73501] |
Fri, 12 November 2010 05:34 |
Haje Korth
Messages: 651
Registered: May 1997
|
Senior Member |
|
|
>
> When using more that 4 bridges, I have the problem where I can not
> destroy/cleanup the bridges, and the program gets stuck waiting for
> the bridge to die.
Same problem here. Have an objarr with 12 idl_bridges doing batch
processing. Initially I tried to destroy the objects after each
execution only to find that some of the objects cannot be destroyed. I
changed the code to kill the entire objarr at the very end of
processing. This seems to work from the command line, but not from the
Workbench, whcih still hangs. I reported the problem to ITTVIS but
they cannot reproduce it. Bummer....
Haje
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
