| problems with IDL-IDL bridge spawning [message #67442] |
Wed, 22 July 2009 18:35 |
Marshall Perrin
Messages: 44
Registered: December 2005
|
Member |
|
|
Hi all,
Recently I've started using the IDL-IDL bridge to parallelize some
simulations. Only certain parts of the algorithm are suitable for this,
so my approach is to alternate, thus:
- run Do_Stuff_1 in main IDL session,
- spawn 8 bridges, each of which runs a copy of Do_Stuff_2
- when they're done, back in the main session, obj_destroy all the
bridges, run Do_Stuff_3, and then loop back to the start for the
next iteration.
Thus the code alternates between running bridged code and just regular
IDL. At first I thought things were working fine, but for some reason,
part way through the 5th overall loop, I'm unable to start any more
bridges. I get this terribly helpful error message,
% Unable to start IDL_IDLBRIDGE slave process.
Oddly, this happens after I've started 7 of my 8 threads, repeatably
each time, during the 5th iteration of my main loop. Is there some
reason one can't open more than 40 IDL sessions total? You would think
any such limit would get reset when one kills off a previous process.
I enclose below a bit of sample code, which repeatably encounters
this problem on my 8-core Mac Pro, crashing partway through the 7th
iteration every time. On my 2-core laptop, it only spawns 2 threads
per loop, and makes it to the 54th loop before dying.
Further oddities: After a crash, .reset does NOT fix the problem;
I have to exit and restart IDL. And if I launch two separate IDL
sessions and run my test code in them both, they each independently
run and die at the same exact point, without apparently affecting
one another. So it's a per-IDL-session limit, not a total OS processes
thing.
Am I doing anything obviously wrong here, or is this a real bug?
- Marshall
pro bridgetest
nt = 100
nbparallel = !CPU.TPOOL_NTHREADS ; 1 thread per core
bridges = ptrarr(nbparallel)
for it=0L,nt-1 do begin
print, "Running loop iteration", it
for ipar=0L,nbparallel-1 do begin
bridges[ipar] = ptr_new(obj_new('IDL_IDLBridge'))
(*bridges[ipar])->Execute, $
'for c=0,10 do detector = findgen(1000,1000)'
endfor
going = 1
stats =intarr(nbparallel)
; wait for them to finish
while (going) do begin
for ipar=0L,nbparallel-1 do $
stats[ipar] = (*bridges[ipar])->Status()
if total(stats) eq 0 then going=0
endwhile
message,/info, "Parallel computation done!"
message,/info, " Destroying bridges"
for ipar=0L,nbparallel-1 do obj_destroy, (*bridges[ipar])
endfor
end
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
