comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Multiple Processing

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Multiple Processing [message #91122]

Mon, 08 June 2015 01:18

Kai Heckel
Messages: 51
Registered: April 2015

Member

Hallo everyone!

I am currently trying to process a time series on a server. I have three scrtips which are interacting during the processing. I'd like to do it yearwise (year by year). If I process the years one by one there is no problem but if I start multiple processes there is....

Imagine it like this:

1) 1 process (IDL window) -> one year

--> WORKS!

2) 4 processes (4 IDL windows) -> 4 years (separately; one per window)

--> DOESN'T WORK!

The problem always occurs after 30 min or so....I don't see what the problem is. Maybe something wrong with the LUN's?

Thanks!

Report message to a moderator

Re: Multiple Processing [message #91125 is a reply to message #91122]

Mon, 08 June 2015 05:12

markb77
Messages: 217
Registered: July 2006

Senior Member

hi Kai,

You have given basically zero information about your problem, other than the fact that your code doesn't work.

Perhaps you should look at how the script accesses your files? Do you have multiple processes trying to access the same file?

Mark

Report message to a moderator

Re: Multiple Processing [message #91126 is a reply to message #91125]

Mon, 08 June 2015 05:39

Kai Heckel
Messages: 51
Registered: April 2015

Member

Am Montag, 8. Juni 2015 14:12:56 UTC+2 schrieb superchromix:
> hi Kai,
>
> You have given basically zero information about your problem, other than the fact that your code doesn't work.
>
> Perhaps you should look at how the script accesses your files? Do you have multiple processes trying to access the same file?
>
> Mark

Hey Mark!

I have a wrapper (.sav) containing different functions. Every of my files needs to be processed using the wrapper. In a FOR-loop the .sav is copied to the folder of the current file. So each files uses it's own .sav file. So there shouldn't be any overlap. Imagine that all files for the process are in separate folders. So there shouldn't be any overlap of data access

Report message to a moderator

Re: Multiple Processing [message #91128 is a reply to message #91126]

Mon, 08 June 2015 05:58

Fabzi
Messages: 305
Registered: July 2010

Senior Member

Hi Kai,

On 06/08/2015 02:39 PM, Kai Heckel wrote:
> Am Montag, 8. Juni 2015 14:12:56 UTC+2 schrieb superchromix:
>>> hi Kai,
>>>
>>> You have given basically zero information about your problem
>
> [snip]

I think that Mark was also referring to the way you describe your
problem. "doesn't work" is not as useful as giving us: the traceback
record of the error message, the estimated line of code where the
workflow is interrupted, etc.

(http://www.catb.org/esr/faqs/smart-questions.html#beprecise)

Report message to a moderator

Re: Multiple Processing [message #91134 is a reply to message #91128]

Mon, 08 June 2015 08:43

sajan_unb
Messages: 5
Registered: June 2009

Junior Member

Did you check whether it Is a buffer space problem.... Related to how much RAM you have on your system

Report message to a moderator

Re: Multiple Processing [message #91143 is a reply to message #91134]

Mon, 08 June 2015 23:20

Kai Heckel
Messages: 51
Registered: April 2015

Member

Am Montag, 8. Juni 2015 17:43:09 UTC+2 schrieb sajan_unb:
> Did you check whether it Is a buffer space problem.... Related to how much RAM you have on your system

Thanks for your answers so far!

The exact error code I get is just "131" which is an internal undefined error in the .sav-file. As the data sets are processed in a FOR-loop, the code "131" will simply give the information that this file could not be processed and then continue with the next one.
That's why I can not give you more detailed information...there simply is not error information. Anyway, the error occurs (when I have e.g. 4 processes) at the same time in all windows....
If I stop 3 processes and reduce the number to one remaining process, the "131" disappears and the file are from that point processed correctly.

@sajan_unb:
I have 32GB RAM. One process used approx. 30 GB RAM. What do you mean with "buffer space problem"?

Thanks so much!

Report message to a moderator

Re: Multiple Processing [message #91154 is a reply to message #91143]

Tue, 09 June 2015 09:07

Craig Markwardt
Messages: 1869
Registered: November 1996

Senior Member

On Tuesday, June 9, 2015 at 2:20:52 AM UTC-4, Kai Heckel wrote:
> @sajan_unb:
> I have 32GB RAM. One process used approx. 30 GB RAM.

Wait, isn't this the problem right here? You're running four processes right?

Report message to a moderator

Previous Topic:	Bug or desired behavior in lambda functions?
Next Topic:	buggy dictionary

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 11:26:59 PDT 2025

Total time taken to generate the page: 0.00671 seconds