comp.lang.idl-pvwave archive - RDF feed
type: channel
description: Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
items: idl parallel processing
items: Re: idl parallel processing

idl parallel processing
type: item
description: Hello, I have a procedure below. It want to call my procedure in my main program and do parallel processing on the do loop. How can use the IDL_Bridge . Any suggestion pro computation,data=data,siteN=siteN,result result=fltarr(n_elements(siteN)) FOR i= 0,n_elements(siteN)-1 do begin y=where(data eq siteN(i)) if y(0) ge 0 then begin result(i)=1 endif else begin result(i)=0 endelse ENDFOR end
creator: siumtesfai
date: 2017-05-18T22:05:47-00:00

Re: idl parallel processing
type: item
description: Yes, you can use the IDL Bridge for this. But if you have IDL 8.4 or later, then more valuable would be using the .HASVALUE() static method. Your code would then be result= bytarr(n_elements(siteN)) FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) The reasons this is much faster are (1) you don't need to compute the output vector of WHERE(). All you care about is whether the siteN[i] value is present in the data array-- you don't care where it is. And (2) the .hasvalue() method will return as soon as it finds a single case where the siteN[i] value is present, so you skip having to search the entire data array --Wayne On Thursday, May 18, 2017 at 6:05:51 PM UTC-4, Sium T wrote: > Hello, > > I have a procedure below. It want to call my procedure in my main program and do parallel processing on the do loop. > > How can use the IDL_Bridge . Any suggestion > > pro computation,data=data,siteN=siteN,result > > result=fltarr(n_elements(siteN)) > > FOR i= 0,n_elements(siteN)-1 do begin > y=where(data eq siteN(i)) > if y(0) ge 0 then begin > result(i)=1 > endif else begin > result(i)=0 > endelse > ENDFOR > > end
creator: wlandsman
date: 2017-05-19T14:59:05-00:00

Re: idl parallel processing
type: item
description: On Friday, May 19, 2017 at 10:59:07 AM UTC-4, wlandsman wrote: > Yes, you can use the IDL Bridge for this. But if you have IDL 8.4 or later, then more valuable would be using the .HASVALUE() static method. Your code would then be > > result= bytarr(n_elements(siteN)) > FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) > > The reasons this is much faster are (1) you don't need to compute the output vector of WHERE(). All you care about is whether the siteN[i] value is present in the data array-- you don't care where it is. And (2) the .hasvalue() method will return as soon as it finds a single case where the siteN[i] value is present, so you skip having to search the entire data array > > --Wayne > > On Thursday, May 18, 2017 at 6:05:51 PM UTC-4, Sium T wrote: >> Hello, >> >> I have a procedure below. It want to call my procedure in my main program and do parallel processing on the do loop. >> >> How can use the IDL_Bridge . Any suggestion >> >> pro computation,data=data,siteN=siteN,result >> >> result=fltarr(n_elements(siteN)) >> >> FOR i= 0,n_elements(siteN)-1 do begin >> y=where(data eq siteN(i)) >> if y(0) ge 0 then begin >> result(i)=1 >> endif else begin >> result(i)=0 >> endelse >> ENDFOR >> >> end Thanks Wayne I tried your method result= bytarr(n_elements(siteN)) FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) However, I got this error message. Object reference type required in this context:
creator: siumtesfai
date: 2017-05-22T02:59:50-00:00

Re: idl parallel processing
type: item
description: On Monday, May 22, 2017 at 4:59:52 AM UTC+2, Sium T wrote: > On Friday, May 19, 2017 at 10:59:07 AM UTC-4, wlandsman wrote: >> Yes, you can use the IDL Bridge for this. But if you have IDL 8.4 or later, then more valuable would be using the .HASVALUE() static method. Your code would then be >> >> result= bytarr(n_elements(siteN)) >> FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) >> >> The reasons this is much faster are (1) you don't need to compute the output vector of WHERE(). All you care about is whether the siteN[i] value is present in the data array-- you don't care where it is. And (2) the .hasvalue() method will return as soon as it finds a single case where the siteN[i] value is present, so you skip having to search the entire data array >> >> --Wayne >> >> On Thursday, May 18, 2017 at 6:05:51 PM UTC-4, Sium T wrote: >>> Hello, >>> >>> I have a procedure below. It want to call my procedure in my main program and do parallel processing on the do loop. >>> >>> How can use the IDL_Bridge . Any suggestion >>> >>> pro computation,data=data,siteN=siteN,result >>> >>> result=fltarr(n_elements(siteN)) >>> >>> FOR i= 0,n_elements(siteN)-1 do begin >>> y=where(data eq siteN(i)) >>> if y(0) ge 0 then begin >>> result(i)=1 >>> endif else begin >>> result(i)=0 >>> endelse >>> ENDFOR >>> >>> end > > Thanks Wayne > > I tried your method > result= bytarr(n_elements(siteN)) > FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) > > However, I got this error message. > > Object reference type required in this context: Hi, what do you get if you type at the command line: help, !version
creator: Helder Marchetto
date: 2017-05-22T09:35:47-00:00

Re: idl parallel processing
type: item
description: On Monday, May 22, 2017 at 5:35:49 AM UTC-4, Helder wrote: > On Monday, May 22, 2017 at 4:59:52 AM UTC+2, Sium T wrote: >> On Friday, May 19, 2017 at 10:59:07 AM UTC-4, wlandsman wrote: >>> Yes, you can use the IDL Bridge for this. But if you have IDL 8.4 or later, then more valuable would be using the .HASVALUE() static method. Your code would then be >>> >>> result= bytarr(n_elements(siteN)) >>> FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) >>> >>> The reasons this is much faster are (1) you don't need to compute the output vector of WHERE(). All you care about is whether the siteN[i] value is present in the data array-- you don't care where it is. And (2) the .hasvalue() method will return as soon as it finds a single case where the siteN[i] value is present, so you skip having to search the entire data array >>> >>> --Wayne >>> >>> On Thursday, May 18, 2017 at 6:05:51 PM UTC-4, Sium T wrote: >>>> Hello, >>>> >>>> I have a procedure below. It want to call my procedure in my main program and do parallel processing on the do loop. >>>> >>>> How can use the IDL_Bridge . Any suggestion >>>> >>>> pro computation,data=data,siteN=siteN,result >>>> >>>> result=fltarr(n_elements(siteN)) >>>> >>>> FOR i= 0,n_elements(siteN)-1 do begin >>>> y=where(data eq siteN(i)) >>>> if y(0) ge 0 then begin >>>> result(i)=1 >>>> endif else begin >>>> result(i)=0 >>>> endelse >>>> ENDFOR >>>> >>>> end >> >> Thanks Wayne >> >> I tried your method >> result= bytarr(n_elements(siteN)) >> FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) >> >> However, I got this error message. >> >> Object reference type required in this context: > > Hi, > what do you get if you type at the command line: > help, !version I have IDL version 8.2.3 . HasValue works with version 8.4 or above. So I need to use idl_bridge. But it becomes challenging to me Here is my trial code . First I have this procedure. It takes for ever to compute Shourly result. Can you help with how …
creator: siumtesfai
date: 2017-05-22T19:06:06-00:00

Re: idl parallel processing
type: item
description: As I mentioned, you can only use the .hasvalue static method if you have IDL 8.4 or later. If you have an earlier version of IDL you can still get some speed improvement using the following function. function hasvalue,array, value ; Determine if scalar value is present in array ; Emulates the .hasvalue static method introduced in IDL 8.4 return,~array_equal( array NE value, 1b) end and use it like this. pro computation,data=data,siteN=siteN,result N = N_elements(siteN) result=fltarr(N,/nozero) FOR i= 0,N-1 do result[i] = hasvalue(data, siteN[i]) END I find hasvalue() is a factor of two faster than using WHERE(), but a factor of five slower than using the .hasvalue method. The actual speed ratios depend on the size of the arrays, and the fraction of time that .hasvalue is true. On Sunday, May 21, 2017 at 10:59:52 PM UTC-4, Sium T wrote: > On Friday, May 19, 2017 at 10:59:07 AM UTC-4, wlandsman wrote: >> Yes, you can use the IDL Bridge for this. But if you have IDL 8.4 or later, then more valuable would be using the .HASVALUE() static method. Your code would then be >> >> result= bytarr(n_elements(siteN)) >> FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) >> >> The reasons this is much faster are (1) you don't need to compute the output vector of WHERE(). All you care about is whether the siteN[i] value is present in the data array-- you don't care where it is. And (2) the .hasvalue() method will return as soon as it finds a single case where the siteN[i] value is present, so you skip having to search the entire data array >> >> --Wayne >> >> On Thursday, May 18, 2017 at 6:05:51 PM UTC-4, Sium T wrote: >>> Hello, >>> >>> I have a procedure below. It want to call my procedure in my main program and do parallel processing on the do loop. >>> >>> How can use the IDL_Bridge . Any suggestion >>> >>> pro computation,data=data,siteN=siteN,result >>> >>> result=fltarr(n_elements(siteN)) >>> >>> FOR i= 0,n_elements(siteN)-1 do begin >>> y=where(data eq site…
creator: wlandsman
date: 2017-05-22T19:06:41-00:00

Re: idl parallel processing
type: item
description: On Monday, May 22, 2017 at 3:06:45 PM UTC-4, wlandsman wrote: > As I mentioned, you can only use the .hasvalue static method if you have IDL 8.4 or later. If you have an earlier version of IDL you can still get some speed improvement using the following function. > > function hasvalue,array, value > ; Determine if scalar value is present in array > ; Emulates the .hasvalue static method introduced in IDL 8.4 > return,~array_equal( array NE value, 1b) > end > > and use it like this. > > pro computation,data=data,siteN=siteN,result > N = N_elements(siteN) > result=fltarr(N,/nozero) > FOR i= 0,N-1 do result[i] = hasvalue(data, siteN[i]) > END > > I find hasvalue() is a factor of two faster than using WHERE(), but a factor of five slower than using the .hasvalue method. The actual speed ratios depend on the size of the arrays, and the fraction of time that .hasvalue is true. > > On Sunday, May 21, 2017 at 10:59:52 PM UTC-4, Sium T wrote: >> On Friday, May 19, 2017 at 10:59:07 AM UTC-4, wlandsman wrote: >>> Yes, you can use the IDL Bridge for this. But if you have IDL 8.4 or later, then more valuable would be using the .HASVALUE() static method. Your code would then be >>> >>> result= bytarr(n_elements(siteN)) >>> FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) >>> >>> The reasons this is much faster are (1) you don't need to compute the output vector of WHERE(). All you care about is whether the siteN[i] value is present in the data array-- you don't care where it is. And (2) the .hasvalue() method will return as soon as it finds a single case where the siteN[i] value is present, so you skip having to search the entire data array >>> >>> --Wayne >>> >>> On Thursday, May 18, 2017 at 6:05:51 PM UTC-4, Sium T wrote: >>>> Hello, >>>> >>>> I have a procedure below. It want to call my procedure in my main program and do parallel processing on the do loop. >>>> >>>> How can use the IDL_Bridge . Any suggestion >>>> >>>> pro computation,data=data,siteN=siteN,…
creator: siumtesfai
date: 2017-05-22T19:34:05-00:00

Re: idl parallel processing
type: item
description: On Monday, May 22, 2017 at 3:06:45 PM UTC-4, wlandsman wrote: > As I mentioned, you can only use the .hasvalue static method if you have IDL 8.4 or later. If you have an earlier version of IDL you can still get some speed improvement using the following function. > > function hasvalue,array, value > ; Determine if scalar value is present in array > ; Emulates the .hasvalue static method introduced in IDL 8.4 > return,~array_equal( array NE value, 1b) > end > > and use it like this. > > pro computation,data=data,siteN=siteN,result > N = N_elements(siteN) > result=fltarr(N,/nozero) > FOR i= 0,N-1 do result[i] = hasvalue(data, siteN[i]) > END > > I find hasvalue() is a factor of two faster than using WHERE(), but a factor of five slower than using the .hasvalue method. The actual speed ratios depend on the size of the arrays, and the fraction of time that .hasvalue is true. > > On Sunday, May 21, 2017 at 10:59:52 PM UTC-4, Sium T wrote: >> On Friday, May 19, 2017 at 10:59:07 AM UTC-4, wlandsman wrote: >>> Yes, you can use the IDL Bridge for this. But if you have IDL 8.4 or later, then more valuable would be using the .HASVALUE() static method. Your code would then be >>> >>> result= bytarr(n_elements(siteN)) >>> FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) >>> >>> The reasons this is much faster are (1) you don't need to compute the output vector of WHERE(). All you care about is whether the siteN[i] value is present in the data array-- you don't care where it is. And (2) the .hasvalue() method will return as soon as it finds a single case where the siteN[i] value is present, so you skip having to search the entire data array >>> >>> --Wayne >>> >>> On Thursday, May 18, 2017 at 6:05:51 PM UTC-4, Sium T wrote: >>>> Hello, >>>> >>>> I have a procedure below. It want to call my procedure in my main program and do parallel processing on the do loop. >>>> >>>> How can use the IDL_Bridge . Any suggestion >>>> >>>> pro computation,data=data,siteN=siteN,…
creator: siumtesfai
date: 2017-05-22T19:40:04-00:00

Re: idl parallel processing
type: item
description: Hi guys, Please remember that some time ago I developed a library to manage parallel processing under IDL. It is very easy to use and I strongly encourage the community to take a look at it and use it. https://github.com/bernatp3rs/idl_cpu_pm/wiki Best >:D
creator: natha
date: 2017-05-23T12:29:51-00:00

Re: idl parallel processing
type: item
description: On 05/22/2017 09:06 PM, Sium T wrote: > On Monday, May 22, 2017 at 5:35:49 AM UTC-4, Helder wrote: >> On Monday, May 22, 2017 at 4:59:52 AM UTC+2, Sium T wrote: >>> On Friday, May 19, 2017 at 10:59:07 AM UTC-4, wlandsman wrote: >>>> Yes, you can use the IDL Bridge for this. But if you have >>>> IDL 8.4 or later, then more valuable would be using the >>>> .HASVALUE() static method. Your code would then be >>>> >>>> result= bytarr(n_elements(siteN)) >>>> FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) >>>> >>>> The reasons this is much faster are (1) you don't need to >>>> compute the output vector of WHERE(). All you care about is >>>> whether the siteN[i] value is present in the data array-- you >>>> don't care where it is. And (2) the .hasvalue() method will >>>> return as soon as it finds a single case where the siteN[i] >>>> value is present, so you skip having to search the entire data >>>> array >>>> >>>> --Wayne >>>> >>>> On Thursday, May 18, 2017 at 6:05:51 PM UTC-4, Sium T wrote: >>>> > Hello, >>>> > >>>> > I have a procedure below. It want to call my procedure in my >>>> > main program and do parallel processing on the do loop. >>>> > >>>> > How can use the IDL_Bridge . Any suggestion >>>> > >>>> > pro computation,data=data,siteN=siteN,result >>>> > >>>> > result=fltarr(n_elements(siteN)) >>>> > >>>> > FOR i= 0,n_elements(siteN)-1 do begin >>>> > y=where(data eq siteN(i)) >>>> > if y(0) ge 0 then begin >>>> > result(i)=1 >>>> > endif else begin >>>> > result(i)=0 >>>> > endelse >>>> > ENDFOR >>>> > >>>> > end >>> >>> Thanks Wayne >>> >>> I tried your method >>> result= bytarr(n_elements(siteN)) >>> FOR i=0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) >>> >>> However, I got this error message. >>> >>> Object reference type required in this context: >> >> Hi, what do you get if you type at the command line: help, >> !version > > I have IDL version 8.2.3 . HasValue works with version 8.4 or > above. > > So I need to use idl_br…
creator: Markus Schmassmann
date: 2017-05-23T12:59:27-00:00

Re: idl parallel processing
type: item
description: On Tuesday, May 23, 2017 at 8:59:31 AM UTC-4, Markus Schmassmann wrote: > On 05/22/2017 09:06 PM, Sium T wrote: >> On Monday, May 22, 2017 at 5:35:49 AM UTC-4, Helder wrote: >>> On Monday, May 22, 2017 at 4:59:52 AM UTC+2, Sium T wrote: >>>> On Friday, May 19, 2017 at 10:59:07 AM UTC-4, wlandsman wrote: >>>> > Yes, you can use the IDL Bridge for this. But if you have >>>> > IDL 8.4 or later, then more valuable would be using the >>>> > .HASVALUE() static method. Your code would then be >>>> > >>>> > result= bytarr(n_elements(siteN)) >>>> > FOR i= 0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) >>>> > >>>> > The reasons this is much faster are (1) you don't need to >>>> > compute the output vector of WHERE(). All you care about is >>>> > whether the siteN[i] value is present in the data array-- you >>>> > don't care where it is. And (2) the .hasvalue() method will >>>> > return as soon as it finds a single case where the siteN[i] >>>> > value is present, so you skip having to search the entire data >>>> > array >>>> > >>>> > --Wayne >>>> > >>>> > On Thursday, May 18, 2017 at 6:05:51 PM UTC-4, Sium T wrote: >>>> >> Hello, >>>> >> >>>> >> I have a procedure below. It want to call my procedure in my >>>> >> main program and do parallel processing on the do loop. >>>> >> >>>> >> How can use the IDL_Bridge . Any suggestion >>>> >> >>>> >> pro computation,data=data,siteN=siteN,result >>>> >> >>>> >> result=fltarr(n_elements(siteN)) >>>> >> >>>> >> FOR i= 0,n_elements(siteN)-1 do begin >>>> >> y=where(data eq siteN(i)) >>>> >> if y(0) ge 0 then begin >>>> >> result(i)=1 >>>> >> endif else begin >>>> >> result(i)=0 >>>> >> endelse >>>> >> ENDFOR >>>> >> >>>> >> end >>>> >>>> Thanks Wayne >>>> >>>> I tried your method >>>> result= bytarr(n_elements(siteN)) >>>> FOR i=0,n_elements(siteN)-1 do result[i] = data.hasvalue(siteN[i]) >>>> >>>> However, I got this error message. >>>> >>>> Object reference type required in this context: >>> >>> Hi, what do you get if you …
creator: siumtesfai
date: 2017-05-24T22:20:20-00:00