|
|
|
|
|
|
| Re: IDL and Beowolf ? [message #25991 is a reply to message #25829] |
Thu, 02 August 2001 09:40   |
nobody@nowhere.com (S
Messages: 55
Registered: July 2001
|
Member |
|
|
On 31 Jul 2001 12:53:47 -0700, David Bruhwiler <bruhwile@txcorp.com> wrote:
> dmarshall@ivory.trentu.ca wrote in message news:
>> I just finished reading The Do-It-Yourself Supercomputer by William W.
>> Hargrove, Forrest M. Hoffman and Thomas Sterling in this months Scientific
>> American
>> http://www.sciam.com/2001/0801issue/0801hargrove.html
>> And wondered if anyone had any thoughts/experience with running IDL
>> applications on such a cluster.
>>
>> Dave.
>
> Hi Dave,
>
> At Tech-X Corporation, we have prototyped the use of MPI (message
> passing interface) with IDL and have succesfully tested this simple
> prototype on a Beowolf style linux cluster. Tech-X will soon receive
> funding from the US Department of Energy for further proof-of-principle
> development of this idea. This next round of development will occur
> over a six month period, beginning in September.
>
> Tech-X just issued a press release on this subject, and I just posted
> that press release to this news group.
>
> Cheers,
> David
Ugh, I just read your press release, ex-squeeze me for asking to re-post it :^)!
--
Steve S.
steve@NOSPAMmailaps.org
remove NOSPAM before replying
|
|
|
|
| Re: IDL and Beowolf ? [message #25992 is a reply to message #25829] |
Thu, 02 August 2001 09:36 |
nobody@nowhere.com (S
Messages: 55
Registered: July 2001
|
Member |
|
|
On 31 Jul 2001 12:53:47 -0700, David Bruhwiler <bruhwile@txcorp.com> wrote:
> dmarshall@ivory.trentu.ca wrote in message news:
<GGxou8.KBJ.A.ebony@news.trentu.ca>...
>> I just finished reading The Do-It-Yourself Supercomputer by William W.
>> Hargrove, Forrest M. Hoffman and Thomas Sterling in this months Scientific
>> American
>> http://www.sciam.com/2001/0801issue/0801hargrove.html
>> And wondered if anyone had any thoughts/experience with running IDL
>> applications on such a cluster.
>>
>> Dave.
>
> Hi Dave,
>
> At Tech-X Corporation, we have prototyped the use of MPI (message
> passing interface) with IDL and have succesfully tested this simple
> prototype on a Beowolf style linux cluster. Tech-X will soon receive
> funding from the US Department of Energy for further proof-of-principle
> development of this idea. This next round of development will occur
> over a six month period, beginning in September.
>
> Tech-X just issued a press release on this subject, and I just posted
> that press release to this news group.
>
> Cheers,
> David
Wow! Things have certainly developed in the past year or so. I might have
missed this, but somebody mentioned the licensing issue, how is that dealt
with? I once had a huge data-set which required me to do a large number of
iterative deconvolution calculations. I ended-up using a batch file which
would run a subset of the data and then do a save and exit before the 7-min
demo mode would expire, then a shell script would fire-up idl again, run
the batch file (which in turn read another file to figure out where I left
off) and so on ad infinitum. This sounds like cheating, I know, but actually
I had an IDL license on my desktop, but wanted to use a new Linux cluster
we had put together, and couldn't see installing IDL on each machine (the
cost of the IDL licenses exceeds the cost of the hardware!). So that was
my solution. I never did get the clustering to work with IDL, though. I didn't
try too hard, one because of the licensing issue, and two because I couldn't
figure out how to get IDL to use MPI. I thought IDL was kind of like the old
BASIC, you type it in and run it, that's my interpretation of the term "inter-
pretive". Yet, to use MPI with my C programs, I compiled in certain MPI libs,
so how can that be done in IDL? Maybe you could "re-enlighten" me and
re-post your press release?
--
Steve S.
steve@NOSPAMmailaps.org
remove NOSPAM before replying
|
|
|
|
|
|
| Re: IDL and Beowolf ? [message #26103 is a reply to message #25991] |
Fri, 03 August 2001 10:06 |
Axel Schweiger
Messages: 11
Registered: November 1995
|
Junior Member |
|
|
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
On a slightly different note. My sense is that a great deal of progress
in
<br>terms of speed for big applications could be made by having a
translator/compiler
<br>for IDL. I know you can write code to run efficiently in IDL as
well but becomes
<br>very hard when you have a lot of branches in your code. Just
my 2 cents worth.
<p>Axel
<br>
<p>"Steve Smith" wrote:
<blockquote TYPE=CITE>On 31 Jul 2001 12:53:47 -0700, David Bruhwiler <bruhwile@txcorp.com>
wrote:
<br>>dmarshall@ivory.trentu.ca wrote in message news:
<br>>> I just finished reading The Do-It-Yourself Supercomputer by William
W.
<br>>> Hargrove, Forrest M. Hoffman and Thomas Sterling in this months
Scientific
<br>>> American
<br>>> <a href="http://www.sciam.com/2001/0801issue/0801hargrove.html">http://www.sciam.com/2001/0801issue/0801hargrove.html</a>
<br>>> And wondered if anyone had any thoughts/experience with running
IDL
<br>>> applications on such a cluster.
<br>>>
<br>>> Dave.
<br>>
<br>>Hi Dave,
<br>>
<br>>At Tech-X Corporation, we have prototyped the use of MPI (message
<br>>passing interface) with IDL and have succesfully tested this simple
<br>>prototype on a Beowolf style linux cluster. Tech-X will soon
receive
<br>>funding from the US Department of Energy for further proof-of-principle
<br>>development of this idea. This next round of development will
occur
<br>>over a six month period, beginning in September.
<br>>
<br>>Tech-X just issued a press release on this subject, and I just posted
<br>>that press release to this news group.
<br>>
<br>>Cheers,
<br>>David
<p>Ugh, I just read your press release, ex-squeeze me for asking to re-post
it :^)!
<p>--
<br>Steve S.
<p>steve@NOSPAMmailaps.org
<br>remove NOSPAM before replying</blockquote>
<pre>--
#----------------------------------------------------------- ------------------
# Axel Schweiger & amp;nbsp; & amp;nbsp; & amp;nbsp; & amp;nbsp; & amp;nbsp; & amp;nbsp; & amp;nbsp; Tel:206-543-1312
# University of Washington Fax:206-616-3142
# Polar Science Center/Applied Physics Laboratory
# 1013 NE 40th Street & ;nbsp; & ;nbsp; & ;nbsp; & ;nbsp; & ;nbsp; & ;nbsp;
# Seattle, Wa 98105 & nbsp; & nbsp; & nbsp; & nbsp; & nbsp; axel@apl.washington.edu
#----------------------------------------------------------- ------------------ </pre>
</html>
|
|
|
|
| Re: IDL and Beowolf ? [message #26124 is a reply to message #25992] |
Thu, 02 August 2001 20:20 |
bruhwile
Messages: 4
Registered: July 2001
|
Junior Member |
|
|
Steve Smith wrote...
>> At Tech-X Corporation, we have prototyped the use of MPI (message
>> passing interface) with IDL and have succesfully tested this simple
>> prototype on a Beowolf style linux cluster. Tech-X will soon receive
>> funding from the US Department of Energy for further proof-of-principle
>> development of this idea. This next round of development will occur
>> over a six month period, beginning in September.
>
> Wow! Things have certainly developed in the past year or so. I might have
> missed this, but somebody mentioned the licensing issue, how is that dealt
> with? I once had a huge data-set which required me to do a large number of
> iterative deconvolution calculations. I ended-up using a batch file which
> would run a subset of the data and then do a save and exit before the 7-min
> demo mode would expire, then a shell script would fire-up idl again, run
> the batch file (which in turn read another file to figure out where I left
> off) and so on ad infinitum. This sounds like cheating, I know, but actually
> I had an IDL license on my desktop, but wanted to use a new Linux cluster
> we had put together, and couldn't see installing IDL on each machine (the
> cost of the IDL licenses exceeds the cost of the hardware!). So that was
> my solution. I never did get the clustering to work with IDL, though. I didn't
> try too hard, one because of the licensing issue, and two because I couldn't
> figure out how to get IDL to use MPI. I thought IDL was kind of like the old
> BASIC, you type it in and run it, that's my interpretation of the term "inter-
> pretive". Yet, to use MPI with my C programs, I compiled in certain MPI libs,
> so how can that be done in IDL?
Hi Steve,
We're aware of the licensing issue, and so is RSI. It's a subject
under discussion.
IDL provides two mechanisms for invoking compiled C or Fortran code
from within a script. The MPI libraries are compiled C code, so there
is no fundamental difficulty with making MPI calls from IDL scripts.
However, there are some tricky technical issues that we had to resolve
in order to make the initial prototype work.
Sorry to be vague, but we're not yet ready to reveal the details and
we have not yet tested the concept on a serious problem. We'll learn
over the next few months just how well our approach will work.
Cheers,
David
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
