Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
a plea for more reliable mathematical routines
> > In addition, it seems that RSI has done a sloppy job integrating LSODE > into IDL. they don't support all the options the original LSODE has, > including the useful ones, like single-stepping through the integration. > the documentation seems haphazardly put together. for example, their > description of output value STATUS=3 is > > "The integration was performed successfully, and no roots were found" > > what roots?? anyway, this makes me doubt the correctness of the > implementation. I have the same uneasiness about the implementation of mathematics routines in IDL, having found some simple errors in things like CURVEFIT over the past few years. If RSI wants to make inroads into the serious scientific computing arena, they will have to hire some mathematicians who will take the time and care to make sure that the mathematical functions really are properly handled. Otherwise, folks will head off to MATLAB or Fortran (gasp!) or other languages where you can count on getting a Bessel function when you call a Bessel function, or get a random number when you want one. I for one would prefer that RSI consolidate their current program structure and shore up the computational and mathematical functions to be competitive with other programs. Dick French
Re: a plea for more reliable mathematical routines
Shoring up current functions may be a hard sell, but if RSI wants to protect it's user base from erosion it would nonetheless be a good idea. What if using the highly touted new mine locating application of IDL, somebody gets blown up because of some of IDL's sloppy math routines. I'm sure RSI's lawyers have made sure that they wouldn't be libel for that but nonetheless it wouldn't reflect very well on their software (I know, I know, that's being a little dramatic !). IDL does a lot of things very well so I've accommodated myself to its shortcomings (e.g. some bad math routines, no opportunity for taking advantage of the now ubiquitous presence of multiprocessors,...) but the question is always how long can one hold out. As an example of accommodation we do most of our sensitive calculations in called Fortran or C routines. As a result, for us IDL becomes close to just a portable GUI builder (which it is good at) but there are more and more competing products that provide that functionality. > Richard G. French wrote: > >> I for one wouldprefer that RSI consolidate their current >> program structure and shore up the computational and >> mathematical functions to be competitive with other >> programs. > > I guess the problem is that it's difficult to "sell". Just > imagine the "IDL Version 10" sales slogan: > > Now with accurate numerical functions! > > Regards, > > Stein Vidar -- Karl Young UCSF,VA Medical Center MRS Unit (114M) 4150 Clement Street San Francisco, CA 94121 Email: kyoung@itsa.ucsf.edu Phone: (415) 750-2158 lab (415) 750-9463 home FAX: (415) 668-2864
Richard G. French wrote: > I for one wouldprefer that RSI consolidate their current > program structure and shore up the computational and > mathematical functions to be competitive with other > programs. I guess the problem is that it's difficult to "sell". Just imagine the "IDL Version 10" sales slogan: Now with accurate numerical functions! Regards, Stein Vidar
yeah, if i weren't ``hooked'' on IDL because of having been using it for many years doing data manipulation and stuff, i'd ditch it in favor of matlab or python. sigh, habits die hard.. In article <37D82EA9.BA62A369@wellesley.edu>, rfrench@mediaone.net wrote: > I have the same uneasiness about the implementation of mathematics > routines in IDL, having > found some simple errors in things like CURVEFIT over the past few > years. If RSI wants > to make inroads into the serious scientific computing arena, they will > have to hire some > mathematicians who will take the time and care to make sure that the > mathematical functions > really are properly handled. Otherwise, folks will head off to MATLAB or > Fortran (gasp!) or > other languages where you can count on getting a Bessel function when > you call a Bessel > function, or get a random number when you want one. I for one would > prefer that RSI > consolidate their current program structure and shore up the > computational and mathematical > functions to be competitive with other programs. > > Dick French > Sent via Deja.com http://www.deja.com/ Share what you know. Learn what you don't.
In article <7re25a$dvm$1@news.doit.wisc.edu>, "Liam Gumley" <Liam.Gumley@ssec.wisc.edu> writes: > I believe there is a market for either an add-on Mathematical Toolbox, or > preferably built-in access to a selection of routines from a well-respected > mathematical library like BLAS, LAPACK, CMLIB, NAG etc. For example, NAG > developed an add-on library for Matlab: > > http://www.nag.co.uk/nagware/NN.html > > I think many people would be more than willing to either upgrade their IDL > version, or buy an add-on toolbox, if it gave them access to a set of > high-quality numerical routines. A user survey would no doubt tell RSI very > quickly which routines people would like to see (Bessel functions and random > numbers have been mentioned). > Well, I'm using my own math routines for anything that really matters, at least I know what's in them and how far I can trust them. Some significant problems with the IDL math routines I can think about off hand (other than outright errors, at times, are: 1) Special functions only working for real input, not complex. Downright tidiculous implementation since usually same algorithm that works in the real domain will work in the complex one as well. 2) The implementation of double precision borders on the fraudelent. What I mean is, most special functions accept a keyword /DOUBLE and, when set, will return a double precision result. When one checks the output, though, one finds that this is still a result of a single precision calculation, only recast to type DOUBLE. This is worse than being said on the onset that double precision is not available. And, again, it is ridiculous since same algorithms that work for single will work for double as well. Mati Meron | "When you argue with a fool, meron@cars.uchicago.edu | chances are he is doing just the same"
Richard G. French <rfrench@wellesley.edu> wrote in message news:37D82EA9.BA62A369@wellesley.edu... > I have the same uneasiness about the implementation of mathematics > routines in IDL, having > found some simple errors in things like CURVEFIT over the past few > years. If RSI wants > to make inroads into the serious scientific computing arena, they will > have to hire some > mathematicians who will take the time and care to make sure that the > mathematical functions > really are properly handled. Otherwise, folks will head off to MATLAB or > Fortran (gasp!) or > other languages where you can count on getting a Bessel function when > you call a Bessel > function, or get a random number when you want one. I believe there is a market for either an add-on Mathematical Toolbox, or preferably built-in access to a selection of routines from a well-respected mathematical library like BLAS, LAPACK, CMLIB, NAG etc. For example, NAG developed an add-on library for Matlab: http://www.nag.co.uk/nagware/NN.html I think many people would be more than willing to either upgrade their IDL version, or buy an add-on toolbox, if it gave them access to a set of high-quality numerical routines. A user survey would no doubt tell RSI very quickly which routines people would like to see (Bessel functions and random numbers have been mentioned). Cheers, Liam. http://cimss.ssec.wisc.edu/~gumley/
In article <37DCCE9A.F1AC4BF1@zedat.fu-berlin.de>, fit@functional-imaging.com wrote: > Hi, > > so let us discuss strategies to migrate from IDL to something reasonable > (almost everything without common blocks and childish attempts to be object > oriented). > Can you give some examples of ``something reasonable''? I am very much in the dark about other options other than C++. Mirko Sent via Deja.com http://www.deja.com/ Share what you know. Learn what you don't.
Arno (fruncan@zedat.fu-berlin.de) writes: > so let us discuss strategies to migrate from IDL to something reasonable > (almost everything without common blocks and childish attempts to be object > oriented). Oh, come on. I'll be the first to admit that IDL is not perfect. But it's a hell of a lot better than most of the alternatives. And after seeing the kinds of programs I can write with objects, I find your characterization of objects as "childish" to be ridiculous. Cheers, David P.S. Just for the record, I agree completely with Richard French that some kind of consolidation of what is already *in* IDL to make it work correctly is badly overdue. I would be happy (as I'm sure many of you would be) to forgo six months of new features to have the NLEVELS keyword to the CONTOUR command actually compute N levels. :-) -- David Fanning, Ph.D. Fanning Software Consulting Phone: 970-221-0438 E-Mail: davidf@dfanning.com Coyote's Guide to IDL Programming: http://www.dfanning.com/ Toll-Free IDL Book Orders: 1-888-461-0155
Hi, so let us discuss strategies to migrate from IDL to something reasonable (almost everything without common blocks and childish attempts to be object oriented). Arno ushomirs@my-deja.com wrote: > yeah, if i weren't ``hooked'' on IDL because of having been using > it for many years doing data manipulation and stuff, > i'd ditch it in favor of matlab or python. sigh, habits die hard.. > > In article <37D82EA9.BA62A369@wellesley.edu>, > rfrench@mediaone.net wrote: > >> I have the same uneasiness about the implementation of mathematics >> routines in IDL, having >> found some simple errors in things like CURVEFIT over the past few >> years. If RSI wants >> to make inroads into the serious scientific computing arena, they will >> have to hire some >> mathematicians who will take the time and care to make sure that the >> mathematical functions >> really are properly handled. Otherwise, folks will head off to MATLAB > or >> Fortran (gasp!) or >> other languages where you can count on getting a Bessel function when >> you call a Bessel >> function, or get a random number when you want one. I for one would >> prefer that RSI >> consolidate their current program structure and shore up the >> computational and mathematical >> functions to be competitive with other programs. >> >> Dick French >> > > Sent via Deja.com http://www.deja.com/ > Share what you know. Learn what you don't. -- Functional Imaging Technologies GmbH Siemensstr. 40/41 12247 Berlin Germany fon.: +49 (0)30 76 90 24 80 fax.: +49 (0)30 76 90 24 81 mailto:fit@functional-imaging.com htp://www.functional-imaging.com
Arno (fruncan@zedat.fu-berlin.de) writes: > I definitely disagree. It is inferior to Java, Python, C/C++ (if You're able to > program a little bit of OpenGL and Motif yourself) to name only some, far too > expensive, introducing new bugs with every release (maybe a merger with Micro$ > would be adequate), lacking hooks for any reasonable development environment (or > have You ever managed to get it to work with Rose or SNiFF+ to name only a few). Of course it is inferior to Java, Python, C/C++. These languages (with the exception of C, which is what IDL is written in) didn't even exist when IDL was written. You would hope that new languages would be better than old ones. But do you re-write *your* programs every time a new, better language comes around? I sure don't. I don't even have a clue what Rose or SNiFF+ are, and I barely know anything at all about Java and Python. And that is part of my point. I spend a lot of time with people struggling to learn IDL. I'm sure they (like me) look at your alphabet soup of new languages to learn and think "Right. All that just to get a line plot!?" Yeah, OK, if you know Java and Perl and can throw in a little Motif programming so you could get some simple graphic on the display, maybe you can do something better than IDL. (Although heaven help you if your boss suddenly decides the whole mess should be ported to the Mac.) If so, I'm all for it. Go for it. But my point is that even some no-account programmer like me can take IDL and figure it out well enough in a short amount of time to make a handsome living. I'm pretty darn sure that wouldn't have happened if I would have chosen Python or C++ as my language of choice. And I've noticed that anyone who can mention five programming languages in the same sentence rarely likes IDL. Too simple, too high level, too "non-programmer" orientated. Too true. But that is *exactly* why it appeals to me and my friends. :-) > Secondly, I definitely did not characterize objects as childish but the way > they're used and implemented in IDL (look folks, now we're object oriented !). No, I suspect you are all for objects, as any thinking person would be. :-) What you object to (pun intended) is that IDL doesn't look like C++ or Java. It's a valid point. Or at least it *would* be if we were talking about a language that had been written recently. But we are talking about a language that is 16 years old! I mean, honestly, that fact that IDL is still selling as well as it does is not a testament to what a great language it is. It is a testament to how hard it is to write something like it that can beat it in the marketplace. Software like IDL is not expected to live for 16 years! The life span of almost any software program is surely limited to single digits, just *because* new programming languages come along that offer new, more powerful features. I think the fact that something remotely *like* objects can be grafted onto IDL in such as way as to greatly extend the power of the language is remarkable. I wouldn't have expected it, and I'm grateful to have it, even if it isn't implemented perfectly. I've no beef with the people who want accurate numerical functions and software that works like the documentation says it should work. I think this, rather than new features, should be the primary focus at RSI, as I've told them many, many times. But I have little patience with people who complain that IDL isn't like this or that. No, it's not. And it is not ever going to be like this or that. Not until somebody in a garage somewhere decides that they are going to take the very latest, most powerful language and build the whole damn thing over again from the ground up. Somebody has to be looking at the ol' man and thinking "I can do better than that." Perhaps that somebody is you, Arno. If so, sign me up for the first shipment. But in the meantime, I'm going to forego the alphabet fog and write myself an IDL program. Best Regards, David -- David Fanning, Ph.D. Fanning Software Consulting Phone: 970-221-0438 E-Mail: davidf@dfanning.com Coyote's Guide to IDL Programming: http://www.dfanning.com/ Toll-Free IDL Book Orders: 1-888-461-0155
FIT wrote: > I definitely disagree. It is inferior to Java, Python, C/C++ (if You're able to > program a little bit of OpenGL and Motif yourself) to name only some, far too > expensive, introducing new bugs with every release (maybe a merger with Micro$ > would be adequate), lacking hooks for any reasonable development environment (or > have You ever managed to get it to work with Rose or SNiFF+ to name only a few). The following excerpt from the 'History of IDL' in the IDL demo summarizes the philosophy behind the initial development of IDL in the early 1980s: "It became clear to him (David Stern) that his colleagues needed a computer language that went beyond the functionality of FORTRAN and provided easier, faster application development, data analysis and visualization. As a solution, Stern wrote the Mariner Mars Spectral Editor (an IDL prototype), a software language that successfully allowed scientists to test hypotheses without employing a programmer every time they needed to write or modify an application." The strength of IDL lies in enabling researchers to get results fast without a ton of programming. In the languages you mention, how may lines of code are required to read and display an 8-bit 512x512 gray scale image? In IDL, it takes four lines: openr, lun, 'image.dat', /free_lun image = bytarr(512, 512) readu, lun, image tvscl, image Those four lines of code will work on any IDL platform, and in under a minute you're looking at an image. I believe that most IDL users don't want to learn "a little bit of OpenGL and Motif" just to display an image. You must understand that for most IDL users, the *program* isn't the point: the *data* and the *visualization* are the point. Most of my colleagues don't get paid for writing elegant OOP applications; they get paid for coming up with new algorithms, visualizations, and publications from the analysis of remotely sensed data. IDL isn't perfect by any means, but as a cross-platform tool that provides researchers with a rapid analysis and visualization environment, it is hard to beat IMHO. Cheers, Liam. -- Liam E. Gumley Space Science and Engineering Center, UW-Madison http://cimss.ssec.wisc.edu/~gumley
In article <37DE1600.5E99BDE4@zedat.fu-berlin.de>, fit@functional-imaging.com wrote: > I definitely disagree. It is inferior to Java, Python, C/C++ (if You're able to > program a little bit of OpenGL and Motif yourself) to name only some, far too > expensive, introducing new bugs with every release (maybe a merger with Micro$ > would be adequate), lacking hooks for any reasonable development environment (or > have You ever managed to get it to work with Rose or SNiFF+ to name only a few). I would agree if you compare them as general purpose languages. But for data analysis and writing imaging routines, I presume that IDL beats these, since it was designed (with flaws) for that purpose. You can accomplish the same with the languages you mentioned, but with how much effort. I restrict my comment for small and medium sized applications. For a huge application with millions of lines of code, it may be more worthwile to go to Java/C++/..., simply because of the ruggedgness and the development tools. Regarding the above issues I would prefer a comparison of IDL with PV-Wave, matlab, mathcad -- none of which I use. > Secondly, I definitely did not characterize objects as childish but the way > they're used and implemented in IDL (look folks, now we're object oriented !). > What has been done there to the object paradigm is pretty much the same as they > did to numerical mathematics (look folks, we've the numerical recipes > implemented, ok the results are shaky at best, but look we have them > implemented). To incorporate an object oriented paradigm (encompassing, yes > David, a development process as well) is a little different to providing a syntax > of o->x() form. > I agree that 5.2 is not up to C++ regarding oop, but with some programming conventions, can you achieve much of the same results? Like, you cannot define a private/public interface, but can you as a programmer label an interface as such and use it in a consistant way. I agree it is inferior to an explicit declaration, but better than nothing. (here I am threading a ``tiny bit'' beyond my expertise) Mirko Sent via Deja.com http://www.deja.com/ Share what you know. Learn what you don't.
Liam Gumley wrote: > > Richard G. French <rfrench@wellesley.edu> wrote in message > news:37D82EA9.BA62A369@wellesley.edu... >> I have the same uneasiness about the implementation of mathematics >> routines in IDL, having >> found some simple errors in things like CURVEFIT over the past few >> years. If RSI wants >> to make inroads into the serious scientific computing arena, they will >> have to hire some >> mathematicians who will take the time and care to make sure that the >> mathematical functions >> really are properly handled. Otherwise, folks will head off to MATLAB or >> Fortran (gasp!) or >> other languages where you can count on getting a Bessel function when >> you call a Bessel function, or get a random number when you want one. > > I believe there is a market for either an add-on Mathematical Toolbox, or > preferably built-in access to a selection of routines from a well-respected > mathematical library like BLAS, LAPACK, CMLIB, NAG etc. For example, NAG > developed an add-on library for Matlab: > > http://www.nag.co.uk/nagware/NN.html > > I think many people would be more than willing to either upgrade their IDL > version, or buy an add-on toolbox, if it gave them access to a set of > high-quality numerical routines. A user survey would no doubt tell RSI very > quickly which routines people would like to see (Bessel functions and random > numbers have been mentioned). > > Cheers, > Liam. > http://cimss.ssec.wisc.edu/~gumley/ In our group we do rely on a number of the built-in math routines of IDL and I would really appreciate if this group could assemble a warning list of bugs in the math routines of IDL. IMO most of the IDL user/programmers do simple checks for the correctness of their code but they might never check the math routines in detail. I would also prefer to have access to a full set of IMSL or NAG or ... The implementation of the Numerical recipies sucks since a number of routines are not available. Some features are avialable through the astro/JHU .. libs (Thanks to these folks) but the standard quality control of IMSL/NAG wont be possible. I also think that each mathematical function/procedure needs describtion of the formula/algorithm used. Some of the routines ie. R_CORRELATE have it, but the help description of P_CORRELATE or CURVEFIT is just incomplete. The note: "This routine is written in the IDL language. Its source code can be found in the file r_correlate.pro in the lib subdirectory of the IDL distribution." sounds like "Dear user: if you want to debug our routine please feel free to do so." I think it is great that the source is available, however, I dont want to spend my time debugging RSI provided routines. Best, Theo
David Fanning wrote: > Arno (fruncan@zedat.fu-berlin.de) writes: > >> so let us discuss strategies to migrate from IDL to something reasonable >> (almost everything without common blocks and childish attempts to be object >> oriented). > > Oh, come on. I'll be the first to admit that IDL is not > perfect. But it's a hell of a lot better than most of the > alternatives. And after seeing the kinds of programs I > can write with objects, I find your characterization of > objects as "childish" to be ridiculous. > > Cheers, > I definitely disagree. It is inferior to Java, Python, C/C++ (if You're able to program a little bit of OpenGL and Motif yourself) to name only some, far too expensive, introducing new bugs with every release (maybe a merger with Micro$ would be adequate), lacking hooks for any reasonable development environment (or have You ever managed to get it to work with Rose or SNiFF+ to name only a few). Secondly, I definitely did not characterize objects as childish but the way they're used and implemented in IDL (look folks, now we're object oriented !). What has been done there to the object paradigm is pretty much the same as they did to numerical mathematics (look folks, we've the numerical recipes implemented, ok the results are shaky at best, but look we have them implemented). To incorporate an object oriented paradigm (encompassing, yes David, a development process as well) is a little different to providing a syntax of o->x() form. Regards, Arno > > David > > P.S. Just for the record, I agree completely with Richard > French that some kind of consolidation of what is already > *in* IDL to make it work correctly is badly overdue. I would > be happy (as I'm sure many of you would be) to forgo six > months of new features to have the NLEVELS keyword to the > CONTOUR command actually compute N levels. :-) > Yes, just make the darn thing work (at least for once in its history) !! > > -- > David Fanning, Ph.D. > Fanning Software Consulting > Phone: 970-221-0438 E-Mail: davidf@dfanning.com > Coyote's Guide to IDL Programming: http://www.dfanning.com/ > Toll-Free IDL Book Orders: 1-888-461-0155 -- Functional Imaging Technologies GmbH Siemensstr. 40/41 12247 Berlin Germany fon.: +49 (0)30 76 90 24 80 fax.: +49 (0)30 76 90 24 81 mailto:fit@functional-imaging.com htp://www.functional-imaging.com
Mirko Vukovic (mvukovic@taz.telusa.com) writes: > 1) Poor Arno. He is getting flamed, but without > stabs like his, there would not be much progress Indeed, but I think Arno is doing alright making his points. I just get the feeling that he is quite a bit younger than I am and lacks the--shall we say--historical perspective that comes part and parcel with sore knees, gray hair, and a two inch vertical jump on those damn overheads. :-( In any case, he is absolutely right about this: any software written in today's modern languages that acts like IDL does wouldn't be considered very well written by most of us. Cheers, David -- David Fanning, Ph.D. Fanning Software Consulting Phone: 970-221-0438 E-Mail: davidf@dfanning.com Coyote's Guide to IDL Programming: http://www.dfanning.com/ Toll-Free IDL Book Orders: 1-888-461-0155
In article <MPG.124804e8864d03df9898ec@news.frii.com>, davidf@dfanning.com (David Fanning) wrote: > I don't even have a clue what Rose or SNiFF+ are, ... Two things. 1) Poor Arno. He is getting flamed, but without stabs like his, there would not be much progress 2) As for Rose, he is probably refering to products from Rational Rose. AFAIK, this is a software program for program development. I read a book by Rumbaugh et al (they run the company), where a problem is decomposed along three orthogonal axes: 1)objects, their properties and relationships, 2)data flow and operations 3)program states. I used that approach a couple of times with great success. Every time I tried to design a complex program without it was an exercise in poor program design - a poorly understood program with confusing routines, data structures, you name it. Mirko Sent via Deja.com http://www.deja.com/ Share what you know. Learn what you don't.
Liam Gumley wrote: > > .... You must understand that for most IDL users, the *program* isn't > the point: the *data* and the *visualization* are the point. Most of my > colleagues don't get paid for writing elegant OOP applications; they get > paid for coming up with new algorithms, visualizations, and publications > from the analysis of remotely sensed data. > Fully agreed at this point. But the question was how to get reliable math in IDL. Worse than spending time on programming is providing wrong data and procedures. If your experiment/group/department uses your code based on bad math in IDL then your are stuck and your colleages come up with: There several ready to use libaries (IMSL, NAG, ...) why dont you use them. Cheers Theo http://www.kfa-juelich.de/icg/icg3/MITARBEITER/th.brauers.ht ml
Theo Brauers wrote: > In our group we do rely on a number of the built-in math routines of IDL and > I would really appreciate if this group could assemble a warning list of > bugs in the math routines of IDL. IMO most of the IDL user/programmers do > simple checks for the correctness of their code but they might never > check the math routines in detail. I fully agree with Theo, I've been reading this interesting thread and I'm a bit concerned. I don't use heavy duty math routines extensively, but I use a few that I could not be sure if they are really working correctly. Some people posting to this thread have mentioned (seemingly knowingly) problems in some IDL math routines. I've seen a few examples mentioned, but a list of known problems would be very useful. Is there such a list compiled anywhere? If not, could those doing the talking help in creating one? -Jonathan
In article <37E0B8CA.2911FF2C@zedat.fu-berlin.de>, fit@functional-imaging.com wrote: > I definitely do not see anything more. Linking with numerous publicly > available libraries gives You better functionality and - as image processing > mostly is mathematics and IDL is especially poor there - more reliable > results. names, names, please! > >> >> >> I restrict my comment for small and medium sized applications. For >> a huge application with millions of lines of code, it may be more >> worthwile to go to Java/C++/..., simply because of the ruggedgness >> and the development tools. >> > > Everything above say 1000 LOC intended to be reused should definitely be > designed (!!) and implemented properly (meaning not IDL). Well, I sure hope that you are wrong. I'm now writing a bunch of routines (about 30 so far), and I am going to great pains to make them understaindable for a non-me (or even me a couple of months ago). I hope that your view does not prove 100% correct :-) >> I agree that 5.2 is not up to C++ regarding oop, but with some >> programming conventions, can you achieve much of the same results? >> Like, you cannot define a private/public interface, but can >> you as a programmer label an interface as such and use it in >> a consistant way. I agree it is inferior to an explicit declaration, >> but better than nothing. (here I am threading a ``tiny bit'' beyond >> my expertise) >> > > 1.) That's exactly what OO is about. It's not just an syntactic > (in)convenience but design and programming for an interface and for reuse > (not code). Much of the result of OO efforts is the interface and thus IDL's > pseudo OO will not (not !!) achieve any of the results a moderately > experoenced designer will achieve with OO methodology. > 2.) There are no two programmers on this globe who do the same thing > consistently the same way. > >> hmmm, I'll give you that one. Good point. Mirko Sent via Deja.com http://www.deja.com/ Share what you know. Learn what you don't.