| Ready to quit after 25 years... [message #76672] |
Thu, 23 June 2011 14:49  |
Haje Korth
Messages: 651
Registered: May 1997
|
Senior Member |
|
|
I never thought that I would ever post a message with this subject
here. But after hours of utter frustration of not getting the job done
because of lack of decent documentation for recent IDL versions, I see
no way forward.
It all started so nicely trying to do a vector plot on a polar view of
the Earth using the commands:
dbmap=map('Stereographic',dimensions=dimensions,center_latit ude=90,center_longitude=0,limit=[latmin,lonmin,latmax,lonmax ],
$
position=[0.05,0.05,0.45,0.9],grid_longitude=45,grid_latitud e=10,label_show=0,transparency=50,name='dbmap')
vector=vector(v_east,v_north1,lon,lat,/overplot,data_locatio n=0,/
head_proportional,head_size=0.5,length_scale=0.5)
The sh*t hit the fan when I realized that the same vector plotted at
two different latitudes does not have the same direction. I assume
this is because of the decreasing radius of the latitude circles from
the equator toward the poles. Since this recovery I have tried
uncountable number of methods to account for this to no avail.
Conveniently, the examples in the documentation all show rectangular
grids which do not suffer from this problem.
I was able to plot the vectors using the arrow function, but doing
this one vector at a time takes so long that I would be embarrassed
sharing this code with anyone. So as for the work around.... FAIL
So I am at a loss with nowhere to turn. This is just one a many
examples of problems with the new graphics system that were either
very tedious or impossible to sort.I do not ever see this new graphics
system being adopted widely if people cannot teach themselves how to
use it.
Enough ranting, gotta dig for another language to get this code
done....
Disappointed,
Haje
|
|
|
|
|
|
| Re: Ready to quit after 25 years... [message #76785 is a reply to message #76672] |
Thu, 30 June 2011 13:59  |
Haje Korth
Messages: 651
Registered: May 1997
|
Senior Member |
|
|
Chris,
yay, google groups is working again. I blieve we have sorted all this
out by e-mail in the mean time. It is my guess that vectorizing ARROW
(and SYMBOL) would reduce the arg_present overhead and do the trick to
speed this up. Also, keep me posted on what you find out what's going
wrong with vector in a stereographic projection.
Cheers,
haje
On Jun 28, 5:40 pm, Chris Torrence <gorth...@gmail.com> wrote:
> Hi Haje,
>
> I think my previous post got swallowed, so I'll try again.
>
> Sorry to hear that you are having so many problems. I've already talked to Mark Piper about adding the ability for the Arrow and Symbol functions to accept array arguments. Regarding the Vectors on non-rectangular maps, I'll take a look at the code to see what it is doing.
>
> As far as the Arg_Present issue, I wonder if this is partly because of having to loop over all of the Arrows. Perhaps if the Arrow function were changed to accept arrays, then this problem would go away. Regardless, I'll take a look at our implementation of Arg_Present to see if there are any optimizations that we can do.
>
> Hope you're having a better day today. Keep giving us feedback on the new graphics - we definitely appreciate it!
>
> Cheers,
> Chris
>
> Dr. Christopher Torrence
> Lead Architect for Desktop Products
> ITT Visual Information Solutions
|
|
|
|
| Re: Ready to quit after 25 years... [message #76859 is a reply to message #76672] |
Tue, 12 July 2011 05:16 |
Haje Korth
Messages: 651
Registered: May 1997
|
Senior Member |
|
|
Chris,
thanks for the insight on the inner workings of profiler, and your
change makes a lot of sense to not lead the user down the wrong path.
I look forward to the vectorized arrow function to get 2+ order
magnitude speed improvement. I volunteer as beta tester. :-)
Haje
On Jul 8, 12:53 pm, Chris Torrence <gorth...@gmail.com> wrote:
> Hi Bob,
>
> Thanks for the kind remarks!
>
> Now, regarding the problem with Arg_Present - it turns out to not be a problem with Arg_Present at all. The Profiler introduces a slight overhead when it computes the time spent per iteration and records the information in an internal data structure. For most routines this overhead is negligible compared to the execution time of the routine. However, for very fast routines (like Arg_Present), the overhead is significant. The real problem is that Arg_Present was getting called 15 million times. So what you were really measuring for things like Arg_Present, Obj_Isa, etc. is just the speed of the profiler!
>
> I just made a change to the way the Profiler works. Now, by default it will not include the following routines: ARG_PRESENT, KEYWORD_SET, N_ELEMENTS, OBJ_ISA, OBJ_VALID, PTR_VALID, SIZE. You can still turn on profiling for these routines, but you need to do it explicitly. For example:
> PROFILER, /SYSTEM ; turns on profiling for all system routines except the above
> PROFILER, 'ARG_PRESENT', /SYSTEM ; also turns it on
>
> Hopefully with this change, your profiling results will no longer be biased towards routines like Arg_Present and N_Elements, and you will be better able to discover the real bottlenecks in your code.
>
> For example, in Haje's case with the Arrow function, the real problem is that he has to call Arrow 1500 times because it doesn't accept array arguments. We're also working on that for IDL 8.2.
>
> Cheers,
> Chris
> ITTVIS
|
|
|
|
| Re: Ready to quit after 25 years... [message #76867 is a reply to message #76672] |
Mon, 11 July 2011 14:54 |
Michael Galloy
Messages: 1114
Registered: April 2006
|
Senior Member |
|
|
On 7/11/11 9:45 AM, Paulo Penteado wrote:
> On Jul 9, 12:22 am, David Fanning<n...@dfanning.com> wrote:
>> Paulo Penteado writes:
>>> I agree. Which is why I do not use direct graphics. Too annoying.
>>
>>> Now I use a mix of iTools and New Graphics, because some of the
>>> functionality from the iTools GUI is not in NG.
>>
>>> To me, the problem with both systems is not complexity, it is the lack
>>> of documentation.
>>
>> Paulo, I always thought you were the one who was going
>> to write the NG book. How about it? Should we do it
>> together? Perhaps I wouldn't be so frustrated if I
>> had someone who knew the answers! :-)
>
> And I was thinking that Michael was more likely to write it, until I
> read his post above.
>
> It would be interesting to write, NG and iTools are fun to work with,
> despite the frustrating lack of documentation. Which I always found
> odd: they had all the work to develop these systems, but the
> functionality goes unused because because nobody gets told about it.
I would be interested in writing about new graphics, but not until
things calm down and ITT VIS plugs some of the holes in the new graphics
including the documentation.
Mike
--
Michael Galloy
www.michaelgalloy.com
Modern IDL, A Guide to Learning IDL: http://modernidl.idldev.com
Research Mathematician
Tech-X Corporation
|
|
|
|
| Re: Ready to quit after 25 years... [message #76873 is a reply to message #76672] |
Mon, 11 July 2011 08:45 |
penteado
Messages: 866
Registered: February 2018
|
Senior Member
Administrator |
|
|
On Jul 9, 12:22 am, David Fanning <n...@dfanning.com> wrote:
> Paulo Penteado writes:
>> I agree. Which is why I do not use direct graphics. Too annoying.
>
>> Now I use a mix of iTools and New Graphics, because some of the
>> functionality from the iTools GUI is not in NG.
>
>> To me, the problem with both systems is not complexity, it is the lack
>> of documentation.
>
> Paulo, I always thought you were the one who was going
> to write the NG book. How about it? Should we do it
> together? Perhaps I wouldn't be so frustrated if I
> had someone who knew the answers! :-)
And I was thinking that Michael was more likely to write it, until I
read his post above.
It would be interesting to write, NG and iTools are fun to work with,
despite the frustrating lack of documentation. Which I always found
odd: they had all the work to develop these systems, but the
functionality goes unused because because nobody gets told about it.
|
|
|
|
|
|
|
|
| Re: Ready to quit after 25 years... [message #76882 is a reply to message #76672] |
Fri, 08 July 2011 13:07 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Haje Korth writes:
> The issues I am having is that all the information on the new graphics
> system (NG) I absorb from the forum should be available via the IDL
> documentation in absence of third party books. In the past i went to
> David's book where the help was insufficient. I was desperately
> waiting for him to publish a book on the matter, but I gather from the
> forum he does not believe in it presumably due to lack in flexibility.
Mostly I don't "believe" in them because I don't know
anyone who uses them. I can't really say why this is so.
I don't use them because I don't think they behave
intuitively. I always find myself getting annoyed when
I use them. Life is too hard to work with tools that
annoy you! :-)
Personally, I think they haven't been accepted because
no one has a chance in hell of building a tool on their
own. They are unbelievably complex. I can teach someone,
even someone who knows next to nothing about IDL, how
to build a new Coyote Graphics tool in half an hour.
You get the same functionality, it's much faster than
these function graphics tools, and you can understand
how it works. Why would anyone want to use anything else!?
I've thought about writing a book about them from time
to time. But, writing a book is so hard, and takes so
much out of you, that I don't think I could do it and
handle the inevitable frustration with the lack of
documentation, etc. at the same time. I'd be kicking
the dog all day long! Plus, my wife's tolerance for
sulleness is not all that high these days. :-(
Also, I agree with Mike. There is no shelf life for
a book. In another year and a half, we will have another
"new" system to deal with. The last thing I need is
another 1000 books filling up my garage. :-)
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thue. ("Perhaps thos speakest truth.")
|
|
|
|
| Re: Ready to quit after 25 years... [message #76883 is a reply to message #76672] |
Fri, 08 July 2011 12:33 |
Michael Galloy
Messages: 1114
Registered: April 2006
|
Senior Member |
|
|
On 7/8/11 10:58 AM, Haje Korth wrote:
> Hi Bob,
> no doubt the response from Chris and colleagues to arising issues is
> superb and prompt. Communication has significantly improved under the
> ITT umbrella and cannot be compared to the KODAK firewall I tried to
> climb over unsuccessfully most of the time (Thanks to Ali Bahrami for
> opening a back door here and there).
I agree.
> The issues I am having is that all the information on the new graphics
> system (NG) I absorb from the forum should be available via the IDL
> documentation in absence of third party books. In the past i went to
> David's book where the help was insufficient. I was desperately
> waiting for him to publish a book on the matter, but I gather from the
> forum he does not believe in it presumably due to lack in flexibility.
> Mike's new book is great but the 15-page NG chapter is not
> comprehensive enough to fill the holes I have in using the system. To
> my knowledge no other books exist describing NG. My lack of knowledge
> combined with the inability to fill it, results presently in regular
> outbursts of anger. :-)
Yes, I agree that I don't have enough NG information in the book to
describe it sufficiently to switch to it for daily tasks, but I am stuck
in a similar situation as iTools: the information is not documented yet.
I could provide some extra routines, hacks, and examples that workaround
some of the limitations of the current new graphics, but the next
version of IDL would probably change them drastically.
I originally had a chapter on developing with the iTools in the book,
but between writing the examples and writing the text of the chapter a
new version or two of IDL came out and my examples no longer worked (and
that was sticking to the *documented* interface which was certainly not
adequate to begin with)! I didn't want to get in the same position with
iTool's offspring, NG.
And while I'm complaining about the docs (which I consider to be a
strength of IDL's, by the way), are we going to get the full set of
documentation in the online help browser again? I really miss the
External Development Guide.
I did see a positive change in the docs recently. While Googling for one
of the ENVI routines (I think, ENVI_OPEN_FILE), the first hit was the
PDF of the ENVI Programming Guide on ITT VIS' own website! It was the
PDF, not an HTML page, and I couldn't find any IDL documentation, but
it's a great to see that maybe the online documentation might be put
online someday! Let's hope that it wasn't an oversight for the PDF to
leak out...
> Apparently some of the issues I am having are due to, let's say,
> unoptimized codes. While this is not unexpected, the present
> development cycle implies that these probably small fixes which would
> make my life easier will takes 6-12 months to reach my system. It is
> very likely that the codes I am using right now will no longer be in
> use by the time a fix becomes available. This leaves me with the
> question, whether it is actually worth bothering with the new
> technology. No, I do not think I want to know the answer. :-)
While I think the NG routines are very exciting, I can't see myself
switching to them yet. I still use direct graphics for 2D graphics and
hand coded object graphics for 3D graphics.
> Oh, well just my two cents, back to the pile of work in front of me...
Mike
--
Michael Galloy
www.michaelgalloy.com
Modern IDL, A Guide to Learning IDL: http://modernidl.idldev.com
Research Mathematician
Tech-X Corporation
|
|
|
|
| Re: Ready to quit after 25 years... [message #76934 is a reply to message #76672] |
Wed, 13 July 2011 13:11 |
ben.bighair
Messages: 221
Registered: April 2007
|
Senior Member |
|
|
Hi,
On Jul 13, 11:23 am, Chris Torrence <gorth...@gmail.com> wrote:
> Hi Ben,
> No, the SYSTEM keyword just tells the Profiler that you want to profile built-in system routines (things like FFT, TOTAL, etc) as opposed to .pro routines like DIST, HANNING (and your own routines). It has nothing to do with whether things like ARG_PRESENT are profiled or not. except for the fact that you need to specify /SYSTEM when you do want to turn on profiling for those routines (since they are built in routines).
> Hope this makes it a bit more clear.
Yes - I get it now. Now all I need is IDL 8!
Thanks!
BEn
|
|
|
|
| Re: Ready to quit after 25 years... [message #76943 is a reply to message #76672] |
Wed, 13 July 2011 08:23 |
chris_torrence@NOSPAM
Messages: 528
Registered: March 2007
|
Senior Member |
|
|
Hi Ben,
No, the SYSTEM keyword just tells the Profiler that you want to profile built-in system routines (things like FFT, TOTAL, etc) as opposed to .pro routines like DIST, HANNING (and your own routines). It has nothing to do with whether things like ARG_PRESENT are profiled or not. except for the fact that you need to specify /SYSTEM when you do want to turn on profiling for those routines (since they are built in routines).
Hope this makes it a bit more clear.
-Chris
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
