| missing data [message #2244] |
Wed, 15 June 1994 10:29  |
caron
Messages: 16
Registered: May 1994
|
Junior Member |
|
|
1) efficiency of where():
Can anyone comment on the efficiency of using:
max = max( my_arr(where(my_arr ne missing_data)))
to find the maximum value of an array, but excluding
missing values?
the array here has size 63000 elements, and only a few (0 - 30)
are "missing". I am imagining that the where() command returns
an array of indices, which then is used to grab just those indices
out of my_arr.
In a higher level language I could make a single pass through my_arr
to find the max value, excluding the missing data as I go.
Is there a more efficient way to do this in IDL, or is this just
the price we pay for its generality?
2) general ramblings on missing data and contour():
Can anyone comment on "good" ways to handle missing data? I map missing
data to a very large number, so that I can exclude it from plots using
the max_value keyword. Currently I believe that countour() does not actually
work correctly with max_value. Contour lines seem to stop at regions of
missing data, and contour fills get really messed up. I suppose you could
claim this is reasonable, but RSI hasn't gotten back to me as to whether they
consider this behavior a feature or a bug (hello?).
What should contour do in the presence of missing data? Well, it could
ignore it, and draw contours based on the good data. Since the contour
algorithm probably assumes a regular grid, you could "fill in" the
missing data using simple interpolation. This is equivilent to assuming
that the data represents a sampling of an underlying continuous physical
value, which is pretty much what contouring assumes.
On the other hand, you could argue that you wouldnt want to show the user
data thats " not really there", so contours must not go through regions
of missing data. If thats the assumption contour() makes, I can replace
my missing data myself with interpolated values if I want continuous contours.
Can RSI verify thats the assumption they make with no plans to fix this
behavior or add an option to change it? Does RSI have any highly technical
notes describing various assumptions like this that we could have access to?
How about starting one?
Another variant would be to fill the areas of missing data with some color,
or outline it. You cant do that now with contour because youll get every
contour line between the missing data value and the adjacent good data value.
3) On bloodsucking support fees:
$200/year is a pretty low price for service. Any vertical niche company
like RSI needs to make customer service self supporting. I would like for RSI
to make avaliable things like known bugs, and other tech documents, that cost
them little and can save users lots. Also an e-mail address to RSI to report
possible bugs. Also a language summary on a single page. Also a "see also"
section for each routine in the reference manual. Also hypertext help. Ok Ill
stop now.....
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
