| IDL not interpreting numbers correctly? [message #77367] |
Fri, 26 August 2011 17:37  |
Ashley Berg
Messages: 14
Registered: February 2011
|
Junior Member |
|
|
I'm having a strange problem with IDL. I have an array of numbers
that are supposed to all add up to 100. So to check, I say:
bad = where(array gt 100. or array lt 100.)
and it says it finds 4 numbers that meet this criteria. However, when
I say:
print, array(bad)
the numbers are all 100.000! So for some reason IDL says these
numbers are greater than or less than 100 when actually they are equal
to 100. Has anyone else ever had a problem like this?
|
|
|
|
| Re: IDL not interpreting numbers correctly? [message #77421 is a reply to message #77367] |
Wed, 31 August 2011 05:51  |
rogass
Messages: 200
Registered: April 2008
|
Senior Member |
|
|
On 27 Aug., 02:37, Ashley Berg <ashley.b...@gmail.com> wrote:
> I'm having a strange problem with IDL. I have an array of numbers
> that are supposed to all add up to 100. So to check, I say:
>
> bad = where(array gt 100. or array lt 100.)
>
> and it says it finds 4 numbers that meet this criteria. However, when
> I say:
>
> print, array(bad)
>
> the numbers are all 100.000! So for some reason IDL says these
> numbers are greater than or less than 100 when actually they are equal
> to 100. Has anyone else ever had a problem like this?
Hi,
we have all the same problem :)
Maybe you can use this:
function cr_c,a,b,type,prec=prec
szt = size(a,/type)+size(b,/type)
small = (szt eq 10 or szt eq 14 or szt eq 18)? $
(machar(/double)).eps : (machar()).eps
prec = ~n_elements(prec)? $
make_array(1,type=size(small,/type),value=1) : $
make_array(1,type=size(small,/type),value=prec)
small *= prec
type = ~n_elements(type)? 'eq' : type
case type of
'eq' : return, (abs(a-b) lt small)[0]
'ne' : return, (abs(a-b) gt small)[0]
'lt' : return, ((b-a) gt small)[0]
'le' : return, ((b-a) ge small)[0]
'gt' : return, ((a-b) gt small)[0]
'ge' : return, ((a-b) ge small)[0]
endcase
end
IDL> help,(4.70*100.) eq 470.0
<Expression> BYTE = 0
IDL> help,cr_c(4.70*100.,470.0,'eq')
<Expression> BYTE = 0
IDL> help,cr_c(4.70*100.,470.0,'eq',prec=1000)
<Expression> BYTE = 1
Cheers
CR
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
