| Re: IDL not interpreting numbers correctly? [message #77421 is a reply to message #77367] |
Wed, 31 August 2011 05:51  |
rogass
Messages: 200
Registered: April 2008
|
Senior Member |
|
|
On 27 Aug., 02:37, Ashley Berg <ashley.b...@gmail.com> wrote:
> I'm having a strange problem with IDL. I have an array of numbers
> that are supposed to all add up to 100. So to check, I say:
>
> bad = where(array gt 100. or array lt 100.)
>
> and it says it finds 4 numbers that meet this criteria. However, when
> I say:
>
> print, array(bad)
>
> the numbers are all 100.000! So for some reason IDL says these
> numbers are greater than or less than 100 when actually they are equal
> to 100. Has anyone else ever had a problem like this?
Hi,
we have all the same problem :)
Maybe you can use this:
function cr_c,a,b,type,prec=prec
szt = size(a,/type)+size(b,/type)
small = (szt eq 10 or szt eq 14 or szt eq 18)? $
(machar(/double)).eps : (machar()).eps
prec = ~n_elements(prec)? $
make_array(1,type=size(small,/type),value=1) : $
make_array(1,type=size(small,/type),value=prec)
small *= prec
type = ~n_elements(type)? 'eq' : type
case type of
'eq' : return, (abs(a-b) lt small)[0]
'ne' : return, (abs(a-b) gt small)[0]
'lt' : return, ((b-a) gt small)[0]
'le' : return, ((b-a) ge small)[0]
'gt' : return, ((a-b) gt small)[0]
'ge' : return, ((a-b) ge small)[0]
endcase
end
IDL> help,(4.70*100.) eq 470.0
<Expression> BYTE = 0
IDL> help,cr_c(4.70*100.,470.0,'eq')
<Expression> BYTE = 0
IDL> help,cr_c(4.70*100.,470.0,'eq',prec=1000)
<Expression> BYTE = 1
Cheers
CR
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
