| Re: IDL not interpreting numbers correctly? [message #77365] |
Fri, 26 August 2011 20:27 |
Jeremy Bailin
Messages: 618
Registered: April 2008
|
Senior Member |
|
|
On 8/26/11 8:37 PM, Ashley Berg wrote:
> I'm having a strange problem with IDL. I have an array of numbers
> that are supposed to all add up to 100. So to check, I say:
>
> bad = where(array gt 100. or array lt 100.)
>
> and it says it finds 4 numbers that meet this criteria. However, when
> I say:
>
> print, array(bad)
>
> the numbers are all 100.000! So for some reason IDL says these
> numbers are greater than or less than 100 when actually they are equal
> to 100. Has anyone else ever had a problem like this?
The Sky Is Falling
http://www.idlcoyote.com/math_tips/sky_is_falling.html
-Jeremy.
|
|
|
|
| Re: IDL not interpreting numbers correctly? [message #77366 is a reply to message #77365] |
Fri, 26 August 2011 18:00  |
Ashley Berg
Messages: 14
Registered: February 2011
|
Junior Member |
|
|
On Aug 26, 6:37 pm, Ashley Berg <ashley.b...@gmail.com> wrote:
> I'm having a strange problem with IDL. I have an array of numbers
> that are supposed to all add up to 100. So to check, I say:
>
> bad = where(array gt 100. or array lt 100.)
>
> and it says it finds 4 numbers that meet this criteria. However, when
> I say:
>
> print, array(bad)
>
> the numbers are all 100.000! So for some reason IDL says these
> numbers are greater than or less than 100 when actually they are equal
> to 100. Has anyone else ever had a problem like this?
So, I also printed off the numbers minus 100, and found out they are
off by 1.4210855e-14. So IDL just wasn't printing out to that many
decimal places. Is there some way to get IDL to print off all the
decimal places instead of printing a rounded number?
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
