| Re: Conversion Error? [message #16679] |
Mon, 16 August 1999 00:00 |
davidf
Messages: 2866
Registered: September 1996
|
Senior Member |
|
|
Laurent Chardon (chardons@NOT_THISsprint.ca) writes:
> I can't figure this out. Is it a bug or something that I can not understand?
> This is with IDL 5.2 running on a PII 333 on Windows NT.
>
> If I do
>
> IDL> size=double(12990)
> IDL> help,size
> SIZE DOUBLE = 12990.000
> IDL> c=fltarr(size)
> IDL> help,c
>
> I get
>
> C FLOAT = Array[12990]
>
> which is what I expect.
>
> But... If I do
>
> IDL> size=(double(200-70.1))/0.01
> IDL> help,size
> SIZE DOUBLE = 12990.000
> IDL> c=fltarr(size)
> IDL> help,c
>
> I get
>
> C FLOAT = Array[12989]
>
> Does anyone know why??? I am starting to not trust any of my previous
> calculations done with IDL. Or is this a "feature" of the Pentium II?
I think your problem lies here:
IDL> size=(double(12990))
IDL> print, fix(size)
12990
IDL> size=(double(200-70.1))/0.01
IDL> print, fix(size)
12989
Of course, you need an *integer* as the argument of
FLTARR, since you can't have a partial element. Those
darn computer representations of numbers have done us
in again. :-)
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting
Phone: 970-221-0438 E-Mail: davidf@dfanning.com
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Toll-Free IDL Book Orders: 1-888-461-0155
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
