comp.lang.idl-pvwave archive: archive » problem with FIX function

Home » Public Forums » archive » problem with FIX function

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

problem with FIX function [message #93881]

Tue, 15 November 2016 06:12

Ali Gamal
Messages: 98
Registered: June 2013

Member

Hi, all

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
IDL> x=40251.00
IDL> print,fix(x)
-25285
;;;;;;;;;;;;;;;;;;;;;;;;;;;;
How it is result, what is the promplem?

Report message to a moderator

Re: problem with FIX function [message #93882 is a reply to message #93881]

Tue, 15 November 2016 06:23

Burch
Messages: 28
Registered: December 2013

Junior Member

On Tuesday, November 15, 2016 at 8:12:19 AM UTC-6, AGW wrote:
> Hi, all
>
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> IDL> x=40251.00
> IDL> print,fix(x)
> -25285
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> How it is result, what is the promplem?

By default FIX() returns an integer type, which ranges from -32768 to +32767. You need to use the TYPE keyword to specify an output type appropriate for your situation. For, instance setting "TYPE = 3" (Long integer) behaves as you would expect:

IDL> print, fix(40251.00, type = 3)
40251

IDL Documentation: http://www.harrisgeospatial.com/docs/FIX.html

-Jeff

Report message to a moderator

Re: problem with FIX function [message #93883 is a reply to message #93881]

Tue, 15 November 2016 06:27

natha
Messages: 482
Registered: October 2007

Senior Member

The problem is underflow... Note that 16-bit signed integers range from -32,768 to +32,768

Use LONG(x) or FIX(x,TYPE=3)
Nata

Report message to a moderator

Re: problem with FIX function [message #93884 is a reply to message #93881]

Tue, 15 November 2016 06:29

yuxipang
Messages: 3
Registered: April 2014

Junior Member

On Tuesday, November 15, 2016 at 9:12:19 AM UTC-5, AGW wrote:
> Hi, all
>
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> IDL> x=40251.00
> IDL> print,fix(x)
> -25285
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> How it is result, what is the promplem?
x is beyond the limit of a signed integer range [-32,768 to +32,767] when using fix(x). Try this instead ...
print, fix(x, type=3) ; convert to long integer or
print, long(x)

Hope this helps!

Report message to a moderator

Re: problem with FIX function [message #93886 is a reply to message #93881]

Tue, 15 November 2016 10:38

Ali Gamal
Messages: 98
Registered: June 2013

Member

On Tuesday, November 15, 2016 at 4:12:19 PM UTC+2, AGW wrote:
> Hi, all
>
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> IDL> x=40251.00
> IDL> print,fix(x)
> -25285
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> How it is result, what is the promplem?

Thanks, Now if

x=b+c+dd+d+1720991.5=2444351.5

IDL print x=2444352.0

why, how can I solve it?

Report message to a moderator

Re: problem with FIX function [message #93887 is a reply to message #93886]

Tue, 15 November 2016 11:30

Craig Markwardt
Messages: 1869
Registered: November 1996

Senior Member

On Tuesday, November 15, 2016 at 1:38:57 PM UTC-5, AGW wrote:
> On Tuesday, November 15, 2016 at 4:12:19 PM UTC+2, AGW wrote:
>> Hi, all
>>
>> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>> IDL> x=40251.00
>> IDL> print,fix(x)
>> -25285
>> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>> How it is result, what is the promplem?
>
> Thanks, Now if
>
> x=b+c+dd+d+1720991.5=2444351.5
>
> IDL print x=2444352.0

Hard to know without understanding the variables and what formatting options you used to print it.

The statement
print, 2444351.5, format='(F0)'
does result in the correct output.

Report message to a moderator

Re: problem with FIX function [message #93889 is a reply to message #93881]

Tue, 15 November 2016 12:10

Ali Gamal
Messages: 98
Registered: June 2013

Member

b=2.0-a+long((a/4.0))
c=long([365.25*y1]-0.75)
dd=long(30.6001*(am1+1))
d=d1+(h1/24.0)

Report message to a moderator

Previous Topic:	Re: Divide the world into hexagons
Next Topic:	Copying HDF5 datasets to a new HDF5 file

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 11:37:02 PDT 2025

Total time taken to generate the page: 0.00688 seconds