comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » decimal precision

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

decimal precision [message #92464]

Tue, 22 December 2015 13:41

omar ali
Messages: 13
Registered: December 2015

Junior Member

Hi,
I have a variable of 8 digits precision and i need to make it of to precision. for example
x=7.86567123 and i want it to be x=7.87
Hint, i do not need to print it. I need to let idl use it in memory like that x=7.87

Thanks in advance

O.A

Report message to a moderator

Re: decimal precision [message #92465 is a reply to message #92464]

Tue, 22 December 2015 13:46

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

omar ali writes:

>
> Hi,
> I have a variable of 8 digits precision and i need to make it of to precision. for example
> x=7.86567123 and i want it to be x=7.87
> Hint, i do not need to print it. I need to let idl use it in memory like that x=7.87

xprime = Round(x * 100)/100.0

Cheers,

David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thue. ("Perhaps thou speakest truth.")

Report message to a moderator

Re: decimal precision [message #92470 is a reply to message #92464]

Wed, 23 December 2015 04:37

omar ali
Messages: 13
Registered: December 2015

Junior Member

بتاريخ الثلاثاء، 22 ديسمبر، 2015 10:41:09 م UTC+1، كتب omar ali:
> Hi,
> I have a variable of 8 digits precision and i need to make it of to precision. for example
> x=7.86567123 and i want it to be x=7.87
> Hint, i do not need to print it. I need to let idl use it in memory like that x=7.87
>
>
> Thanks in advance
>
> O.A

Hi David,
Thank you very much for the swift answer. Indeed, I could solve digits precision issue but i could not use it for my main goal. Indeed, I am cutting a tiff format and I need to have the corners` coordinates in two decimals precision. I need the corners to be like this:
NORTHER_LATITUDE>40
SOUTHER_LATITUDE>30
WESTER_LONGITUDE>-13.05
EASTER_LONGITUDE>0.01
thanks in advance

Report message to a moderator

Re: decimal precision [message #92472 is a reply to message #92464]

Wed, 23 December 2015 05:18

omar ali
Messages: 13
Registered: December 2015

Junior Member

Hi David,
Details of my exact problem is as follow:
I am writing a geotiff with the following command "example(1)". the tiff is successfully created. However, when I check the corners coordinate using other software or even in idl I find it like "example (2)". I want the coordinates to be calculated as "example(3)"

Thanks in advance,
O.a

Example(1)
----------
output='w:\tiff\resized.tif'
write_tiff, output, t00, geotiff=geo, /float, PLANARCONFIG=2

Example(2)
----------
Corner Coordinates:
Upper Left ( -13.0500002, 40.0000000)
Lower Left ( -13.0500002, 30.0000000)
Upper Right ( 0.0099998, 40.0000000)
Lower Right ( 0.0099998, 30.0000000)

Example(3)
----------
Corner Coordinates:
Upper Left ( -13.05, 40.00)
Lower Left ( -13.05, 30.00)
Upper Right ( 0.01, 40.00)
Lower Right ( 0.01, 30.00)

Report message to a moderator

Re: decimal precision [message #92473 is a reply to message #92472]

Wed, 23 December 2015 06:20

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

omar ali writes:

> Details of my exact problem is as follow:
> I am writing a geotiff with the following command "example(1)". the tiff is successfully created. However, when I check the corners coordinate using other software or even in idl I find it like "example (2)". I want the coordinates to be calculated as "example(3)"

I think your idea of the way computers work is a little oversimplified.
This is always a good place to start:

https://www.idlcoyote.com/math_tips/sky_is_falling.html
https://www.idlcoyote.com/math_tips/storenum.html

I think you are in good shape without doing anything else. :-)

Cheers,

David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thue. ("Perhaps thou speakest truth.")

Report message to a moderator

Previous Topic:	tiff_coordinates
Next Topic:	It's about time

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 15:13:34 PDT 2025

Total time taken to generate the page: 0.00581 seconds