comp.lang.idl-pvwave archive - RDF feed
type: channel
description: Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
items: Re: Excluding Decimal Places in the Values of a Variable

Re: Excluding Decimal Places in the Values of a Variable
type: item
description: Lisa08 writes: > > Hi everyone, > > I have a question that I think should be very simple to answer but I can't seem to find a solution for it. > > I am trying to chop off the last two decimal places for the values of a variable. The values for my variable have 5 decimal places and I want to trim it down to 3. For example, let's say my variable is: > > x=[[1.35679, 2.65487], [3.65789, 4.56455]] > > So I want to exclude the last two decimal places and have it be: > > x=[[1.356, 2.654], [3.657, 4.564]] > > > Now, I know how to do this using a "PRINT" command as below but I don't know how to tell IDL to save the output that was printed out to a variable. > > > IDL> print, x, FORMAT='(F8.3)' > 1.357 > 2.655 > 3.658 > > Also, Is there a way to make it "not round up"? I have found a routine that does what I want, excludes the final two decimal places and doesn't round up, but the problem with it is that it converts the values to a string to do this and then I can't seem to convert them back to floats as it gives me the following error: > > Type Conversion error: Unable to convert given STRING to Float > > The routine I found online to do this is called DECIMALS.pro and is available here: > > https://people.ok.ubc.ca/erosolo/idl/lib/decimals.pro I would do it this way: IDL> x=[[1.35679, 2.65487], [3.65789, 4.56455]] IDL> x = Fix(x*1000)/1000. Cheers, David -- David Fanning, Ph.D. Fanning Software Consulting, Inc. Coyote's Guide to IDL Programming: http://www.idlcoyote.com/ Sepore ma de ni thue. ("Perhaps thou speakest truth.")
creator: David Fanning
date: 2013-02-14T20:55:56-00:00

Re: Excluding Decimal Places in the Values of a Variable
type: item
description: On Thursday, February 14, 2013 2:55:56 PM UTC-6, David Fanning wrote: > Lisa08 writes: > > > >> > >> Hi everyone, > >> > >> I have a question that I think should be very simple to answer but I can't seem to find a solution for it. > >> > >> I am trying to chop off the last two decimal places for the values of a variable. The values for my variable have 5 decimal places and I want to trim it down to 3. For example, let's say my variable is: > >> > >> x=[[1.35679, 2.65487], [3.65789, 4.56455]] > >> > >> So I want to exclude the last two decimal places and have it be: > >> > >> x=[[1.356, 2.654], [3.657, 4.564]] > >> > >> > >> Now, I know how to do this using a "PRINT" command as below but I don't know how to tell IDL to save the output that was printed out to a variable. > >> > >> > >> IDL> print, x, FORMAT='(F8.3)' > >> 1.357 > >> 2.655 > >> 3.658 > >> > >> Also, Is there a way to make it "not round up"? I have found a routine that does what I want, excludes the final two decimal places and doesn't round up, but the problem with it is that it converts the values to a string to do this and then I can't seem to convert them back to floats as it gives me the following error: > >> > >> Type Conversion error: Unable to convert given STRING to Float > >> > >> The routine I found online to do this is called DECIMALS.pro and is available here: > >> > >> https://people.ok.ubc.ca/erosolo/idl/lib/decimals.pro > > > > I would do it this way: > > > > IDL> x=[[1.35679, 2.65487], [3.65789, 4.56455]] > > IDL> x = Fix(x*1000)/1000. > > > > Cheers, > > > > David > > > > -- > > David Fanning, Ph.D. > > Fanning Software Consulting, Inc. > > Coyote's Guide to IDL Programming: http://www.idlcoyote.com/ > > Sepore ma de ni thue. ("Perhaps thou speakest truth.") Wow, so simple and easy. I always make things more complicated than they have to be. Thanks so much! Lisa
creator: Lisa08
date: 2013-02-14T21:17:08-00:00

Re: Excluding Decimal Places in the Values of a Variable
type: item
description: Lisa08 writes: > Wow, so simple and easy. I always make things more complicated than they have to be. Thanks so much! Oh, sorry. When you said "I have a question that I think should be very simple to answer" I thought you were looking for the simple solution. I'll give you the more complicated on next time. ;-) Cheers, David -- David Fanning, Ph.D. Fanning Software Consulting, Inc. Coyote's Guide to IDL Programming: http://www.idlcoyote.com/ Sepore ma de ni thue. ("Perhaps thou speakest truth.")
creator: David Fanning
date: 2013-02-14T21:31:28-00:00

Re: Excluding Decimal Places in the Values of a Variable
type: item
description: Another simple question: why fix() and not floor()? it seems like the difference is a return type of INT vs LONG. anything else? Gianguido On Thursday, February 14, 2013 2:55:56 PM UTC-6, David Fanning wrote: > > > I would do it this way: > > > > IDL> x=[[1.35679, 2.65487], [3.65789, 4.56455]] > > IDL> x = Fix(x*1000)/1000. > > > > Cheers, > > > > David > > > > -- > > David Fanning, Ph.D. > > Fanning Software Consulting, Inc. > > Coyote's Guide to IDL Programming: http://www.idlcoyote.com/ > > Sepore ma de ni thue. ("Perhaps thou speakest truth.")
creator: cgguido
date: 2013-02-14T21:41:50-00:00

Re: Excluding Decimal Places in the Values of a Variable
type: item
description: Gianguido Cianci writes: > Another simple question: why fix() and not floor()? it seems like the difference is a return type of INT vs LONG. anything else? I was wondering that myself. My rule on "simple solutions" is to go with the first one that enters my head. ;-) Cheers, David -- David Fanning, Ph.D. Fanning Software Consulting, Inc. Coyote's Guide to IDL Programming: http://www.idlcoyote.com/ Sepore ma de ni thue. ("Perhaps thou speakest truth.")
creator: David Fanning
date: 2013-02-14T21:49:06-00:00

Re: Excluding Decimal Places in the Values of a Variable
type: item
description: On 02/14/13 15:55, David Fanning wrote: > > I would do it this way: > > IDL> x=[[1.35679, 2.65487], [3.65789, 4.56455]] > IDL> x = Fix(x*1000)/1000. > > Cheers, > > David How's this for a more complicated answer: FUNCTION SetDP, x, n, double=double num = 10L^n rnum = KEYWORD_SET(double) ? DOUBLE(num) : FLOAT(num) RETURN, FLOOR(x*num)/rnum END IDL> x=[[1.35679, 2.65487], [3.65789, 4.56455]] IDL> print, x 1.35679 2.65487 3.65789 4.56455 IDL> print, SetDP(x,2), format='(f18.15)' 1.350000023841858 2.650000095367432 3.650000095367432 4.559999942779541 IDL> print, SetDP(x,2,/double), format='(f18.15)' 1.350000000000000 2.650000000000000 3.650000000000000 4.560000000000000 :o) Sorry.... cheers, paulv
creator: Paul Van Delst[1]
date: 2013-02-19T20:32:53-00:00