| Re: Double Accuracy [message #29813 is a reply to message #29679] |
Tue, 12 March 2002 07:55   |
Craig Markwardt
Messages: 1869
Registered: November 1996
|
Senior Member |
|
|
Hi Michael--
I actually think that David is right. You are dealing with two things
here. First of all, the printing precision by default is too low.
Instead of using INFO, why not use more precision in your format
statement, as in:
print, ratio, format='(D0)'
Then I think you will see that RATIO is kept to its full double
precision.
Now, on to the question of why it's *plotted* wrong. Up until
recently IDL only kept its plot variables in single precision floating
point. Any double precision values would be truncated down to single
precision. Since you are using PVWAVE, I am sure that you are still
using the "old" plotting engine of IDL.
The solution for you is to subtract the mean value, or some other
fiducial value, from the double precision values before plotting. If
you really need to, you can relabel the axis ticks, but that gets
involved.
Craig
michaeltcruz@yahoo.com (Michael Cruz) writes:
> Thanks David but I'm sure that's not the problem. I probably should
> have mentioned that I'm plotting several hundred of these values that
> differ by less than the seven digits of truncated value and they are
> end up being the same value. I was just using the INFO function to
> show that the value was indeed getting changed to a DOUBLE.
>
>
>
> "David Burridge" <davidb@clogic.f9.co.uk> wrote in message news:<Pkkj8.6140$OP.179583@stones>...
>> Hi Micheal,
>>
>> "Michael Cruz" <michaeltcruz@yahoo.com> wrote in message
>> news:8e32c554.0203111046.35bd07c0@posting.google.com...
>>> I am in the process of writing a program to read in exponential data
>>> and plot its relationship to another variable. The trouble I am
>>> having is that the values I am reading in are very small and I am
>>> losing accuracy when I convert from a STRING to DOUBLE. Below is the
>>> procedure I am using with a sample of the input and output. As you
>>> can see, the output for this RATIO value is truncated to eight digits
>>> which makes the value useless. I am fairly new to PVWAVE so I could
>>> be making some fundamental mistakes. Appreciate help anyone can give.
>> <snip>
>>
>> I'm more of an IDL user myself, but could it be the *printing* of the value
>> that's truncating it, rather than the stored value itself? I notice that
>> you're using "info" to print your ratio variable - what if you use 'print'
>> like all the other values in the program?
>>
>> Hope this is useful,
>>
>> Dave
>>
>> David Burridge
>> Burridge Computing
--
------------------------------------------------------------ --------------
Craig B. Markwardt, Ph.D. EMAIL: craigmnet@cow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
------------------------------------------------------------ --------------
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
