| Re: Double Accuracy [message #29804 is a reply to message #29679] |
Tue, 12 March 2002 11:50   |
michaeltcruz
Messages: 4
Registered: March 2002
|
Junior Member |
|
|
Okay, thanks to both of you Craig and David. The plotting shortcoming
explains why I have been having this trucation problem. I thought it
may have been due to the way I was handling the conversion.
Craig, I couldn't get the format='(D0)' to pass muster with the
complier but format = '(d)' gives me most of the accuracy. Do you
know of any website or reference material that goes into depth about
the use of the format command that would be helpful?
Thanks Again,
Mike
Craig Markwardt <craigmnet@cow.physics.wisc.edu> wrote in message news:<onvgc1g4qw.fsf@cow.physics.wisc.edu>...
> Hi Michael--
>
> I actually think that David is right. You are dealing with two things
> here. First of all, the printing precision by default is too low.
> Instead of using INFO, why not use more precision in your format
> statement, as in:
>
> print, ratio, format='(D0)'
>
> Then I think you will see that RATIO is kept to its full double
> precision.
>
> Now, on to the question of why it's *plotted* wrong. Up until
> recently IDL only kept its plot variables in single precision floating
> point. Any double precision values would be truncated down to single
> precision. Since you are using PVWAVE, I am sure that you are still
> using the "old" plotting engine of IDL.
>
> The solution for you is to subtract the mean value, or some other
> fiducial value, from the double precision values before plotting. If
> you really need to, you can relabel the axis ticks, but that gets
> involved.
>
> Craig
>
>
>
> michaeltcruz@yahoo.com (Michael Cruz) writes:
>
>
>> Thanks David but I'm sure that's not the problem. I probably should
>> have mentioned that I'm plotting several hundred of these values that
>> differ by less than the seven digits of truncated value and they are
>> end up being the same value. I was just using the INFO function to
>> show that the value was indeed getting changed to a DOUBLE.
>>
>>
>>
>> "David Burridge" <davidb@clogic.f9.co.uk> wrote in message news:<Pkkj8.6140$OP.179583@stones>...
>>> Hi Micheal,
>>>
>>> "Michael Cruz" <michaeltcruz@yahoo.com> wrote in message
>>> news:8e32c554.0203111046.35bd07c0@posting.google.com...
>>>> I am in the process of writing a program to read in exponential data
>>>> and plot its relationship to another variable. The trouble I am
>>>> having is that the values I am reading in are very small and I am
>>>> losing accuracy when I convert from a STRING to DOUBLE. Below is the
>>>> procedure I am using with a sample of the input and output. As you
>>>> can see, the output for this RATIO value is truncated to eight digits
>>>> which makes the value useless. I am fairly new to PVWAVE so I could
>>>> be making some fundamental mistakes. Appreciate help anyone can give.
>>> <snip>
>>>
>>> I'm more of an IDL user myself, but could it be the *printing* of the value
>>> that's truncating it, rather than the stored value itself? I notice that
>>> you're using "info" to print your ratio variable - what if you use 'print'
>>> like all the other values in the program?
>>>
>>> Hope this is useful,
>>>
>>> Dave
>>>
>>> David Burridge
>>> Burridge Computing
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
