comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Double Accuracy

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Double Accuracy [message #29815 is a reply to message #29679]

Tue, 12 March 2002 07:24

michaeltcruz
Messages: 4
Registered: March 2002

Junior Member

Thanks David but I'm sure that's not the problem. I probably should
have mentioned that I'm plotting several hundred of these values that
differ by less than the seven digits of truncated value and they are
end up being the same value. I was just using the INFO function to
show that the value was indeed getting changed to a DOUBLE.

"David Burridge" <davidb@clogic.f9.co.uk> wrote in message news:<Pkkj8.6140$OP.179583@stones>...
> Hi Micheal,
>
> "Michael Cruz" <michaeltcruz@yahoo.com> wrote in message
> news:8e32c554.0203111046.35bd07c0@posting.google.com...
>> I am in the process of writing a program to read in exponential data
>> and plot its relationship to another variable. The trouble I am
>> having is that the values I am reading in are very small and I am
>> losing accuracy when I convert from a STRING to DOUBLE. Below is the
>> procedure I am using with a sample of the input and output. As you
>> can see, the output for this RATIO value is truncated to eight digits
>> which makes the value useless. I am fairly new to PVWAVE so I could
>> be making some fundamental mistakes. Appreciate help anyone can give.
> <snip>
>
> I'm more of an IDL user myself, but could it be the *printing* of the value
> that's truncating it, rather than the stored value itself? I notice that
> you're using "info" to print your ratio variable - what if you use 'print'
> like all the other values in the program?
>
> Hope this is useful,
>
> Dave
>
> David Burridge
> Burridge Computing

Report message to a moderator

[Message index]

		Double Accuracy By: michaeltcruz on Mon, 11 March 2002 10:46
		Re: Double Accuracy By: michaeltcruz on Thu, 14 March 2002 10:17
		Re: Double Accuracy By: Craig Markwardt on Tue, 12 March 2002 12:38
		Re: Double Accuracy By: michaeltcruz on Tue, 12 March 2002 11:50
		Re: Double Accuracy By: Craig Markwardt on Tue, 12 March 2002 07:55
		Re: Double Accuracy By: michaeltcruz on Tue, 12 March 2002 07:24
		Re: Double Accuracy By: David Burridge on Tue, 12 March 2002 02:01

Previous Topic:	color table question
Next Topic:	Re: Visual C++ compiler option for >1GB of memory

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 22:07:54 PDT 2025

Total time taken to generate the page: 0.16323 seconds