comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Double Accuracy

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Double Accuracy [message #29818 is a reply to message #29679]

Tue, 12 March 2002 02:01

David Burridge
Messages: 33
Registered: January 1998

Member

Hi Micheal,

"Michael Cruz" <michaeltcruz@yahoo.com> wrote in message
news:8e32c554.0203111046.35bd07c0@posting.google.com...
> I am in the process of writing a program to read in exponential data
> and plot its relationship to another variable. The trouble I am
> having is that the values I am reading in are very small and I am
> losing accuracy when I convert from a STRING to DOUBLE. Below is the
> procedure I am using with a sample of the input and output. As you
> can see, the output for this RATIO value is truncated to eight digits
> which makes the value useless. I am fairly new to PVWAVE so I could
> be making some fundamental mistakes. Appreciate help anyone can give.
<snip>

I'm more of an IDL user myself, but could it be the *printing* of the value
that's truncating it, rather than the stored value itself? I notice that
you're using "info" to print your ratio variable - what if you use 'print'
like all the other values in the program?

Hope this is useful,

Dave

David Burridge
Burridge Computing

Report message to a moderator

[Message index]

		Double Accuracy By: michaeltcruz on Mon, 11 March 2002 10:46
		Re: Double Accuracy By: michaeltcruz on Thu, 14 March 2002 10:17
		Re: Double Accuracy By: Craig Markwardt on Tue, 12 March 2002 12:38
		Re: Double Accuracy By: michaeltcruz on Tue, 12 March 2002 11:50
		Re: Double Accuracy By: Craig Markwardt on Tue, 12 March 2002 07:55
		Re: Double Accuracy By: michaeltcruz on Tue, 12 March 2002 07:24
		Re: Double Accuracy By: David Burridge on Tue, 12 March 2002 02:01

Previous Topic:	color table question
Next Topic:	Re: Visual C++ compiler option for >1GB of memory

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 20:03:06 PDT 2025

Total time taken to generate the page: 1.35858 seconds