| Re: Simple format code issue. [message #68990] |
Wed, 09 December 2009 15:20 |
penteado
Messages: 866
Registered: February 2018
|
Senior Member
Administrator |
|
|
On Dec 9, 9:06 pm, polystethylene <polystethyl...@hotmail.com> wrote:
> That's a good tip. Is it therefore the case that I could read in a
> value to 11 digits of precision - but that it won't be correct if I
> use floats instead of doubles?
Yes, it is the case. Floats are only good to some 6 digits, and
exponents to about 37. Doubles go to about 15 digits.
You may find useful to see
http://www.dfanning.com/math_tips/sky_is_falling.html
|
|
|
|
| Re: Simple format code issue. [message #68991 is a reply to message #68990] |
Wed, 09 December 2009 15:06  |
polystethylene
Messages: 28
Registered: February 2009
|
Junior Member |
|
|
That's a good tip. Is it therefore the case that I could read in a
value to 11 digits of precision - but that it won't be correct if I
use floats instead of doubles?
It was as I suspected - I saw the values in the 'variables' tab in the
debug window of the workspace, which only showed the values to 4.d.p,
when really they were to 6dp/10sf all along.
Thanks for the responses; as ever this place is full of helpful
people. Always nice to get a response from someone who's book I own as
well :D
|
|
|
|
| Re: Simple format code issue. [message #68999 is a reply to message #68991] |
Wed, 09 December 2009 13:09 |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article
<a40dd69e-ed04-4a18-969e-5c41c9acffc7@d10g2000yqh.googlegroups.com>,
polystethylene <polystethylene@hotmail.com> wrote:
> This one should be a doddle to resolve; but I can't for the life of me
> see what I'm doing wrong.
>
> I have 2 columns of data like this:
>
> 5147.475098 0.987662
> 5147.476562 0.985164
> 5147.478516 0.975080
> 5147.479980 0.991375
> 5147.481445 0.982826
> 5147.482910 0.987368
>
> etc...
>
> I would normally read this in with a format code like this:
>
> (2x,F0,3x,F0) - or sometimes drop the x's as you don't really need
> them...
>
> Anyway, this code reads the data in to 8 sig fig (1st column), and 6
> sig fig (second column)
>
> So then I tried a code of (F10.6, F0)... and I get the same issue - I
> can't get more than 4 d.p precision...
>
> I tried it with Doubles as well; but no luck. This has got to be
> simple right? Format codes are a weakness of mine, so I'm totally
> stumped.
>
> Any ideas?
>
> Cheers,
> Stefan
If you really want 11 digits of precision, make sure you are reading
into double precision floating point variables.
Ken Bowman
|
|
|
|
| Re: Simple format code issue. [message #69001 is a reply to message #68999] |
Wed, 09 December 2009 12:09 |
penteado
Messages: 866
Registered: February 2018
|
Senior Member
Administrator |
|
|
On Dec 9, 6:02 pm, polystethylene <polystethyl...@hotmail.com> wrote:
> This one should be a doddle to resolve; but I can't for the life of me
> see what I'm doing wrong.
>
> I have 2 columns of data like this:
>
> 5147.475098 0.987662
> 5147.476562 0.985164
> 5147.478516 0.975080
> 5147.479980 0.991375
> 5147.481445 0.982826
> 5147.482910 0.987368
>
> etc...
>
> I would normally read this in with a format code like this:
>
> (2x,F0,3x,F0) - or sometimes drop the x's as you don't really need
> them...
>
> Anyway, this code reads the data in to 8 sig fig (1st column), and 6
> sig fig (second column)
>
> So then I tried a code of (F10.6, F0)... and I get the same issue - I
> can't get more than 4 d.p precision...
>
> I tried it with Doubles as well; but no luck. This has got to be
> simple right? Format codes are a weakness of mine, so I'm totally
> stumped.
>
> Any ideas?
>
> Cheers,
> Stefan
What makes you think you do not have more than 4 digits? This
description makes it sound like the sky is falling.
Anyway, if you have nicely formatted columns like that in your file,
you probably do not even need to specify a format, depending on how
you read it.
Can you tell us exactly what you use to read the file and why you
think there is a problem?
|
|
|
|
| Re: Simple format code issue. [message #69002 is a reply to message #69001] |
Wed, 09 December 2009 12:07 |
polystethylene
Messages: 28
Registered: February 2009
|
Junior Member |
|
|
You know what, I'm just being stupid.
It is being held to the desired precision isn't it? I'm just not
printing it out at the desired precision. What a fool.
Sorry guys - nothing to see here, please disperse :D
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
