comp.lang.idl-pvwave archive
Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
Home » Public Forums » archive » strange behaviour of bytscl by large arrays
Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend  		Return to the default flat view Create a new topic Submit Reply
Re: strange behaviour of bytscl by large arrays [message #80005 is a reply to message #79963] Thu, 26 April 2012 07:59 Go to previous messageGo to previous message
Lajos Foldy is currently offline  Lajos Foldy
Messages: 176
Registered: December 2011
Senior Member
On Wednesday, April 25, 2012 11:11:43 PM UTC+2, Kenneth P. Bowman wrote:

> Since this turns out to be a floating-point precision issue, does DINDGEN
> use a long64 counter?

I think this is not a precision issue. Float can represent numbers up to 10^38 with a relative error of 10^-7. For huge values FINDGEN() creates indices with much bigger errors and this is the consequence of the current implementation, not the nature of floating point representation.

Double has a relative error of 10^-16 so DINDGEN does not need an integer counter.

regards,
Lajos

Report message to a moderator

[Message index]
 
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Previous Topic: Re: Time series.
Next Topic: IDL crashes after Mac OSX 10.7.3 upgrade from 10.7.2

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ] [ PDF ]

Current Time: Wed Oct 08 15:52:24 PDT 2025

Total time taken to generate the page: 0.00448 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.7.
Copyright ©2001-2013 FUDforum Bulletin Board Software