comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Conversion Error?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Conversion Error? [message #16822 is a reply to message #16680]

Mon, 16 August 1999 00:00

m218003
Messages: 56
Registered: August 1999

Member

>> IDL> size=(double(200-70.1))/0.01
>> IDL> help,size
>> SIZE DOUBLE = 12990.000
>> IDL> c=fltarr(size)
>> IDL> help,c
>>
>> I get
>>
>> C FLOAT = Array[12989]
>>
>> Does anyone know why???

That might be in the FAQ by now? Anyway: try
print,(double(200-70.1))/0.01,format='(f24.14)'
and you'll see what you really get.

Solution: add a tiny little bit to your double number before truncating it
by using it as an index (LONG please, not INTEGER !). To determine how much
to add, you should compute the ALOG10 and subtract about 9 or 10 from it.
In your case that would give you something like -5 which means you would
add 10^-5 = 0.0001 .. hardly noticeable but efficient in getting correct
numbers. **BUT**: as any speed fanatics will tell you: logs and likes are
very costly in computer resources, so whenever you know the numerical range
of values use a constant rather than the computation above.

Regards,
Martin.

Report message to a moderator

[Message index]

		Conversion Error? By: Laurent Chardon on Mon, 16 August 1999 00:00
		Re: Conversion Error? By: m218003 on Tue, 17 August 1999 00:00
		Re: Conversion Error? By: steinhh on Tue, 17 August 1999 00:00
		Re: Conversion Error? By: Med Bennett on Mon, 16 August 1999 00:00
		Re: Conversion Error? By: davidf on Mon, 16 August 1999 00:00
		Re: Conversion Error? By: Laurent Chardon on Mon, 16 August 1999 00:00
		Re: Conversion Error? By: m218003 on Mon, 16 August 1999 00:00

Previous Topic:	looking for some software to convert idl format data to text
Next Topic:	seperators in menus

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Dec 05 01:28:36 PST 2025

Total time taken to generate the page: 1.44165 seconds