comp.lang.idl-pvwave archive: archive » Re: Please explain this if you can: (short)

Home » Public Forums » archive » Re: Please explain this if you can: (short)

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Please explain this if you can: (short) [message #3210]

Mon, 05 December 1994 05:46

pjclinch
Messages: 27
Registered: May 1993

Junior Member

Russ Welti (rwelti@chroma.mbt.washington.edu) wrote:
: Try this:

: IDL> a=bytarr(20)
: IDL> a(3)=244
: IDL> print,min(a)
: 0
: IDL> print,max(a)
: 244
: IDL> print,abs(max(a)-min(a))
: 12
: IDL> print,max(a)-min(a)
: 244

Whatever it is, it also affects Wave Advantage 4.2... It seems to be a
problem purely with Byte data:

WAVE> b=244
WAVE> c=byte(b)
WAVE> info,b,c
B INT = 244
C BYTE = 244
WAVE> print,abs(b),abs(c)
244 12

I'd suspect *very* strongly that the ABS routine is deciding that a BYTE
is signed data woth a value from -128 through to 127. If this were
actually the case, the sign bit would be set by our "244", and we'd have
a value of -12 in the system (not entirely coincidental that 244+12=256!).
ABS(-12) is, of course, 12, so to me it looks like there's a bug in ABS
relating to the sign it expects from BYTE data.

As BYTEs are designated as being unsigned, there doesn't seem to be much
point in using ABS with them, and if you're using signed data with
unsigned data... well, "don't" is my advice there, but it doesn't excuse
the bug.

Pete.
--
Peter Clinch Dundee Teaching Hospitals NHS Trust
voice: 44 1382 660111 x 3637 snail: Directorate of Medical Physics
fax: 44 1382 640177 Ninewells Hospital
email: p.j.clinch@dundee.ac.uk Dundee DD1 9SY Scotland UK

Report message to a moderator

[Message index]

		Re: Please explain this if you can: (short) By: pjclinch on Mon, 05 December 1994 05:46
		Re: Please explain this if you can: (short) By: greec on Mon, 05 December 1994 04:58
		Re: Please explain this if you can: (short) By: sjt on Mon, 05 December 1994 02:36

Previous Topic:	Re: how to calculate a running total of a vector
Next Topic:	WIDGET_TEXT

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 15:33:18 PDT 2025

Total time taken to generate the page: 0.00480 seconds