| Re: Please explain this if you can: (short) [message #3210] |
Mon, 05 December 1994 05:46 |
pjclinch
Messages: 27
Registered: May 1993
|
Junior Member |
|
|
Russ Welti (rwelti@chroma.mbt.washington.edu) wrote:
: Try this:
: IDL> a=bytarr(20)
: IDL> a(3)=244
: IDL> print,min(a)
: 0
: IDL> print,max(a)
: 244
: IDL> print,abs(max(a)-min(a))
: 12
: IDL> print,max(a)-min(a)
: 244
Whatever it is, it also affects Wave Advantage 4.2... It seems to be a
problem purely with Byte data:
WAVE> b=244
WAVE> c=byte(b)
WAVE> info,b,c
B INT = 244
C BYTE = 244
WAVE> print,abs(b),abs(c)
244 12
I'd suspect *very* strongly that the ABS routine is deciding that a BYTE
is signed data woth a value from -128 through to 127. If this were
actually the case, the sign bit would be set by our "244", and we'd have
a value of -12 in the system (not entirely coincidental that 244+12=256!).
ABS(-12) is, of course, 12, so to me it looks like there's a bug in ABS
relating to the sign it expects from BYTE data.
As BYTEs are designated as being unsigned, there doesn't seem to be much
point in using ABS with them, and if you're using signed data with
unsigned data... well, "don't" is my advice there, but it doesn't excuse
the bug.
Pete.
--
Peter Clinch Dundee Teaching Hospitals NHS Trust
voice: 44 1382 660111 x 3637 snail: Directorate of Medical Physics
fax: 44 1382 640177 Ninewells Hospital
email: p.j.clinch@dundee.ac.uk Dundee DD1 9SY Scotland UK
|
|
|
|
| Re: Please explain this if you can: (short) [message #3211 is a reply to message #3210] |
Mon, 05 December 1994 04:58  |
greec
Messages: 2
Registered: November 1994
|
Junior Member |
|
|
In article <Pine.SOL.3.91.941202142557.14825A-100000@chroma> Russ Welti <rwelti@chroma.mbt.washington.edu> writes:
> Try this:
>
> IDL> a=bytarr(20)
> IDL> a(3)=244
> IDL> print,min(a)
> 0
> IDL> print,max(a)
> 244
> IDL> print,abs(max(a)-min(a))
> 12
> IDL> print,max(a)-min(a)
> 244
>
What is happening is that the bytes are being treated as signed
numbers. When you execute the 'print,abs(max(a)-min(a))'.
You get the same thing happening when you do
WAVE> print, sqrt((max(a)-min(a))*(max(a)-min(a)))
12.0000
Don't know if this is any help at all. But it does explain
what is happening.
Chris
greec@essex.ac.uk
|
|
|
|
| Re: Please explain this if you can: (short) [message #3212 is a reply to message #3211] |
Mon, 05 December 1994 02:36 |
sjt
Messages: 72
Registered: November 1993
|
Member |
|
|
Russ Welti (rwelti@chroma.mbt.washington.edu) wrote:
: Try this:
: IDL> a=bytarr(20)
: IDL> a(3)=244
: IDL> print,min(a)
: 0
: IDL> print,max(a)
: 244
: IDL> print,abs(max(a)-min(a))
: 12
: IDL> print,max(a)-min(a)
: 244
: What gives?
: \
: Russ Welti /-\
: (c-g)
: University of Washington \-/
: Dept. of Molecular Biotechnology M/S FJ-20 /
: Seattle, WA 98195 /-\
: (206) 685-3840 voice (a-t)
: (206) 685-7344 FAX \-/
: rwelti@u.washington.edu \
Evidently ABS isn't very bright! BYTE quantities are unsigned (try print,
min(a) - max(a)). But ABS for takes MSB set => negative and does a two's
complement on the argument.
--
+------------------------+---------------------------------- --+---------+
| James Tappin, | School of Physics & Space Research | O__ |
| sjt@star.sr.bham.ac.uk | University of Birmingham | -- \/` |
| "If all else fails--read the instructions!" | |
+----------------------------------------------------------- --+---------+
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
