# Bitwise and Logical Operations

**QUESTION:** Can IDL do bitwise operations? Can it do logical operations? Is its
behaviour logical?

**ANSWER:** Yes. Yes. Not entirely.

[This article was prepared by brave volunteer,
Mark Hadfield. Mark is a long-time IDL user and newsgroup contributor from New Zealand. If *you* would like to write an article, please contact the Article Chairman.]

Many programming languages (eg. Fortran) provide a logical data type and a set of logical operators. For example, Fortran logical variables can take values ..TRUE. and .FALSE. and support the logical operators .AND., .OR. and .NOT., according to the rules of Boolean arithmetic. So, for example

(.TRUE. .AND. ..FALSE.) evaluates to .FALSE.

(.TRUE. .OR. ..FALSE.) evaluates to .TRUE.

(.NOT. .TRUE.) evaluates to .FALSE.

IDL does not have a logical data type as such, but it allows you to use other data types to represent logical values. Most IDL users are aware that integers are often used in this way. (By they way, I'm using the word "integer" here to cover all the different integer types supported by IDL: signed and unsigned, 8, 16, 32 and 64 bit.) Here is the official rule:

- Odd integers are true and even integers are false.

But many users are unaware (well, I was) that other data types are also considered to be true or false.

- For floating point numbers, zero is false and all other values are true.
- Complex numbers follow the same rule as floating point numbers, with the imaginary part ignored.
- For strings, the empty string is false and all others are true.

Now I can't see why anyone would use a non-integer data type to represent a logical value, but—just in case you come across code where this has been done—you might want to remember that the rule for floating point numbers is different from the rule for integers. For example:

IDL> if 2 then print, 'true' else print, 'false' false IDL> if 2.0 then print, 'true' else print, 'false' true

OK, so what about the logical operators? IDL has logical operators AND, OR and NOT, which act on logical values just as you would expect. Well, not quite, because they don't accept string or complex operands and their behaviour is a little weird when you start mixing integer and floating point operands. For example:

IDL> print, 2 or 0. 2.00000

What's this? By the rules, 2 is false and 0. is false, but the result is 2., which is true. I think what happens is that the 2 (false) gets silently converted to 2. (true) before the "or" is taken. Yuk! For me the lesson is to use only integers to represent logical values.

For completeness, I should mention the XOR (exclusive or) operator, which you can read about in the manual. For reasons that escape me, it doesn't accept floating point operands.

The fact that there is more than one integer value that can represent "true", and more than one that can represent "false", can get a little confusing. IDL functions conventionally return 1 for true and 0 for false. So do comparison operators. (The data type can be a byte, short or long integer at the whim of the programmer.) If we feed 0s and 1s in any combination to the AND, OR and XOR operators then the result is always a 0 or a 1. But the NOT operator is more creative:

IDL> print, not 0, not 1 -1 -2

Now these results are correct in a logical sense, because -1 (odd) is true and -2 (even) is false. But what if we have a couple of logical expressions, and we want to test if they return results that are logically equivalent, ie. they are both false or both true? Fortran has an .EQV. operator that does just this, but IDL does not. You can't safely use the integer comparison operator, EQ, because two integers that are logically equivalent may not be equal. In this case you can get the result you want with the NOT and XOR operators—I'll leave it as an exercise to work out how.

But how on earth does (not 0) evaluate to -1? The answer is that AND, OR, NOT and XOR and bitwise operators (for integers, though not for floating point values, but let's ignore that for now). In other words, they operate separately on each of the bits that make up the integer. For example, the binary representation of a 16-bit zero is 0000000000000000. The NOT operator flips each binary digit, so (not 0) in binary is 1111111111111111, which corresponds to the signed integer -1. In these terms the rule that odd integers are true and even integers are false means that the logical value of an integer depends only on its rightmost (least significant) bit. All the other bits are just along for the ride.

I won't go into bitwise operations much except to note that you do all sorts of clever things with them. For example the !D system variable stores 18 separate logical values in a long integer in its FLAG tag. Bit 4 specifies whether the device supports color, and the following code tests it.

IDL> if (!d.flags and 2^4) ne 0 then print, 'Colorful' else print, 'Colorless' Colorful

But really, this sort of thing should be left to the C programmers.

OK, so IDL supports logical values (implemented as integers) and logical operators
(implemented as bitwise operators). This is self-consistent, though a little peculiar.
Are there any pitfalls? You've guessed it, yes there are. There is a very important IDL
function that is often used to process logical expressions, but doesn't respect the "even
is false, odd is true" rule. That function is **WHERE**. Here is an example that illustrates
the problem. Suppose we have a floating array in which some of the values may be NaN. We
want to replace each of these NaNs with a large, finite number. (Why do we want to do
this? Oh, I don't know, perhaps we want to write the data to a file format that doesn't
allow NaNs. I just wanted an example in which it is natural to use the NOT operator.) The
recommended way to test for the presence of a NaN (or other non-finite value) is the
**FINITE** function. So

IDL> array = fltarr(5) & array [2] = !values.f_nan IDL> print, array 0.000000 0.000000 NaN 0.000000 0.000000 IDL> for i=0,n_elements(array )-1 do if not finite(array [i]) then array [i] = 1.E36 IDL> print, array 0.000000 0.000000 1.00000e+036 0.000000 0.000000

But hold it! Isn't using a FOR loop bad style, not to mention slow for large arrays? OK,
so the conventional way to do this sort of thing in IDL uses the **WHERE** function. There
are a couple of examples in the documentation for **WHERE** (but you may notice that neither
of them uses the NOT operator). Our example becomes:

IDL> array = fltarr(5) & array [2] = !values.f_nan IDL> print, array 0.000000 0.000000 NaN 0.000000 0.000000 IDL> nan = where(not finite(array ), count) IDL> if count gt 0 then array [nan] = 1.E36 IDL> print, array 1.00000e+036 1.00000e+036 1.00000e+036 1.00000e+036 1.00000e+036

That's odd. Our test has flagged all the elements of *array *as non-finite. The reason is that
the **WHERE** function returns the indices of all the *non-zero* elements in its
input, not all the *logically true* elements. The **WHERE** function in our example is
acting exactly as the documentation says it should, and so is the NOT operator, but I,
for one, find the result surprising. A similar issue arises with the **KEYWORD_SET**
function, which looks as if it should deal with logical input data but returns 1 for all
non-zero values.

Ideally IDL would have a logical data type, with only two values, true and false. It doesn't, and retrofitting one would lead to all sorts of backwards-compatibility problems. Another possibility would be to use -1 for true, as is done in Basic, since (not 0) is -1 and (not -1) is 0. But the convention that functions and comparison operators return 1 for true is well established, so this would also lead to backwards-compatibility problems.

The problem can be avoided as follows. Consistently use 0 for false and 1 for true. Replace expressions involving NOT with logically equivalent expressions that always result in 0 or 1. Here are some alternatives to "not finite(array)":

finite(array) eq 0 1 - finite(array) (not finite(array)) and 1

Most IDL programmers seem to use the first form. To my mind it obscures the intention of the code somewhat, but I guess you get used to it. The second form is the one I've tended to use. The third form uses the fact that ANDing an integer with 1 sets all but the rightmost bit to 0. I've been toying with the idea of bundling the third form into a function called something like LNOT (logical not):

function LNOT, x return, (not x) and 1 end

Since NOT is a unary expression, it can be replaced with a function without harming readability too much; "not finite(array)" becomes

lnot(finite(array))

Anyway, rewriting the test expression in any of these forms fixes the problem, for example

IDL> array = fltarr(5) & array[2] = !values.f_nan IDL> print, array 0.000000 0.000000 NaN 0.000000 0.000000 IDL> nan = where(finite(array) eq 0, count) IDL> if count gt 0 then array[nan] = 1.E36 IDL> print, array 0.000000 0.000000 1.00000e+036 0.000000 0.000000

**ACKNOWLEDGEMENTS:** Thanks to David Fanning for inviting me to prepare this article
(and sort out the issue for myself). The article draws on several discussions on
comp.lang.idl-pvwave, with contributions from Craig Markwardt, J. D. Smith, William
Thompson, and others.

**Note:** In IDL 6.0, RSI introduced the **LOGICAL_PREDICATE** compiler option.
Setting this option with the **COMPILE_OPT** statement will cause any predicate expression
to be FALSE if zero and TRUE if anything else. A predicate expression is an expression that is
evaluated as "true" or "false" as part of a statement that controls programs execution (IF, WHILE, REPEAT, etc.).

Copyright © 2003 Mark Hadfield

Last Updated 14 February 2005