Reading Data into Subscripted Arrays

QUESTION: The following discussion occurred on the IDL newsgroup. This question was submitted by Neil Rogers. My answer follows.

I wonder if anyone could solve the following mystery for me. I am trying to read in individual array elements from a file using the ReadF procedure. I have an input file called test.txt containing (for exampe) these values:

  1 2 3 4 5 6

and the following IDL procedure to read them in and print them out again.

  pro test
  array = intarr(6)
  infile = 'test.txt'
  openr, inlun, infile, /get_lun
  for cnt=0,5 do begin
  	readf, inlun, array[cnt]
  endfor
  close, inlun
  free_lun, inlun
  print, array
  return
  end

Unfortunately, although no compile or run-time errors are encountered, nothing is assigned to the array elements and the file pointer stays at the beginning of the file. The output is then,

  0 0 0 0 0 0

Why is this?

ANSWER:The reason you can't read into a subscripted array has to do with the way IDL passes information into and out of procedures and functions. Variables are passed by reference, meaning that it is actually the pointer (a real pointer here) or address that is passed into the procedure or function. Changes made to the variable inside the procedure or function are made to the "real thing". For example, consider this procedure:

   PRO JUNK, variable
   variable = 10
   END

If I call the procedure like this:

   thisVar = 5
   Junk, thisVar
   Print, thisVar
       10

We see that thisVar has been changed by the procedure.

But variables are the only things that are passed by reference. Everything else (and this includes things like structure defererences, expressions, constants, and--yes--subscripted variables) are passed by value. This means that a COPY of the thing itself is passed into the procedure or function.

So, for example, in our little example if we subscript the variable like this, then the variable outside the procedure doesn't change:

   thisVar = 5
   Junk, thisVar[0]
   Print, thisVar
       5

Notice that this code doesn't cause any error messages. In fact it is perfectly valid IDL code to pass a parameter by value and not by reference.

As it happens, this is exactly what happens when you do this:

   array = IntArr(6)
   For j=0,5 DO ReadF, 1, array[j]

The value array[j] is passed by VALUE, not by REFERENCE and thus it's value outside the ReadF procedure can never change, although internally to the ReadF procedure it is doing exactly what it is suppose to be doing (i.e., reading a value from the file into a copy of a subscripted variable).

This behavior is so fundamental to the way IDL works that it is unlikely to change in our lifetime. :-)

Thus, the proper way to write the code above is to either read all the data at once (assuming you know the correct size to make your data variables), or to read the data into a temporary variable and then store that into a subscripted array.

For example, to read all the data at once, the code above would be modified to look like this:

  pro test
  array = intarr(6)
  infile = 'test.txt'
  openr, inlun, infile, /get_lun
  readf, inlun, array
  close, inlun
  free_lun, inlun
  print, array
  return
  end

For additional information about reading ASCII column-format data files, see the Reading ASCII Data Files tip.