Example: Reading Records With Multiple Array Elements

Frequently, data are written to files with each record containing single elements of more than one array. One example might be a file consisting of observations of altitude, pressure, temperature, and velocity with each line or record containing a value for each of the four variables. Because IDL has no equivalent of the FORTRAN implied DO list, special procedures must be used to read or write this type of file.

The first approach, which is the simplest, may be used only if all of the variables have the same data type. An array is created with as many columns as there are variables and as many rows as there are elements. The data are read into this array, the array is transposed storing each variable as a row, and each row is extracted and stored into a variable which becomes a vector. For example, the FORTRAN program which writes the data and the IDL program which reads the data are as follows:

FORTRAN Write:

DIMENSION ALT(100),PRES(100),TEMP(100),VELO(100) 
OPEN (UNIT = 1, STATUS='NEW', FILE='TEST') 
WRITE(1,'(4(1x,g15.5))') 
     (ALT(I),PRES(I),TEMP(I),VELO(I),I=1,100) 

IDL Read:

;Open file for input. 
OPENR, 1, 'test' 
 
;Define variable (NVARS by NOBS). 
A = FLTARR(4,100) 
 
;Read the data. 
READF, 1, A  
 
;Transpose so that columns become rows. 
A = TRANSPOSE(A) 
 
;Extract the variables. 
ALT = A[*, 0] 
PRES = A[*, 1] 
TEMP = A[*, 2] 
VELO = A[*, 3] 

Note that this same example may be written without the implied DO list, writing all elements for each variable contiguously and simplifying matters considerably:

FORTRAN Write:

DIMENSION ALT(100),PRES(100),TEMP(100),VELO(100) 
OPEN (UNIT = 1, STATUS='NEW', FILE='TEST') 
WRITE (1,'(4(1x,G15.5))') ALT,PRES,TEMP,VELO 

IDL Read:

;Define variables. 
ALT = FLTARR(100) 
PRES = ALT & TEMP = ALT & VELO = ALT 
OPENR, 1, 'test' 
READF, 1, ALT, PRES, TEMP, VELO 

A different approach must be taken when the columns contain different data types or the number of lines or records are not known. This method involves defining the arrays, defining a scalar variable to contain each datum in one record, then writing a loop to read each line into the scalars, and then storing the scalar values into each array. For example, assume that a fifth variable, the name of an observer which is of string type, is added to the variable list. The FORTRAN output routine and IDL input routine are as follows:

FORTRAN Write:

DIMENSION ALT(100),PRES(100),TEMP(100),VELO(100) 
CHARACTER * 10 OBS(100) 
OPEN (UNIT = 1, STATUS = 'NEW', FILE = 'TEST') 
WRITE (1,'(4(1X,G15.5),2X,A)') 
  (ALT(I),PRES(I),TEMP(I),VELO(I),OBS(I),I=1,100) 

IDL Read:

;Access file. Read files containing from 1 to 200 records. 
OPENR, 1, 'test' 
 
;Define vector, make it large enough for the biggest case. 
ALT = FLTARR(200) 
 
;Define other vectors using the first.  
PRES = ALT & TEMP = ALT & VELO = ALT 
 
;Define string array.  
OBS = STRARR(200) 
 
;Define scalar string. 
OBSS = '' 
 
;Initialize counter. 
I = 0 
 
WHILE ~ EOF(1) DO BEGIN 
   ;Read scalars. 
   READF, 1, $ 
 
   FORMAT = '(4(1X, G15.5), 2X, A10)', $ 
     ALTS, PRESS, TEMPS, VELOS, OBSS 
 
;Store in each vector. 
   ALT[I] = ALTS & PRES[I] = PRESS & TEMP[I] = TEMPS 
   VELO[I] = VELOS & OBS[I] = OBSS 
 
   ;Increment counter and check for too many records.  
   IF I LT 199 THEN I = I + 1 ELSE STOP, 'Too many records' 
ENDWHILE 

If desired, after the file has been read and the number of observations is known, the arrays may be truncated to the correct length using a series of statements similar to the following:

ALT = ALT[0:I-1] 

The above statement represents a worst case example. Reading is greatly simplified by writing data of the same type contiguously and by knowing the size of the file. One frequently used technique is to write the number of observations into the first record so that when reading the data the size is known.

Warning
It might be tempting to implement a loop in IDL which reads the data values directly into array elements, using a statement such as the following:

FOR I = 0, 99 DO READF, 1, ALT[I], PRES[I], TEMP[I], VELO[I]

This statement is incorrect. Subscripted elements (including ranges) are temporary expressions passed as values to procedures and functions (READF in this example). Parameters passed by value do not pass results back to the caller. The proper approach is to read the data into scalars and assign the values to the individual array elements as follows:

A = 0. & P = 0. & T = 0. & V = 0.
FOR I = 0, 99 DO BEGIN
   READF, 1, A, P, T, V
   ALT[I] = A & PRES[I] = P & TEMP[I] = T & VELO[I] = V
ENDFOR