Changes to IDL 5.4 SAVE Files

With IDL 5.4, IDL became 64-bit capable. The original IDL SAVE/RESTORE format used 32-bit offsets. In order to support 64-bit memory access, the IDL SAVE/RESTORE file format was modified to allow the use of 64-bit offsets within the file, while retaining the ability to read old files that use the 32-bit offsets.

The SAVE command always begins reading any SAVE file using 32-bit offsets. If the 64-bit offset command is detected, 64-bit offsets are then used for any subsequent commands.

In IDL versions capable of writing large files (!VERSION.FILE_OFFSET_BITS EQ 64), SAVE writes a special command at the beginning of the file that switches the format from 32 to 64-bit.

SAVE always starts reading any SAVE file using 32-bit offsets. If it sees the 64-bit offset command, it switches to 64-bit offsets for any commands following that one.

This configuration is fully backward compatible, in that any IDL program can read any SAVE file it has created, or by any earlier IDL version. Note however that files produced in IDL 5.4 using 64-bit offsets are not readable by older versions of IDL.

It has come to our attention that IDL users commonly transfer SAVE/RESTORE data files written by newer IDL versions to sites where they are restored by older versions of IDL. It is not generally reasonable to expect this sort of forward compatibility, and it does not fit the usual definition of backwards compatibility. We have always strived to maintain this compatibility. However, in IDL 5.4 this was not the case. The following steps were taken in IDL 5.5 to minimize the problems caused by the IDL 5.4 save format:

64-bit offsets encoding has been improved. SAVE files written by IDL 5.5 and later should be readable by any previous version of IDL, if the file data does not exceed 2.1 GB in length.

IDL 5.5 and later versions will retain the ability to read the 64-bit offset files produced by IDL 5.4.x, thus ensuring backwards compatibility.

SAVE files written by IDL 5.5 or later versions that contain file data exceeding 2.1GB in length are not readable by older versions of IDL, but will be readable by IDL 5.5 and later versions of IDL that have !VERSION.MEMORY_BITS equal to 64.

The CONVERT_SR54 procedure, a part of the IDL 5.5 user library, can be used to convert SAVE files written within IDL 5.4 into the newer IDL 5.5 format. This allows existing data files to become readable by previous IDL versions. The CONVERT_SR54 procedure is located in the IDL_DIR/lib/obsolete directory.