Warping Images Using Direct Graphics

Image warping requires selection of corresponding control points in an input image and either a reference image or a regular grid. The input image is warped so that the input image control points match the control points specified in the reference image.

Using Direct Graphics, the following example warps the input image, a Magnetic Resonance Image (MRI) proton density scan of a human thoracic cavity, to the reference image, a Computed Tomography (CT) bone scan of the same region. Complete the following steps for a detailed description of the process.

Example Code
See mriwarping_direct.pro in the examples/doc/image subdirectory of the IDL installation directory for code that duplicates this example. Run the example procedure by entering mriwarping at the IDL command prompt or view the file in an IDL Editor window by entering .EDIT mriwarping.pro.

  1. Select the MRI proton density image file:
    2. mriFile= FILEPATH('pdthorax124.jpg', $ 
   SUBDIRECTORY = ['examples', 'data'])
  2. Use READ_JPEG to read in the input image, which will be warped to the CT bone scan image. Then prepare the display device, load a grayscale color table, create a window and display the image:
    3. READ_JPEG, mriFile, mriImg 
DEVICE, DECOMPOSED = 0 
LOADCT, 0 
WINDOW, 0, XSIZE = 256, YSIZE = 256, $ 
   TITLE = 'MRI Proton Density Input Image' 
TV, mriImg
  3. Select the CT bone scan image file:
    4. ctboneFile = FILEPATH('ctbone157.jpg', $ 
   SUBDIRECTORY = ['examples', 'data'])
  4. Use READ_JPEG to read in the reference image and create a window:
    5. READ_JPEG, ctboneFile, ctboneImg 
WINDOW, 2, XSIZE = 483, YSIZE = 410, $ 
   TITLE = 'CT Bone Scan Reference Image'
  5. Load the "Hue Sat Lightness 2" color table, making the image's features easier to distinguish. After displaying the image, return to the gray scale color table.
    6. LOADCT, 20 
TV, ctboneImg 
LOADCT, 0

Proceed with the following section to begin selecting control points.

Direct Graphics Example: Selecting Control Points

This section describes selecting corresponding control points in the two displayed images. The array of control points (Xi, Yi) in the input image will be mapped to the array of points (Xo, Yo) selected in the reference image. The following image shows the points to be selected in the input image.

Figure 5-1: Control Points (CP) Selection in the Input Image

imgwarp01.jpg

  1. Set focus on the first image window:
    2. WSET, 0
  2. Select the first control point using the CURSOR function. After entering the following line, the cursor changes to a cross hair when positioned over the image window. Position the cross hair so that it is on the first control point, "CP 1", depicted by a white circle in the lower-left corned of the previous figure, and click the left mouse button. The x, y coordinate values of the first control point will be saved in the variables xi1, yi1:
    3. CURSOR, xi1, yi1, /DEVICE 
 
    
Note
    
The values for xi1 and yi1 are displayed in the IDL Workbench Variable Watch window. If you are not running the IDL Workbench, you can type PRINT, xi1, yi1 to see the values.
    

    
Note
    
After entering the first line and selecting the first control point in the display window, place your cursor in the IDL command line and press the Up Arrow key. The last line entered is displayed and can be easily modified.
  3. Continue selecting control points. After you enter each of the following lines, select the appropriate control point in the input image as shown in the previous figure:
    4. CURSOR, xi2, yi2, /DEVICE 
CURSOR, xi3, yi3, /DEVICE 
CURSOR, xi4, yi4, /DEVICE 
CURSOR, xi5, yi5, /DEVICE 
CURSOR, xi6, yi6, /DEVICE 
CURSOR, xi7, yi7, /DEVICE 
CURSOR, xi8, yi8, /DEVICE 
CURSOR, xi9, yi9, /DEVICE
  4. Set the focus on the window containing the reference image to prepare to select corresponding control points:
    5. WSET, 2 
 
    
Note
    
The Xi and Yi vectors and the Xo and Yo vectors must be the same length, meaning that you must select the same number of control points in the reference image as you selected in the input image. The control points must also be selected in the same order since the point Xi1, Yi1 will be warped to Xo1, Yo1.
    
 
    The following figure displays the control points to be selected in the next step. 
    

    
   
    Figure 5-2: Control Point (CP) Selection in the Reference Image 
    
  
         
    

    

    imgwarp02.jpg
    


    
    		

    

    

    
 
  5. Select the control points in the reference image. These are the corresponding points to which the input image control points will be warped. After entering each line, select the appropriate control point as shown in the previous figure:
    6. CURSOR, xo1, yo1, /DEVICE 
CURSOR, xo2, yo2, /DEVICE 
CURSOR, xo3, yo3, /DEVICE 
CURSOR, xo4, yo4, /DEVICE 
CURSOR, xo5, yo5, /DEVICE 
CURSOR, xo6, yo6, /DEVICE 
CURSOR, xo7, yo7, /DEVICE 
CURSOR, xo8, yo8, /DEVICE 
CURSOR, xo9, yo9, /DEVICE
  6. Place the control points into vectors (one-dimensional arrays) required by IDL warping routines. WARP_TRI and POLYWARP use the variables Xi, Yi and Xo, Yo as containers for the control points selected in the original input and reference images. Geometric transformations control the warping of the input image (Xi, Yi) values to the reference image (Xo, Yo) values. Enter the following lines to load the control point values into the one-dimensional arrays:
    7. Xi = [xi1, xi2, xi3, xi4, xi5, xi6, xi7, xi8, xi9] 
Yi = [yi1, yi2, yi3, yi4, yi5, yi6, yi7, yi8, yi9] 
Xo = [xo1, xo2, xo3, xo4, xo5, xo6, xo7, xo8, xo9] 
Yo = [yo1, yo2, yo3, yo4, yo5, yo6, yo7, yo8, yo9]

Example Code: Warping and Displaying a Transparent Image Using Direct Graphics

This section uses the control points defined in the previous section to warp the original MRI scan to the CT scan, using both of IDL's warping routines, WARP_TRI and POLY_2D. After outputting the warped image, it will be altered for display as a transparency in Direct Graphics.

  1. Warp the input image, mriImg, onto the reference image using WARP_TRI. This function uses the irregular grid of the reference image, defined by Xo, Yo, as a basis for triangulation, defining the surfaces associated with (Xo, Yo, Xi) and (Xo, Yo, Yi). Each pixel in the input image is then transferred to the appropriate position in the resulting output image as designated by interpolation. Using the WARP_TRI syntax,
    2. Result = WARP_TRI(Xo, Yo, Xi, Yi, Image, OUTPUT_SIZE=vector]
   [, /QUINTIC] [, /EXTRAPOLATE]) 
 
    set the OUTPUT_SIZE equal to the reference image dimensions since this image forms the basis of the warped, output image. Use the EXTRAPOLATE keyword to display the portions of the image which fall outside of the boundary of the selected control points: 
    warpTriImg = WARP_TRI(Xo, Yo, Xi, Yi, mriImg, $ 
   OUTPUT_SIZE=[483, 410], /EXTRAPOLATE) 
 
    
Note
    
Images requiring more aggressive warp models may not have good results outside of the extent of the control points when WARP_TRI is used with the /EXTRAPOLATE keyword.
  2. Create a new window and display the warped image:
    3. WINDOW, 3, XSIZE = 483, YSIZE = 410, TITLE = 'WARP_TRI image' 
TV, warpTriImg 
 
    You can see the how precisely the control points were selected by the amount of distortion in the resulting warped image. The following figure shows little distortion. 
    

    
   
    Figure 5-3: Warped Image Produced with WARP_TRI
    
  
         
    

    

    imgwarp03.gif
  3. Use POLYWARP in conjunction with POLY_2D to create another warped image for comparison with the WARP_TRI image. First use the POLYWARP procedure to create arrays (p, q) containing the polynomial coefficients required by the POLY_2D function:
    4. POLYWARP, Xi, Yi, Xo, Yo, 1, p, q
  4. Using the p, q array values generated by POLYWARP, warp the original image, mriImg, onto the CT bone scan using the POLY_2D function syntax,
    5. Result = POLY_2D( Array, P, Q [, Interp [, Dimx, Dimy]] 
    [, CUBIC={-1 to 0}] [, MISSING=value] ) 
 
    Specify a value of 1 for the Interp argument to use bilinear interpolation and set DimX, DimY equal to the reference image dimensions: 
    warpPolyImg = POLY_2D(mriImg, p, q, 1, 483, 410)
  5. Create a new window and display the image created using POLY_2D:
    6. WINDOW, 4, XSIZE = 483, YSIZE = 410, TITLE = 'Poly_2D image' 
TV, warpPolyImg 
 
    The following image shows little difference from the WARP_TRI image other than more accurate placement in the display window. 
    

    
   
    Figure 5-4: Warped Image Produced with POLY_2D
    
  
         
    

    

    imgwarp04.gif
    


    
    		

    

    

    

     
    Direct Graphics displays in IDL allow you to display a combination of images in the same Direct Graphics window. The following steps display various intensities of the warped image and the reference image in a Direct Graphics window.
  6. Use the XPALETTE tool to view the color table applied to the bone scan image by first entering: 
    7. XPALETTE 
 
    In the XPALETTE utility, display a color table by selecting the Predefined button. In the resulting XLOADCT dialog, scroll down and select Hue Saturation Lightness 2. Click Done. In the XPALETTE utility, click Redraw. Compare the bone scan image, displayed in window 2, to the displayed color table. To mask out the less important background information, select a color close to that of the body color in the image.
     
    The following figure displays a portion of the XPALETTE utility with such a selection. 
    

    
   
    Figure 5-5: Using XPALETTE to Identify Mask Values
    
  
         
    

    

    Warping_Images_Using_Direct_Graphics-5.jpg
  7. Using the knowledge that the body color's index number is 55, mask out the less important background information of the bone scan image by creating an array containing only pixel values greater than 55. Multiply the mask by the image to retain the color information and use BYTSCL to scale the resulting array from 0 to 255: 
    8. ctboneMask = BYTSCL((ctboneImg GT 55) * ctboneImg)
  8. Display a blended image using the full intensity of the bone scan image and a 75% intensity of the warped image. The following statement displays the pixels in the bone scan with the full range of colors in the color table while using the lower 75% of the color table values for the warped image. After adding the arrays, scale the results for display purposes: 
    9. blendImg = BYTSCL(ctboneMask + 0.75 * warpPolyImg)
  9. Create a window and display the result: 
    10. WINDOW, 5, XSIZE = 483, YSIZE = 410, TITLE = 'Blended Image' 
TV, blendImg 
 
    The clavicles and rib bones of the reference image are clearly displayed in the following figure. 
    

    
   
    Figure 5-6: Direct Graphics Display of a Transparent Blended Image
    
  
         
    

    

    imgwarp06.gif

While Direct Graphics supports displaying indexed images as transparent blended images, you could also apply alpha blending to RGB images that are output to a TrueColor display. However, creating image transparencies which retain their color information is more easily accomplished using Object Graphics. For an example of using Object Graphics to display a warped image transparency over another image see Warping Image Objects (Object Programming).