Monday 5 September 2011

Scale Images in Java


When working on image manipulation using Java I found there are so many ways to scale an image. I wanted a fast way, but also one that used averaging. Not all scaling is the same. When you scale down there are different filtering techniques to give different results. The fastest ways do not use any filtering at all, but look the worst. Averaging is smoother looking but take longer. I decided to try out as many scaling algorithms I could find, plus one I did myself. Listed below is a number of different methods to scale an image.

Also you may need to take into consideration any dependencies. I recently helped a friend out with scaling an image for the Android platform. He wanted an averaging algorithm but with no dependencies. My custom algorithm did the trick, but was slow. In the end for best results we combined 2 scaling methods. First a fast to get to a medium size, then the averaging.

First I will show you a table of results, showing the scaled image (160 pixels wide) at double size (320 pixels wide so you can see the pixels closely), the time taken to scale image from 3020 pixels wide to 160 pixels wide, dependencies and classes used.

All these times are based on a Macbook 2.4GHz Intel Core 2 Duo, 4GB RAM with Snow Leopard 10.6.8 installed.

Figure 1. Original Image
In Figure 1 I show the original image I used for this test. The original image size is 3020 x 2008 pixels.

Image (160x120) - Double Size Details

Scale NameScaled Instance - Default
MethodBufferedImage.getScaledInstance()
DependenciesBufferedImage
Filterdefault
Time (ms)940


Scale NameScaled Instance - Average
MethodBufferedImage.getScaledInstance()
DependenciesBufferedImage
Filterarea averaging
Time (ms)970


Scale NameScaled Instance - Fast
MethodBufferedImage.getScaledInstance()
DependenciesBufferedImage
Filter-
Time (ms)542


Scale NameScaled Instance - Replicate
MethodBufferedImage.getScaledInstance()
DependenciesBufferedImage
Filterreplicate
Time (ms)507

Scale NameScaled Instance - Smooth
MethodBufferedImage.getScaledInstance()
DependenciesBufferedImage
Filtersmooth
Time (ms)859

Scale NameGraphics2D draw
MethodGraphics2D.drawImage()
DependenciesGraphics2D
Filter-
Time (ms)4

Scale NameSimple Fast
Methodcustom
Dependencies-
Filter-
Time (ms)3

Scale NameGraphics 2D with RenderingHint - Nearest Neighbour
MethodGraphics2D.setRenderingHint()
DependenciesGraphics2D
FilterNearest Neighbour
Time (ms)1

Scale NameGraphics 2D with RenderingHint - Bilinear
MethodGraphics2D.setRenderingHint()
DependenciesGraphics2D
FilterBilinear
Time (ms)1

Scale NameGraphics 2D with RenderingHint - Bilinear quality
MethodGraphics2D.setRenderingHint()
DependenciesGraphics2D
FilterBilinear with quality on
Time (ms)77

Scale NameGraphics 2D with RenderingHint - Bicubic
MethodGraphics2D.setRenderingHint()
DependenciesGraphics2D
FilterBilinear with quality on
Time (ms)3

Scale NameAffineTransformOp
MethodAffineTransform.getScaleInstance()
DependenciesBufferedImageOp, AffineTransformOp
FilterBicubic
Time (ms)578

Scale NameJAI scale
MethodJAI.create("scale",,)
DependenciesJAI - installed native
FilterInterpolationNearest
Time (ms)173

Scale NameJAI SubsampleAverage
MethodJAI.create("SubsampleAverage",,)
DependenciesJAI - installed native
FilterSubsampling - Average
Time (ms)554

Scale NameCustom Average
Methodcustom
Dependencies-
FilterAverage
Time (ms)1354

1. Scaled Instance -with filter
The Java BufferedImage class has its own scaling method called getScaledInstance(). Its not a fast method call but the code is small and only depends on BufferedImage and Graphics2D classes. The method also provides the ability to add filtering. The filter type can be used with the scaleType parameter in the method call below. The filter types are
  • Image.SCALE_DEFAULT
  • Image.SCALE_AREA_AVERAGING
  • Image.SCALE_FAST
  • Image.SCALE_REPLICATE
  • Image.SCALE_SMOOTH
All these types are shown in the results table at the top of this blog. Average and Smooth giving the best results.

private static BufferedImage scaleWithInstance(BufferedImage source, int desiredWidth, int desiredHeight, int scaleType){
     Image temp1 = source.getScaledInstance(desiredWidth, desiredHeight, scaleType);
        BufferedImage image = new BufferedImage(desiredWidth, desiredHeight, BufferedImage.TYPE_INT_ARGB);
        Graphics2D g = (Graphics2D)image.getGraphics();
        g.drawImage(temp1, 0, 0, null);
        
        return image;
    } 

2. Graphics2D draw
The Graphics2D class also provides a scaling method. Using the drawImage method giving the desired width and height, the method will scale for you. This is one of the fastest calls but it does not use any smoothing filter.
private static BufferedImage scaleImage1(BufferedImage source, int desiredWidth, int desiredHeight){
     BufferedImage image = new BufferedImage(desiredWidth, desiredHeight, BufferedImage.TYPE_INT_RGB);
     Graphics2D g = (Graphics2D)image.getGraphics();
     g.drawImage(source, 0, 0, desiredWidth, desiredHeight, null);
     
     return image;
    } 

3. Custom Fast
This method is one of the fastest scaling methods. It does not depend on any classes, so it can be used in a wide variety of platforms, for example Android. It does not use any smoothing filter. BufferedImage is used in the code but this is only to get to the data buffer (biPixels in code). The data buffer is just one large array of pixels.
public static BufferedImage scaleImage2(BufferedImage image, Dimension size) {
     // If no size given, return original size
     if (size == null || size.width == 0 || size.height == 0) return image;
     
     int width = (int)size.getWidth();
     int height = (int)size.getHeight();
     int origWidth = image.getWidth();
     int origHeight = image.getHeight();
     
     float imageAspect = (float)origWidth / (float)origHeight;
     float canvasAspect = (float)width/(float)height;

     int imgWidth = width;
     int imgHeight = height;
     if (imageAspect < canvasAspect) {
      // Change width
      float w = (float)height * imageAspect;
      imgWidth = (int) w;
     } else {
      // Change height
      float h = (float)width / imageAspect;
      imgHeight = (int) h;
     }
     
     BufferedImage bi = new BufferedImage(imgWidth, imgHeight, BufferedImage.TYPE_INT_RGB);

        WritableRaster biRaster = bi.getRaster();
        int[] biPixels = ( (DataBufferInt) biRaster.getDataBuffer()).getData();           
        
        int count = 0;
        
        float xr = (float)image.getWidth() / (float)imgWidth;
        float yr = (float)image.getHeight() / (float)imgHeight;
        float r = xr;
        if (yr < xr) r = yr;

        int row = 0;
        int col = 0;
        float x = 0; float y = 0;
        for (row = 0; row < imgHeight; row++) {
         x = 0;
         for (col = 0; col < imgWidth; col++) {
          int rgb = image.getRGB((int)x,(int)y);
          x += r;
          biPixels[count]=rgb;
          count++;
         }
         y += r;
        }
        
        return bi;
    }

4. Graphics2D with Rendering Hints
The Graphics2D class provides a scaling method with filtering options which they call Rendering Hints. This method includes a higher quality option for a better smoothing effect. The Rendering Hints available are:
  • RenderingHints.VALUE_INTERPOLATION_NEAREST_NEIGHBOR
  • RenderingHints.VALUE_INTERPOLATION_BILINEAR
  • RenderingHints.VALUE_INTERPOLATION_BICUBIC

public static BufferedImage scaleImage3(BufferedImage img,
                                           int targetWidth,
                                           int targetHeight,
                                           Object hint,
                                           boolean higherQuality)
    {
        int type = (img.getTransparency() == Transparency.OPAQUE) ?
            BufferedImage.TYPE_INT_RGB : BufferedImage.TYPE_INT_ARGB;
        BufferedImage ret = (BufferedImage)img;
        int w, h;
        if (higherQuality) {
            // Use multi-step technique: start with original size, then
            // scale down in multiple passes with drawImage()
            // until the target size is reached
          w = img.getWidth();
          if (w < targetWidth) {
           w = targetWidth;
          }
          h = img.getHeight();
          if (h < targetHeight) {
           h = targetHeight;
          }
        } else {
            // Use one-step technique: scale directly from original
            // size to target size with a single drawImage() call
            w = targetWidth;
            h = targetHeight;
        }
        
        do {
            if (higherQuality && w > targetWidth) {
                w >>= 1;
                if (w < targetWidth) {
                    w = targetWidth;
                }
            }

            if (higherQuality && h > targetHeight) {
                h >>= 1;
                if (h < targetHeight) {
                    h = targetHeight;
                }
            }

            BufferedImage tmp = new BufferedImage(w, h, type);
            Graphics2D g2 = tmp.createGraphics();
            g2.setRenderingHint(RenderingHints.KEY_INTERPOLATION, hint);
            g2.drawImage(ret, 0, 0, w, h, null);

            ret = tmp;
        } while (w != targetWidth || h != targetHeight);

        return ret;
    }
5. AffineTransform 
The AffineTransform class has a method called getScaleInstance() that can be used for scaling. This method is not fastest but it also allows Rendering Hints for filtering.

private static BufferedImage scaleImage8(BufferedImage source, int desiredWidth, int desiredHeight){
     double scale = (double)desiredWidth / (double)source.getWidth();
     BufferedImageOp op = new AffineTransformOp(
                AffineTransform.getScaleInstance(scale, scale),
                new RenderingHints(RenderingHints.KEY_INTERPOLATION,
                                   RenderingHints.VALUE_INTERPOLATION_BICUBIC));
     
     return op.filter(source, null);
    } 

6. JAI
JAI is one of the most flexible scaling methods here. JAI will work without installing native libraries but will be very slow. So to take full advantage of JAI install the native libraries for your machine. JAI also allows a variety of filtering options like smoothing (not shown here).

public static BufferedImage scaleImage9(BufferedImage image, int desiredWidth, int desiredHeight)
    {
     float scale = (float)desiredWidth / (float)image.getWidth();

     ParameterBlock pb = new ParameterBlock();
     pb.addSource(image); // The source image
     pb.add(scale);          // The xScale
     pb.add(scale);          // The yScale
     pb.add(0.0F);           // The x translation
     pb.add(0.0F);           // The y translation
     pb.add(new InterpolationNearest()); // The interpolation 
//     pb.add(Interpolation.getInstance(Interpolation.INTERP_BILINEAR));

     RenderedOp resizedImage = JAI.create("scale", pb, null);
     return resizedImage.getAsBufferedImage();
    }

7. JAI Subsampling
JAI also provides a way to scale your image using a technique called subsampling. Basically subsampling only reads part of your JPG image, eg every xth row. This provides a much faster way of scaling an image, but also does averaging to give your image a nice smooth look. Ive only got subsampling to work with JPGs.

public static BufferedImage scaleImage10(File file,
      int desiredWidth, int desiredHeight) throws IOException {
     
     RenderedOp sourceFile = null;
     FileSeekableStream fss = new FileSeekableStream(file);
     sourceFile = JAI.create("stream", fss);
     
     
     double scale = (double) desiredWidth / (double) sourceFile.getWidth();
     
     ParameterBlock pb = new ParameterBlock();
     pb.addSource(sourceFile); // The source image
     pb.add(scale); // The xScale
     pb.add(scale); // The yScale
     pb.add(0.0F); // The x translation
     pb.add(0.0F); // The y translation
     
//     RenderingHints qualityHints = new RenderingHints(
//       RenderingHints.KEY_RENDERING,
//       RenderingHints.VALUE_RENDER_QUALITY);
//     
     RenderedOp resizedImage = JAI.create("SubsampleAverage", pb);//,
//       qualityHints);
     
     return resizedImage.getAsBufferedImage();
    }

8. Custom Averaging
This method I created myself as a custom way to use a smoothing filter without any dependencies. This is not the fastest averaging method, but its the only one that does not have dependencies. Basically the way it works is it creates a moving average value for each column. It stores these in an array for each colour (RGB).

public static BufferedImage scaleImage11(BufferedImage image, Dimension size) {
     // Sharpen level. 10 is not sharpening (uses all pixels to work out average), 1 is no averaging, Dont go higher than 10.
     final int SHARPEN_LEVEL = 10;
     
     // If no size given, return original size
     if (size == null || size.width == 0 || size.height == 0) return image;
     
     int width = (int)size.getWidth();
     int height = (int)size.getHeight();
     int origWidth = image.getWidth();
     int origHeight = image.getHeight();
     
     float imageAspect = (float)origWidth / (float)origHeight;
     float canvasAspect = (float)width/(float)height;

     int imgWidth = width;
     int imgHeight = height;
     if (imageAspect < canvasAspect) {
      // Change width
      float w = (float)height * imageAspect;
      imgWidth = (int) w;
     } else {
      // Change height
      float h = (float)width / imageAspect;
      imgHeight = (int) h;
     }
     
     BufferedImage bi = new BufferedImage(imgWidth, imgHeight, BufferedImage.TYPE_INT_RGB);

        WritableRaster biRaster = bi.getRaster();
        int[] biPixels = ( (DataBufferInt) biRaster.getDataBuffer()).getData();           
        
        int count = 0;
        
        float xr =  (float)imgWidth / (float)image.getWidth();
        float rx =  (float)image.getWidth() / (float)imgWidth;
        int rxi =  ((int)rx) * SHARPEN_LEVEL / 10;
        float yr = (float)imgHeight / (float)image.getHeight();
        int sb = (int)rxi / 2;
        
        // Red, Green, Blue arrays
        long[] ra = new long[origWidth];
        int[] ga = new int[origWidth];
        int[] ba = new int[origWidth];
        
        int row = 0;
        int col = 0;
        float posy = 0;
        int colCount = 0;
        int owm1 = origWidth - 1;
        for (row = 0; row < origHeight; row++) {
      colCount++;
         posy += yr;
         float posx = 0;
      for (col = 0; col < origWidth; col++) {
    int ir = image.getRGB(col,row);
    int r = ir & 0x00FF0000;
    int g = ir & 0x0000FF00;
    int b = ir & 0x000000FF;
    int ro = 0;
    int go = 0;
    int bo = 0;
    if (row >= rxi) {
     int or = image.getRGB(col,row-rxi);
     ro = or & 0x00FF0000;
     go = or & 0x0000FF00;
     bo = or & 0x000000FF;
    }
    // Keep a running total of red, green, blue
    // Add arrays
             ra[col] += r;
             ga[col] += g;
             ba[col] += b;
             // Must subtract from running total
             // Subtract old
             ra[col] -= ro;
             ga[col] -= go;
             ba[col] -= bo;

             posx += xr;
             // Write pixel
             if ((posx > 1f || (col == owm1 && colCount < imgWidth)) && (posy > 1f)) {
              long rt = 0;
              int gt = 0;
              int bt = 0;
              // if sb is 0, not much scaling so dont do averaging
              if (sb == 0) {
         rt = ra[col];
         gt = ga[col];
         bt = ba[col];
              } else {
               int ct = 0;
               for (int k = (col - sb); k < (col + sb); k++) {
                if (k >= 0 && k < origWidth) {
                 ct++;
                 rt += ra[k];
                 gt += ga[k];
                 bt += ba[k];
                }
               }
               if (ct == 0) ct = 1;
               rt = (rt / ct / rxi) & 0x00FF0000;
               gt = (gt / ct / rxi) & 0x0000FF00;
               bt = (bt / ct / rxi) & 0x000000FF;
              }
              
           int rgb = (int)rt | gt | bt;
              
              biPixels[count]=rgb;
              count++;
             }
          if (posx > 1f) posx -= 1f;
            }
      // Should not be greater than 1
      if (posy > 1f) posy -= 1f;
      
      colCount = 0;
     }

        return bi;
    }

Conclusion
There are many ways to scale an image in Java, but which is the right way for you. If you dont care about filtering and just want the fastest then the Graphics2D or Custom Fast is the better option. If you need a fast averaging routine the Graphics2D with quality on or JAI is your best option. If you cannot have any dependencies for example writing your own mobile platform package, using the custom methods is your only option. I found best results by combining the methods listed above. For example scaling first with the custom fast method to a certain size then custom averaging for remaining size.