Martynas Sateika

Playing with 2D image filters

Published 2014-01-17

In a recent CS313 Mobile Robotics lecture about vision, we were introduced to 2D filters that are applied to images when processing them. It didn’t click at first, so I decided to give it a go and write a short Java program that would allow me to try out different filters on an image and display them on the screen.

I firstly came up with the ImageFilter class that’d store three main fields: the 2D array with filter info, the filter’s factor value and bias value (more on that later). Some checks were placed to ensure that null isn’t being passed, that the filter’s array is square and its dimensions are odd (otherwise, we couldn’t pinpoint the center of the filter).

Next, I set up an ImageWindow class (extending JFrame) that would display any number of images next to each other. Its constructor accepted an array of BufferedImage, computed the total width required to fit all the images, and then drew them next to each other.

It was then only a matter of setting up the applyFilter() method that would accept a BufferedImage and return a new BufferedImage with the filter applied. The only issue was that BufferedImage’s getSubImage() method, while returning a new BufferedImage, actually shares the underlying pixel data with the calling object, and so a new method had to be written that would return a proper copy. This was achieved by creating a blank BufferedImage with the same dimensions as the source, accessing its underlying Graphics object, and drawing the source image inside.

A 2D filter is a mask that goes around an image and performs computations with pixels underneath it. In the applyFilter() method, I needed four nested for loops: the outer two would iterate through the X and Y values of the image, and the inner two would go along the X and Y values of the filter. Next, the important bit: for every pixel of the image, we iterate through the whole filter, positioned so that its center cell is at that pixel. Therefore, when the inner two loops start at (0, 0) (the upper left cell of the 2D filter), we should be pointing to a pixel of the underlying image that is a bit higher and to the left of what the X and Y values of the two outer loops point to (unless, of course, it’s a 1-by-1 filter).

It might be that the whole filter does not fit on the image (think of a 3-by-3 filter starting at the top-left pixel of the image). The easy way to solve this is to wrap around so the values that didn’t fit end up over pixels on the other side of the image.

What is being computed when iterating through a filter? A new color value of the underlying image’s pixel! As we move the mask over a new pixel of the image, we reset three variables – “red”, “green”, and “blue” which then accumulate the sum of products of the image’s R, G and B values of that pixel (respectively) with the values on the filter. Then, once we finish iterating through the whole filter, we simply update the image’s color at that pixel with the values of “red”, “green”, and “blue”. We don't store them just like that, though, this is where the 'factor' and 'bias' mentioned earlier come into play. We multiply the accumulated values by the 'factor', and then add 'bias'. This gives us more flexibility and allows achieving more effects, such as 'emboss'.

The following image should explain what's going on (x' and y' are coordinates of pixels on the image beneath corresponding x and y. The sigma loops over the filter):

To be able to change between different filters, I created an ImageFilterFactory class, which stored in it a number of ImageFilter objects in static fields for easy access, so creating a new filter for testing was as easy as writing:

public static final ImageFilter IMAGE_FILTER_SHARPEN
    = new ImageFilter(new double[][] {
            {-1.0d, -1.0d, -1.0d},
            {-1.0d, 9.0d, -1.0d},
            {-1.0d, -1.0d, -1.0d}
    }, 1.0d, 0.0d); // filter, factor, bias
// Main class:
ImageFilter filter1 = ImageFilterFactory.IMAGE_FILTER_SHARPEN;

The final result of the program, applying the filter above to an image of Lenna, gives the following (original on the left, image with filter on the right):

Original and altered image.