COMPSCI 194-26: Project 2

Fun with Filters & Frequencies

Table of Contents

Introduction

This project goes over automated image straightening and explores image frequency manipulation through applications like image sharpening, hybrid images (static images that change in interpretation as a function of the viewing distance) and image blending.

I generate hybrid images using the approach described in the SIGGRAPH 2006 paper by Oliva, Torralba, and Schyns. The multi resolution blending was implemented as described in the 1983 paper by Burt and Adelson.

Fun with Filters

1. Finite Difference Operator

The first filter we explore is the finite difference operator. We first find the partial derivatives of the cameraman image by convolving the image with finite difference operators D_x and D_y. Then we compute the gradient magnitude image. One we have that computed, we binarize the gradient magnitude image by picking the appropriate threshold (trying to suppress the noise while showing all the real edges).

2. Derivative of Gaussian (DoG) Filter

We noted that the results with just the difference operator were rather noisy. Luckily, we have a smoothing operator handy: the Gaussian filter G. We create a blurred version of the original image by convolving with a gaussian and repeat the procedure in the previous part.

Now we can do the same thing with a single convolution instead of two by creating a derivative of gaussian filters. We compute those by convolving the gaussian with D_x and D_y.

We observe the edges produced by the two techniques lead the same results using the same threshold, as expected.

3. Image Straightening

Now we will automate the image straightening process! It is known that statistically there is a preference for vertical and horizontal edges in most images (due to gravity!). We will use this insight by trying to rotate an image to maximize the number of vertical and horizontal edges.

To straighten an image, we generate a set of proposed rotations. For each proposed rotation angle:

1. Rotate the image by the proposed rotation angle.
2. Compute the gradient angle of the edges in the image while ignoring the edges of the image created by rotation itself by cropping the image after rotation.

Facade

Campanile

House in SF

Leaning Tower of Pisa

We can visually confirm that the process is working correctly in most cases. The images do appear to be more straight than they were originally and if we look at the distribution of the angles then we observe the straightened image has more angles aligned with the relevant angles. Some results are questionable like Leaning Tower of Pisa which is tilted in its original state. The most obvious failure is shown below where the algorithm tries to straighten the angular design of a museum.

Museum

Fun with Frequencies

1. Image 'Sharpening'

The Guassian filter is a low pass filter that retains only the low frequencies. We can subtract the blurred version from the original image to get the high frequencies of the image. An image often looks sharper if it has stronger high frequencies. That means we can make the image appear sharper by adding these high frequencies back to the original image (left). We can also transform this process into a single convolution operation called the unsharp mask filter (right).

We can see both the techniques produce sharper looking images and both the sharp images look the same because both the processes are equivalent.

Just to confirm that this is actually not adding detail to the image let's take a sharp image, blur it & then attempt to sharpen it again and see the results.

2. Hybrid Images

The goal of here is to create hybrid images using the approach described in the SIGGRAPH 2006 paper by Oliva, Torralba, and Schyns. Hybrid images are static images that change in interpretation as a function of the viewing distance. The basic idea is that high frequency tends to dominate perception when it is available, but, at a distance, only the low frequency (smooth) part of the signal can be seen. By blending the high frequency portion of one image with the low-frequency portion of another, you get a hybrid image that leads to different interpretations at different distances.

Nutrek

Darth Sahai

Lizard Person

Let's see the frequency analysis of the last result to see what's going on under the hood.

3. Gaussian and Laplacian Stacks

Gaussian and Laplacian stacks which are kind of like their pyramid counterparts but without the downsampling which means we can stack them up together in a n-dimensional tensor. Gaussian stack compenents are just the result of applying a Gaussian filter varying sigmas to an image, and the Laplacian stack is made up of the differences between two consecutive Gaussian stack levels.

Lincoln

Lizard Person

4. Multiresolution Blending

Now we blend two images seamlessly using a multi resolution blending as described in the 1983 paper by Burt and Adelson. An image spline is a smooth seam joining two image together by gently distorting them. Multiresolution blending computes a gentle seam between the two images seperately at each band of image frequencies, resulting in a much smoother seam.
Here are some of the results

Oraple

Space Birds

Orange Man

Let's see what the different stacks look like for our last blending result.

Orange

Trump

Mask

Orange Man

I had a lot of fun working on this and learned a lot by playing around with image frequencies. Quite cool!