Basic Image Manipulation

[CG]Maxime

269.3K views

GitHub

Open Source Your Knowledge, Become a Contributor

Technology knowledge has to be shared and made accessible for free. Join the movement.

Create Content

Previous: Colors

Filtering

A lot of image processing algorithms rely on the convolution between a kernel (typicaly a 3x3 or 5x5 matrix) and an image. This may sound scary to some of you but that's not as difficult as it sounds:

Let's take a 3x3 matrix as our kernel. For each pixel, the filter multiplies the current pixel value and the other 8 surrounding pixels by the kernel corresponding value. Then it adds the result to get the value of the current pixel. Let's see an example:

Matrix convolution

In this example, the value of each pixel is equal to the double of the pixel that was located above it (e.g. 92 = 46 x 2).

Blur

A simple blur can be done using this kernel:

\frac{1}{9} [\begin{matrix} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \end{matrix}]

This is called the Box Blur. Each pixel is computed as the average of the surrounding pixels.

And here is the kernel for the Gaussian Blur:

\frac{1}{256} [\begin{matrix} 1 & 4 & 6 & 4 & 1 \\ 4 & 16 & 24 & 16 & 4 \\ 6 & 24 & 36 & 24 & 6 \\ 4 & 16 & 24 & 16 & 4 \\ 1 & 4 & 6 & 4 & 1 \end{matrix}]

As you can see, it's a weighted mean of the surrounding pixels that gives more weight to the pixel near the current pixel. This kind of filter is also called a low-pass filter.

Blur

from PIL import Image, ImageDraw
# Load image:
input_image = Image.open("input.png")
input_pixels = input_image.load()
# Box Blur kernel
box_kernel = [[1 / 9, 1 / 9, 1 / 9],
              [1 / 9, 1 / 9, 1 / 9],
              [1 / 9, 1 / 9, 1 / 9]]
# Gaussian kernel
gaussian_kernel = [[1 / 256, 4  / 256,  6 / 256,  4 / 256, 1 / 256],
                   [4 / 256, 16 / 256, 24 / 256, 16 / 256, 4 / 256],
                   [6 / 256, 24 / 256, 36 / 256, 24 / 256, 6 / 256],
                   [4 / 256, 16 / 256, 24 / 256, 16 / 256, 4 / 256],
                   [1 / 256, 4  / 256,  6 / 256,  4 / 256, 1 / 256]]
# Select kernel here:
kernel = box_kernel
# Middle of the kernel
offset = len(kernel) // 2
# Create output image
output_image = Image.new("RGB", input_image.size)
draw = ImageDraw.Draw(output_image)
# Compute convolution between intensity and kernels
for x in range(offset, input_image.width - offset):
    for y in range(offset, input_image.height - offset):
        acc = [0, 0, 0]
        for a in range(len(kernel)):
            for b in range(len(kernel)):
                xn = x + a - offset
                yn = y + b - offset
                pixel = input_pixels[xn, yn]
                acc[0] += pixel[0] * kernel[a][b]
                acc[1] += pixel[1] * kernel[a][b]
                acc[2] += pixel[2] * kernel[a][b]
        draw.point((x, y), (int(acc[0]), int(acc[1]), int(acc[2])))
    
output_image.save("output.png")

Sharpening

The details of an image can be emphasized by using a high-pass filter:

[\begin{matrix} 0 & - 0.5 & 0 \\ - 0.5 & 3 & - 0.5 \\ 0 & - 0.5 & 0 \end{matrix}]

In this kernel, the pixel is boosted when the neighbor pixels are different.

Sharpening

from PIL import Image, ImageDraw
# Load image:
input_image = Image.open("input.png")
input_pixels = input_image.load()
# High-pass kernel
kernel = [[  0  , -.5 ,    0 ],
          [-.5 ,   3  , -.5 ],
          [  0  , -.5 ,    0 ]]
# Middle of the kernel
offset = len(kernel) // 2
# Create output image
output_image = Image.new("RGB", input_image.size)
draw = ImageDraw.Draw(output_image)
# Compute convolution with kernel
for x in range(offset, input_image.width - offset):
    for y in range(offset, input_image.height - offset):
        acc = [0, 0, 0]
        for a in range(len(kernel)):
            for b in range(len(kernel)):
                xn = x + a - offset
                yn = y + b - offset
                pixel = input_pixels[xn, yn]
                acc[0] += pixel[0] * kernel[a][b]
                acc[1] += pixel[1] * kernel[a][b]
                acc[2] += pixel[2] * kernel[a][b]
        draw.point((x, y), (int(acc[0]), int(acc[1]), int(acc[2])))
    
output_image.save("output.png")

This filter can also be improved by applying the transformation only when the pixel is dark enough.

Another approach, called unsharp mask, consist in substracting from the original image a mask created using a low-pass filter. Basically, sharpening is realized by removed the blurry part of the image: $s h a r p e n e d = o r i g i n a l + (o r i g i n a l - b l u r r e d) \times a m o u n t .$

Unsharp Mask

from PIL import Image, ImageDraw
# Load image:
input_image = Image.open("input.png")
input_pixels = input_image.load()
# Low-pass kernel
kernel = [[1 / 9, 1 / 9, 1 / 9],
          [1 / 9, 1 / 9, 1 / 9],
          [1 / 9, 1 / 9, 1 / 9]]
amount = 2
# Middle of the kernel
offset = len(kernel) // 2
# Create output image
output_image = Image.new("RGB", input_image.size)
draw = ImageDraw.Draw(output_image)
# Compute convolution with kernel
for x in range(offset, input_image.width - offset):
    for y in range(offset, input_image.height - offset):
        original_pixel = input_pixels[x, y]
        acc = [0, 0, 0]
        for a in range(len(kernel)):
            for b in range(len(kernel)):
                xn = x + a - offset
                yn = y + b - offset
                pixel = input_pixels[xn, yn]
                acc[0] += pixel[0] * kernel[a][b]
                acc[1] += pixel[1] * kernel[a][b]
                acc[2] += pixel[2] * kernel[a][b]
        new_pixel = (
            int(original_pixel[0] + (original_pixel[0] - acc[0]) * amount),
            int(original_pixel[1] + (original_pixel[1] - acc[1]) * amount),
            int(original_pixel[2] + (original_pixel[2] - acc[2]) * amount)
        )
        draw.point((x, y), new_pixel)
    
output_image.save("output.png")

Edge detection

There are multiple ways to do edge detection. We will present the Sobel Operator here.

The Sobel Operator uses two kernels (one for each direction): $K_{x} = [\begin{matrix} - 1 & 0 & 1 \\ - 2 & 0 & 2 \\ - 1 & 0 & 1 \end{matrix}]$ and $K_{y} = [\begin{matrix} - 1 & - 2 & - 1 \\ 0 & 0 & 0 \\ 1 & 2 & 1 \end{matrix}]$ .

We compute the convolution between the image (converted in black and white) and the two kernels separately. That gives us, for each pixel, the values $m a g_{x}$ and $m a g_{y}$ . The value of the current pixel is set at $\sqrt{m a g_{x}^{2} + m a g_{y}^{2}}$ .

Sobel operator

from PIL import Image, ImageDraw
from math import sqrt
# Load image:
input_image = Image.open("input.png")
input_pixels = input_image.load()
# Calculate pixel intensity as the average of red, green and blue colors.
intensity = [[sum(input_pixels[x, y]) / 3 for y in range(input_image.height)] for x in range(input_image.width)]
# Sobel kernels
kernelx = [[-1, 0, 1],
           [-2, 0, 2],
           [-1, 0, 1]]
kernely = [[-1, -2, -1],
           [0, 0, 0],
           [1, 2, 1]]
# Create output image
output_image = Image.new("RGB", input_image.size)
draw = ImageDraw.Draw(output_image)
# Compute convolution between intensity and kernels
for x in range(1, input_image.width - 1):
    for y in range(1, input_image.height - 1):
        magx, magy = 0, 0
        for a in range(3):
            for b in range(3):
                xn = x + a - 1
                yn = y + b - 1
                magx += intensity[xn][yn] * kernelx[a][b]
                magy += intensity[xn][yn] * kernely[a][b]
        # Draw in black and white the magnitude
        color = int(sqrt(magx**2 + magy**2))
        draw.point((x, y), (color, color, color))
    
output_image.save("output.png")

Open Source Your Knowledge: become a Contributor and help others learn. Create New Content

Open Source Your Knowledge, Become a Contributor

4/4 Filtering

Filtering

Blur

Sharpening

Edge detection

How to plot the Mandelbrot set

How to build a chatbot in less than 50 lines of code

PYTHON: BEGINNER QUIZ (10 Questions)

PYTHON STRING METHODS: center(), rjust() and ljust().