Image Processing Tutorial Using OpenCV

Jaden Booher

Introduction

Machine/deep learning and artificial intelligence have had a massive impact on the development of technology over the last decade. However, with the consistent development of computer vision, the capability for computers to identify and process an image, there is an ever-expanding amount of use cases. From IoT to self-driving, to facial recognition and biometrics, image processing is at the core of these rapidly growing technologies.

What is image processing?

Image processing is “the process of transforming an image into a digital form and performing certain operations to get some useful information from it” (Simplilearn). The purpose is to transform the original image into a new form that can be digested and used by a software. Image processing examples include, but are not limited to, visualization (finding an object not visible in the image), sharpening and restoration (enhancing the image), recognition, (detecting an object in an image), and retrieval (reverse search a database based on the image). In this tutorial, we will explain some of the basic image processing that can be done using Python’s OpenCV library.

Tutorials:

Tutorial 1: Changing Color Spaces

One of the most basic forms of image processing is changing color spaces. There are approximately 150 different color spaces included in the OpenCV library, but this tutorial will focus on the conversion from BGR, which is like RGB but with an inverse subpixel arrangement, to grayscale, also known as black and white. Lots of image processing software and algorithms require greyscale, as this removes unnecessary complexities to allow for more accurate and efficient processing. Let's try the following code:

import cv2

image = cv2.imread('cookies.jpg') # import image 
cv2.imshow('Original',image) # display original image 
grayscale = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) # swap colorspace
cv2.imshow('Grayscale', grayscale) # display grayscale image 

As expected, we were able to convert the colored image to greyscale:

Tutorial 2: Histogram of Color Channels

Another useful feature of OpenCV is the color channel histogram function. A histogram of color channels is a way to visualize how the colors in an image are distributed across different intensity levels. It works by creating a graph that shows the frequency of occurrence of different color values in an image. Color channel histograms can be used in image processing to analyze pixel value distribution, adjust brightness and contrast, segment regions or objects, and recognize objects based on color or texture. They provide valuable insights into the composition of images and can help develop more effective image processing algorithms. Here, we will show how to create a color histogram using OpenCV and the matplotlib library:

import cv2
import numpy as np
from matplotlib import pyplot as plt

img = cv2.imread('house.jpg') # import image
color = ('b','g','r') # define the 3 plots for blue, green, and red

# code for plotting each pixel
for i,col in enumerate(color):
    histr = cv2.calcHist([img],[i],None,[256],[0,256])
    plt.plot(histr,color = col)
    plt.xlim([0,256])
plt.show() # display hisogram
cv2.imshow('Original',image) # display original image 

As you can see below, the color channel histogram uses colored lines to graph the number of occurrences of each pixel based on their color values and number of occurances:

Tutorial 3: Image Smoothing

A third way to use OpenCV for image processing is image smoothing, a.k.a. image blurring, which is a technique used in image processing to reduce noise and sharpness in an image. This noise is removed because it often provides no valuable information and can sometimes lead to image processing and storage issues. Overall, image smoothing’s purpose is to make pictures more visually appealing and easier to analyze by removing unwanted details and emphasizing important features. For example, the picture of the ducklings below has an excessive amount of noise due to the complexities of their feathers sticking out. Here, we will apply 2 different filters to the image of the ducklings below. These filters work by averaging the pixel values in the neighborhood of each pixel, resulting in a smoother image with fewer high-frequency components. Different-sized kernels will result in different extents of blurring. Here, we will use OpenCV and matplotlib to apply a 5x5 filter, which will select the middle pixel and average it with the pixels all around it in a 5x5 pixel area:

import cv2
import numpy as np
from matplotlib import pyplot as plt

img = cv2.imread('ducklings.png') # import image 

kernel = np.ones((5,5),np.float32)/25 # create a kernel to filter with 
dst = cv2.filter2D(img,-1,kernel) # convolve the kernel with the image

# display the images
plt.subplot(121),plt.imshow(img),plt.title('Original')
plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(dst),plt.title('An image smoothed with gaussian smoothing')
plt.xticks([]), plt.yticks([])
plt.show()

Here, you can see how the image on the right is blurred but is still recognizable:

Here is the smoothing done again, but with a 7x7 filter. Notice how it is more blurred than the 5x5:

Tutorial 4: Interactive Foreground Extraction

Have you ever wanted to remove the background of the image but don’t know how to use photoshop? Using interactive foreground extraction will remove the focus of an image using the GrabCut algorithm. This algorithm draws a rectangle around the foreground region, then iterating and segmenting the image until the final, focused image is presented. One of the fascinating aspects of this algorithm is that it can be fine tuned using a “mask,” which allows the user to directly specify the foreground and background. Additionally, this algorithm leverages “strokes,” which allows the user to tell the algorithm if it correctly identified the foreground and background after each iteration, to further finetune the results.

import numpy as np
import cv2

img = cv2.imread('person.jpg') # import image

# define mask; this can be further tuned
mask = np.zeros(img.shape[:2],np.uint8)
bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)

rect = (150,50,500,470) # define rectangle based on pixel locations

# apply grabCut method to extract the foreground
cv2.grabCut(img,mask,rect,bgdModel,fgdModel,20,cv2.GC_INIT_WITH_RECT)
mask2 = np.where((mask==2)|(mask==0),0,1).astype('uint8')
img = img*mask2[:,:,np.newaxis]

cv2.imshow('Foreground Image',img) # display the resulting image

Here, you can see how the girl is pulled from the background:

Though there are a couple of issues with her hair, changing the mask will make the foreground extraction more accurate. Here, you can see that the random snippets of her hair are now gone:

Conclusion

In conclusion, image processing is a powerful tool that allows us to manipulate and analyze digital images. In this tutorial, we learned an array of techniques such as changing color spaces, analyzing histograms of color channels, smoothing images, and interactive foreground extraction. Changing color spaces allows us to change the color space of an image to remove unnecessary complexities to allow for more accurate and efficient processing. Analyzing the histograms of color channels is important for identifying pixel value distribution, adjusting brightness and contrast, and recognizing images based on their color and texture. Image smoothing or image blurring is essential for removing noise, improving image quality, and lowering the storage size of an image. Finally, interactive foreground extraction provides us the ability to extract specific objects or regions of an image to isolate them for further analysis. These techniques are just a small sample of the many tools available in image processing and under the OpenCV library, and with further exploration, one can create stunning and informative images that can be used in a wide range of applications and IoT.

WC: 1151