Pre-processing images with OpenCV

OpenCV is one of the best image processing libraries available - not only does it come with many powerful algorithms like filtering and machine learning just for computer vision but it's also supported across multiple languages (including C++ and Java) and free and Open Source. With Machine Learning quickly becoming an integral part of almost every business, its a good time to deep dive into some of the technology that can teach machines how to make sense of images and video content.

Why Computer Vision?

In the day of the smartphone, we're constantly presented with an incredible amount of digital images and videos. For a long time, a lot of the information gleaned from an image was primarily from the text accompanying the image provided by the user ( description, tags etc). However with the number of images on the internet today, it's important to be able to extract information from the image itself. Teaching machines how to understand the content of an image then, presents itself as enticing problem to solve.

From a distance, most of the challenges faced in the field of Computer Vision seem to be easily solvable by most humans using intuition alone. For example, recognizing the boundaries of an object in two-dimensional space or recognizing people from images they have seen before even if looked at from a different angle. However getting machines to understand these things, is where Computer Vision comes into play

Why OpenCV?

Its free, accessible and extremely powerful. Not only that, but it also has an engaged community which makes both learning and developing solutions much easier.

Image pre-processing

Before sending our image away to an algorithm, its important to preprocess it so that we're feeding in relevant data. Here's a rundown of some useful techniques that can be implemented with OpenCV in order to achieve this

Geometric Transformation:

Just as with any machine learning algorithm, we have to make sure the records of our data compare to each other. With images this can mean scaling/resizing/rotating so that there's a base configuration for our images. OpenCV also provides perspective transformation. The documentation contains great information on how to do this

Denoising:

Since an image is a basically an array of pixels, it can suffer from noise that could mess with the learning algorithms. This can manifest itself as grainy images which if unchecked can throw off computing intensity differences that are important in detecting . The most popular way to remove noise is by applying a smoothing filter like the Gaussian filter.

Some other important filters are bilateral( blurs the image without removing the edges), median filter (which computes the median of pixels under the central window and kernel) or the normalized box filter( which just computes the average).

In addition to this, we can create our own kernels and apply convolutions which allows much greater flexibility

Before and after gaussian:

Bilateral Filter:

Histogram equalization:

Sometimes images can be extremely low in contrast. This can often be visualized in a histogram where intensities are grouped very close to each other. histogram equalization spreads out the intensities improving the contrast of the image

Original historgram:

Equalized Histogram:

Image after histogram equalization:

Colorspace Conversion:

OpenCv reads in images in BGR (Blue Green Red). Sometimes its necessary to convert images into different colorspaces before processing it further. For example,histogram equalization on an RGB image disturbs its colour balance a little to much. Here its favourable to convert HSL( Hue Saturation Luminance) colorspace, apply the equalization and convert back to BGR.

Thresholding:

Thresholded images are basically binary images. Thresholding an image converts it into a matrix of zeros and ones. This is useful when detecting contours or segments. OpenCV comes with functions for both global and adaptive thresholding in addition to an implementation of Otsu's algorithm which finds the best value for a threshold.

Edge detection:

Unlike contours, edges are points that sharply differ from the surrounding image gradient. The most popular to look for edges is using the Sobel operator. With Sobel, we approximate the derivative of the image. This is achieved by using a matrix with positive and negative numbers on opposite sides. This matrix is then moved along the X or Y axes. The difference between the central image and the matrix, if high denotes that there is a sharp increase or decrease in intensity and the area is marked as an edge

Morphological transformations:

Erosion is a method by which pixels are removed from the boundary of the foreground image. This can be a useful technique to determine whether objects touch each other. Dillation involves increasing the boundary pixels of the foreground. Opening and Closing are extensions of erosion and dillation which help remove points of noise

Dilation:

Image Segmentation:

Image segmentation divides up the image into different regions that simplify an image in order to better understand it. A way to do this is using K-means clustering which returns the clusters of colors that are closest to each other in vector space. The clusters can then be replaced with the cluster centers which summarize the image based on the number of cluster centers computed.

Segmentation with K-means:

Contour Detection:

A contour is a curve joining pixels of equal intensity. Contours can be useful in determining the boundary of an image. OpenCV comes with two functions findContours() and drawContours() that find closed curves around an object. This also returns the heirarchy of contours ( contours can be nested within one another). Contours are significant in extracting areas of interest and this differs from image to image.

I hope this serves as a broad perspective into the many ways OpenCV can be used as technology changes to adapt to a world where information can be represented by and derived from images and the many possibilities that this holds.