Scrolling web browser by hand gesture using CV2 module

I have practised this project cause of the learning purpose. Using this program, anyone can scroll through any website. The program can detect any red colour object and by helping this object the program can scroll down-to-top. You need to use any red colour object to scroll.

Doing this project, I have used three modules. The first one is cv2 or computer vision 2, which is mostly used in computer vision, machine language and image processing. This python module commonly uses in real-time project. The second one is the NumPy library, which is working for arrays and 50x faster than a traditional array. It has worked for linear algebra, Fourier transform and matrices. The last one is pyautogui. it's special to control the mouse and keyboard.

import cv2
import numpy as np
import pyautogui

Define a range of red colour, which is defined as BGR format.

lower_red = np.array([0, 80, 80])
upper_red = np.array([10, 255, 255])
prev_y = 0

Using this VideoCapture we define an object. Device index 0 or 1 we pass as an argument. You can capture frame-by-frame. In the end, don't forget to release the capture.

cap = cv2.VideoCapture(0)

 The read function gives a boolean value, if the camera work correctly, then it provides True, otherwise False.

ret, frame = cap.read()

The resize function is used to scaling the image. The first parameter refers to an image source, the second one the pixel size of images and the last one is the interpolation, here we used INTER_AREA to shrinking any image.

frame = cv2.resize(frame, (342, 192), interpolation=cv2.INTER_AREA)

The cvtColor or convert colour one to another. There is 150 colour space conversion in this function.

hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

The inRange function uses to identify the colour range. Here the hsv is the image source, lower_red and the upper_rad value indicates the rage of colour.

mask = cv2.inRange(hsv, lower_red, upper_red)

 The findContours is highly used in shape analysis, object detection and recognition. As you can see there are three parameters, the first one working as a source, the second one to make retrieval mode and the last one is contour approximation method.

contours, hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

The contourArea function is used to calculate the detected area by the findContours function.

area = cv2.contourArea(c)

This function considers a straight rectangle. x and y define the top-left corner and identify the width and height.

x, y, w, h = cv2.boundingRect(c)

Draw a rectangle with this function.

cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)

Pretty awesome logic to move down by space key.

if y < prev_y:
    pyautogui.press('space')

Make two windows. One for Hue Saturation Value and another one for the normal image.

cv2.imshow("HSV Frame", mask)
cv2.imshow('Video Frame', frame) 

Wait for the press a key

if cv2.waitKey(10) == ord("q"):
        break

All of the code included in the while loop.

while True:
    ret, frame = cap.read()
    frame = cv2.resize(frame, (342, 192), interpolation=cv2.INTER_AREA)
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    mask = cv2.inRange(hsv, lower_red, upper_red)
    contours, hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    for c in contours:
        area = cv2.contourArea(c)
        if area > 400:
            x, y, w, h = cv2.boundingRect(c)
            cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
            if y < prev_y:
                pyautogui.press('space')
            prev_y = y
    
    cv2.imshow("HSV Frame", mask)
    cv2.imshow('Video Frame', frame)    
    if cv2.waitKey(10) == ord("q"):
        break

It is used to release an acquire lock. If the lock is locked, the function does the unlock. This function can demolish all of the windows we create.

cap.release()
cv2.destroyAllWindows()

To get source code: https://bit.ly/3rxIKUc