Face Recognition (Transfer Learning)

This article is about understanding the concept of transfer learning.

Transfer learning is a research problem in machine learning that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem.

So basically in transfer learning we use the architecture of pre-created models. We freeze all the layers of the pre-trained models and add few extra layers of our own. Freezing the layers doesn't mean that the data won't passed through them, instead it will only use the weights of the previous layers to achieve better accuracy through the new layers added.

In this practical we will use the famous 'MobileNet' model and add few extra layers to it to use it as a face recognition model.

About the dataset:

I have collected few of my old photos and some random human faces from the internet (the images are not much, but still i was able to achieve 99% accuracy because of transfer learning) .

The code is as follows:

from keras.applications import MobileNet

# MobileNet was designed to work on 224 x 224 pixel input images sizes
img_rows, img_cols = 224, 224

# Re-loads the MobileNet model without the top or FC layers
MobileNet = MobileNet(weights = 'imagenet',
                 include_top = False,
                 input_shape = (img_rows, img_cols, 3))

# Here we freeze the last 4 layers
# Layers are set to trainable as True by default
for layer in MobileNet.layers:
    layer.trainable = False

    

# Let's print our layers
for (i,layer) in enumerate(MobileNet.layers):
   print(str(i) + " "+ layer.__class__.__name__, layer.trainable)

This snippet imports the MobileNet model and excludes the top-most layer so that we can add it by ourselves.

def addingLayers(bottom_model, num_classes):
    """creates the top or head of the model that will be
    placed ontop of the bottom layers"""

    top_model = bottom_model.output
    top_model = GlobalAveragePooling2D()(top_model)
    top_model = Dense(1024,activation='relu')(top_model)
    top_model = Dense(1024,activation='relu')(top_model)
    top_model = Dense(512,activation='relu')(top_model)
    top_model = Dense(num_classes,activation='softmax')(top_model)

    return top_model

Just a function to add our layers to the model.

from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten, GlobalAveragePooling2D
from keras.layers import Conv2D, MaxPooling2D, ZeroPadding2D
from keras.layers.normalization import BatchNormalization
from keras.models import Model

# Set our class number to 3 (Young, Middle, Old)
num_classes = 2

FC_Head = addingLayers(MobileNet, num_classes)

model = Model(inputs = MobileNet.input, outputs = FC_Head)

print(model.summary())

This snippet executes the function defined above and prints the model summary.

from keras.preprocessing.image import ImageDataGenerator

train_data_dir = 'path_to_training_data'
validation_data_dir = 'path_to_testing_data'

# Let's use some data augmentaiton
train_datagen = ImageDataGenerator(
      rescale=1./255,
      rotation_range=45,
      width_shift_range=0.3,
      height_shift_range=0.3,
      horizontal_flip=True,
      fill_mode='nearest')
 
validation_datagen = ImageDataGenerator(rescale=1./255)
 
# set our batch size (typically on most mid tier systems we'll use 16-32)
batch_size = 32
 
train_generator = train_datagen.flow_from_directory(
        train_data_dir,
        target_size=(img_rows, img_cols),
        batch_size=batch_size,
        class_mode='categorical')
 
validation_generator = validation_datagen.flow_from_directory(
        validation_data_dir,
        target_size=(img_rows, img_cols),
        batch_size=batch_size,
        class_mode='categorical')

This snippet imports the images from the storage to train it.

from keras.optimizers import RMSprop
from keras.callbacks import ModelCheckpoint, EarlyStopping

                     
checkpoint = ModelCheckpoint("myface_recog.h5",
                             monitor="val_loss",
                             mode="min",
                             save_best_only = True,
                             verbose=1)

earlystop = EarlyStopping(monitor = 'val_loss',
                          min_delta = 0,
                          patience = 3,
                          verbose = 1,
                          restore_best_weights = True)

# we put our call backs into a callback list
callbacks = [earlystop, checkpoint]

# We use a very small learning rate
model.compile(loss = 'categorical_crossentropy',
              optimizer = RMSprop(lr = 0.001),
              metrics = ['accuracy'])

# Enter the number of training and validation samples here
nb_train_samples = 1097
nb_validation_samples = 272

# We only train 5 EPOCHS
epochs = 5
batch_size = 16

history = model.fit_generator(
    train_generator,
    steps_per_epoch = nb_train_samples // batch_size,
    epochs = epochs,
    callbacks = callbacks,
    validation_data = validation_generator,
    validation_steps = nb_validation_samples)

Training of the model happens after this snippet.

In just 5 epochs, i was able to achieve more than 98% accuracy.

In the other code, we import out model and test it.

from keras.models import load_model

classifier = load_model('myface_recog.h5')

I will be using this model for to test if it detects my face or not.

import os
import cv2
import numpy as np
from os import listdir
from os.path import isfile, join

face_recog_dict = {"[0]": "Gaurav Goyal", //My name
                      "[1]": "Not Gaurav Goyal"}

face_recog_dict_n = {"n0": "Gaurav Goyal",
                      "n1": "Not Gaurav Goyal"}

def draw_test(name, pred, im):
    face = face_recog_dict[str(pred)]
    BLACK = [0,0,0]
    expanded_image = cv2.copyMakeBorder(im, 80, 0, 0, 100 ,cv2.BORDER_CONSTANT,value=BLACK)
    cv2.putText(expanded_image, face, (20, 60) , cv2.FONT_HERSHEY_SIMPLEX,1, (0,0,255), 2)
    cv2.imshow(name, expanded_image)

def getRandomImage(path):
    """function loads a random images from a random folder in our test path """
    folders = list(filter(lambda x: os.path.isdir(os.path.join(path, x)), os.listdir(path)))
    random_directory = np.random.randint(0,len(folders))
    path_class = folders[random_directory]
    print("Class - " + face_recog_dict_n[str(path_class)])
    file_path = path + path_class
    file_names = [f for f in listdir(file_path) if isfile(join(file_path, f))]
    random_file_index = np.random.randint(0,len(file_names))
    image_name = file_names[random_file_index]
    return cv2.imread(file_path+"/"+image_name)    

for i in range(0,10):
    input_im = getRandomImage("path_to_training_data")
    input_original = input_im.copy()
    input_original = cv2.resize(input_original, None, fx=0.5, fy=0.5, interpolation = cv2.INTER_LINEAR)
    
    input_im = cv2.resize(input_im, (224, 224), interpolation = cv2.INTER_LINEAR)
    input_im = input_im / 255.
    input_im = input_im.reshape(1,224,224,3)
    
    # Get Prediction
    res = np.argmax(classifier.predict(input_im, 1, verbose = 0), axis=1)
    
    # Show image with predicted class
    draw_test("Prediction", res, input_original)
    cv2.waitKey(0)

cv2.destroyAllWindows()

This code will print the correct answer on the console and will open an image to predict the image, we can then cross check the working of our model. In my case, I have tested the model for 5 images and it predicted all of them accurately.

As you can see the first prediction is right, let's test it for another image.

The second prediction is also right.

Transfer learning can save our time and resources to give better accuracy, we don't have to worry about creating an architecture or devoting days to train that model.

Thank you,

Gaurav Goyal