Mathless Explanation: Facial Recognition by Eigenfaces

When you unlocked your phone to read this, you're likely to have unlocked it by simply letting the camera scan your face. Have you ever wondered how it works? Well, under the hood is some complex math, but I'm hoping to explain it pretty simply here.

Well, let's start with the name - it explains a lot. The 'faces' part of eigenfaces refers to the process of splitting several images of a subject into an array, then stacking those arrays into a matrix. The 'eigen' part of the name comes from a math term, specifically eigenvectors. I won't bore you with the details, but all matrices have these special vectors with values (eigenvalues) that they correspond to.

We find the eigenvectors that associate with the largest eigenvalues, which I've used to build these 'ghost' images you see above. If we simply take the first eigenvector from a series of subjects, and compare it with the array of an image, the closest match will always have the lowest error, simply the distance between points. So the subject from which we've found this eigenface is the subject of the test image. Of course, the distance between two points in this case is in roughly 4800 dimensions is irrelevant. There are simple linear algebra methods of determining it (called the norm). So, this early image recognition system would store some of the first eigenvectors for a subject, and compare them with the new image.

This works great for things like employee access (where you have a database of images and can compare them), and it can be built upon to have certain tolerances for an approval. In any case, that's my easy explanation of the process of early facial recognition using the eigenfaces approach.