mastasky
Posts: 2
Joined: Sat Jan 12, 2013 3:09 pm

[Tutorial] Simple Face Recognition (OpenCV, Python)

Sat Aug 30, 2014 12:10 pm

Hi all,

This post is about face recognition (=matching a face against a set of known faces), not face detection (=finding a face in an image). For more information on the difference between the two, have a look at this ppt: http://www.ee.pdx.edu/~mperkows/class_4 ... nition.ppt

I recently started looking into OpenCV's face recognition capabilities for a project of mine and found that whilst information is available on numerous websites and the OpenCV documentation, it is either too abstract or too scientific for someone who just wants to whip up a quick script to do face recognition. Hence why I decided to write this tutorial.

Objective
To create a simple Python application that takes an image and then runs OpenCVs Eigenfaces face recognition algorithm against it - returning the ID of the recognized face plus a confidence level.

Requirements
Libraries
We're going to use Python and require the following modules to be installed (via sudo apt-get install *modulename*):
  • python-dev
  • python-opencv
  • libopencv-dev
  • python-numpy
  • python-scipy
  • libatlas-dev (not sure, but can't hurt)
  • libatlas3gf-base (not sure either)
Face database
In order to run face detection, we need a database of faces to recognize. I used the AT&T Face Database (http://www.cl.cam.ac.uk/research/dtg/at ... abase.html). It contains 40 folders (one folder per subject s) with 10 92x112px grayscale images of that subject's face. The directory structure looks like this::

/
--/s1
--/s1/1.pqm
--/s1/2.pqm
--/s1/...
--/s2/1.pqm
...

I recommend deleting folders s11 to s40 in order to increase computation speed.

You of course also need to set up a folder for the face you want to detect (e.g. yourself). Call that s99 and put 10 pictures (format irrelevant, I used JPG) of yourself into it (92x112, grayscale, different light settings etc).

Solution
The code to run the face recognition is now rather simple. It can be broken down into the following steps:
  1. Load flattened training images into array
  2. Perform principal component analysis on the training images
  3. Load test image(s)
  4. Perform principal component analysis on training image(s)
  5. Calculate euclidian distance from test image to each training image and pick result with the smallest distance (=recognized face)
Note: Make sure that NUM_TRAINIMAGES contains the correct number of all your training images. If this is wrong, you will exceptions!

Code: Select all

from sklearn.decomposition import RandomizedPCA
import numpy as np
import glob
import cv2
import math
import os.path
import string

#function to get ID from filename
def ID_from_filename(filename):
    part = string.split(filename, '/')
    return part[1].replace("s", "")
 
#function to convert image to right format
def prepare_image(filename):
    img_color = cv2.imread(filename)
    img_gray = cv2.cvtColor(img_color, cv2.cv.CV_RGB2GRAY)
    img_gray = cv2.equalizeHist(img_gray)
    return img_gray.flat

IMG_RES = 92 * 112 # img resolution
NUM_EIGENFACES = 10 # images per train person
NUM_TRAINIMAGES = 110 # total images in training set

#loading training set from folder train_faces
folders = glob.glob('train_faces/*')
 
# Create an array with flattened images X
# and an array with ID of the people on each image y
X = np.zeros([NUM_TRAINIMAGES, IMG_RES], dtype='int8')
y = []

# Populate training array with flattened imags from subfolders of train_faces and names
c = 0
for x, folder in enumerate(folders):
    train_faces = glob.glob(folder + '/*')
    for i, face in enumerate(train_faces):
        X[c,:] = prepare_image(face)
        y.append(ID_from_filename(face))
        c = c + 1

# perform principal component analysis on the images
pca = RandomizedPCA(n_components=NUM_EIGENFACES, whiten=True).fit(X)
X_pca = pca.transform(X)

# load test faces (usually one), located in folder test_faces
test_faces = glob.glob('test_faces/*')

# Create an array with flattened images X
X = np.zeros([len(test_faces), IMG_RES], dtype='int8')
 
# Populate test array with flattened imags from subfolders of train_faces 
for i, face in enumerate(test_faces):
    X[i,:] = prepare_image(face)
 
# run through test images (usually one)
for j, ref_pca in enumerate(pca.transform(X)):
    distances = []
    # Calculate euclidian distance from test image to each of the known images and save distances
    for i, test_pca in enumerate(X_pca):
        dist = math.sqrt(sum([diff**2 for diff in (ref_pca - test_pca)]))
        distances.append((dist, y[i]))
 
    found_ID = min(distances)[1]

    print "Identified (result: "+ str(found_ID) +" - dist - " + str(min(distances)[0])  + ")"
Now if you put a picture in the right format for yourself in the test_faces folder and run the script, it should return result 99 (yourself) and a distance of less than 1.5 (the lower the better).

SJH333
Posts: 23
Joined: Sat Feb 23, 2013 4:03 pm

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Tue Oct 14, 2014 3:24 pm

Hi mastasky,
Thanks for the tutorial, yours is the first OpenCV Tutorial I have tried.
Here is some feedback:
I needed to install python-sklearn - missing from your list of Libraries

I copy/pasted you code into a file called, find_face.py (I know I should have typed it out myself!)
Download the ATnT image folders into Folder, train_faces (I only used s1 to s10 as recommended).
Put image 10.pgm from s10 into Folder, test_faces (I didn't have time to take 10 photos of myself)
The Code ran fine but the results were NOT as expected...
I replaced the test image with image 10.pgm from s9 and repeated the test, (10 times using images s10 back to s1)
FOLDERS USED: (ATnT example faces)
-------------
/find_face.py
/train_faces/s1 to s10 inc.
/test_faces/Replaced test image 10.pgm from folders s10 back to s1 for each test run.

Here are the results:

Code: Select all

RESULTS:
--------
Python 2.7.3 (default, Mar 18 2014, 05:13:23) 
[GCC 4.6.3] on linux2
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>> 
Identified (result: 1 - dist - 3.67196857177)     (CORRECT ANSWER IS 10)
>>> ================================ RESTART ================================
>>> 
Identified (result: 1 - dist - 2.84274855471)     (CORRECT ANSWER IS 9)
>>> ================================ RESTART ================================
>>> 
Identified (result: 1 - dist - 4.4124286855)     (CORRECT ANSWER IS 8)
>>> ================================ RESTART ================================
>>> 
Identified (result: 1 - dist - 3.26230101349)     (CORRECT ANSWER IS 7)
>>> ================================ RESTART ================================
>>> 
Identified (result: 1 - dist - 4.44605791408)     (CORRECT ANSWER IS 6)
>>> ================================ RESTART ================================
>>> 
Identified (result: 1 - dist - 3.49160490153)     (CORRECT ANSWER IS 5)
>>> ================================ RESTART ================================
>>> 
Identified (result: 1 - dist - 3.27757229639)     (CORRECT ANSWER IS 4)
>>> ================================ RESTART ================================
>>> 
Identified (result: 1 - dist - 3.57134549162)     (CORRECT ANSWER IS 3)
>>> ================================ RESTART ================================
>>> 
Identified (result: 1 - dist - 4.46452801975)     (CORRECT ANSWER IS 2)
>>> ================================ RESTART ================================
>>> 
Identified (result: 1 - dist - 3.30849531859)     (CORRECT ANSWER IS 1) OK!!!
The results are all for image set 1, which is not true and even for my last test where the answer is correct, it has a high distance value, the test photo is exactly the same as in the training folder(!), I was expecting a near zero distance...

I've enjoyed trying this out and I'm keen to learn some more...
Do you have any ideas what I could try to do to resolve my incorrect answers?

Thanks again for the Tutorial,
Steve

vantruong57
Posts: 6
Joined: Sun May 17, 2015 8:13 am

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Wed Aug 26, 2015 6:30 pm

Hi sir, when I run the code, it return : Importerror: No module name sklearn.Decomposition.
Can u help mi fix it?
Thanks a lot

dheeraj
Posts: 1
Joined: Wed Mar 09, 2016 9:21 am

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Thu Mar 10, 2016 4:33 pm

Hi sir,
when I run the code, it return : Importerror: No module name sklearn.Decomposition.
Can u help me finding the solution for it?

nordnet
Posts: 6
Joined: Thu Mar 10, 2016 11:59 pm

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Fri Mar 11, 2016 12:10 am

This post is getting quite old, but since it was very helpful to me, I thought I should post the code how it worked for me.

Code: Select all

from sklearn.decomposition import RandomizedPCA
import numpy as np
import glob
import cv2
import math
import os.path
import string

#function to get ID from filename
def ID_from_filename(filename):
    part = filename.split('/')
    return part[1].replace("s", "")
 
#function to convert image to right format
def prepare_image(filename):

    img_color = cv2.imread(filename)
    img_gray = cv2.cvtColor(img_color, cv2.COLOR_BGR2GRAY)
    img_gray = cv2.equalizeHist(img_gray)
    return img_gray.flat

IMG_RES = 92 * 112 # img resolution
NUM_EIGENFACES = 10 # images per train person
NUM_TRAINIMAGES = 410 # total images in training set

#loading training set from folder train_faces
folders = glob.glob('train_faces/*')
 
# Create an array with flattened images X
# and an array with ID of the people on each image y
X = np.zeros([NUM_TRAINIMAGES, IMG_RES], dtype='int8')
y = []

# Populate training array with flattened imags from subfolders of train_faces and names
c = 0
for x, folder in enumerate(folders):
    train_faces = glob.glob(folder + '/*')
    for i, face in enumerate(train_faces):
        X[c,:] = prepare_image(face)
        y.append(ID_from_filename(face))
        c = c + 1

# perform principal component analysis on the images
pca = RandomizedPCA(n_components=NUM_EIGENFACES, whiten=True).fit(X)
X_pca = pca.transform(X)

# load test faces (usually one), located in folder test_faces
test_faces = glob.glob('test_faces/*')

# Create an array with flattened images X
X = np.zeros([len(test_faces), IMG_RES], dtype='int8')
 
# Populate test array with flattened imags from subfolders of train_faces 
for i, face in enumerate(test_faces):
    X[i,:] = prepare_image(face)
 
# run through test images (usually one)
for j, ref_pca in enumerate(pca.transform(X)):
    distances = []
    # Calculate euclidian distance from test image to each of the known images and save distances
    for i, test_pca in enumerate(X_pca):
        dist = math.sqrt(sum([diff**2 for diff in (ref_pca - test_pca)]))
        distances.append((dist, y[i]))
 
    found_ID = min(distances)[1]
    print("Identified (result: "+ str(found_ID) +" - dist - " + str(min(distances)[0])  + ")")
It's basically the original code with a few minor adjustments that made it able to run with Python 3. I kept all the training folders, and the code works great. I get e.g Identified (result: 99 - dist - 0.9768321992443404)

Feel free to ask questions if you have problems running the code, or getting bad results.

User avatar
idontcar3
Posts: 8
Joined: Wed Mar 09, 2016 2:54 am

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Thu Mar 24, 2016 8:28 am

i'm getting a

Code: Select all

Traceback (most recent call last):
  File "recon.py", line 64, in <module>
    distances.append((dist, y[i-1]))
IndexError: list index out of range
i'm running this on py 2.7, would that be the problem?

i got train_faces/s1-s10
and test/faces/2.pgm

EDIT:
i extracted all 40 folders and made a copy of a folder, to have a total of 410 items, then it went out ok.
i'll read this code carefully when i wake up.

nordnet
Posts: 6
Joined: Thu Mar 10, 2016 11:59 pm

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Thu Mar 24, 2016 12:07 pm

idontcar3 wrote:i'm getting a

Code: Select all

Traceback (most recent call last):
  File "recon.py", line 64, in <module>
    distances.append((dist, y[i-1]))
IndexError: list index out of range
i'm running this on py 2.7, would that be the problem?

i got train_faces/s1-s10
and test/faces/2.pgm

EDIT:
i extracted all 40 folders and made a copy of a folder, to have a total of 410 items, then it went out ok.
i'll read this code carefully when i wake up.

It's important that your folder structure and folder names match those in the code, so you either need to change the part in the code to reflect what you name your own folders, or just change the name of the folders on your computer (or raspberry).

This line needs to match:

Code: Select all

# load test faces (usually one), located in folder test_faces
test_faces = glob.glob('test_faces/*')

Sajin
Posts: 1
Joined: Wed Mar 30, 2016 2:27 pm

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Wed Mar 30, 2016 2:29 pm

when i run the code , I am getting

Code: Select all

Traceback (most recent call last):
  File "faces.py", line 58, in <module>
    for j, ref_pca in enumerate(pca.transform(X)):
  File "/usr/local/lib/python2.7/dist-packages/sklearn/decomposition/pca.py", line 635, in transform
    X = check_array(X)
  File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 407, in check_array
    context))
ValueError: Found array with 0 sample(s) (shape=(0, 10304)) while a minimum of 1 is required.
Please help me

nordnet
Posts: 6
Joined: Thu Mar 10, 2016 11:59 pm

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Thu Mar 31, 2016 11:02 am

Sajin wrote:when i run the code , I am getting

Code: Select all

Traceback (most recent call last):
  File "faces.py", line 58, in <module>
    for j, ref_pca in enumerate(pca.transform(X)):
  File "/usr/local/lib/python2.7/dist-packages/sklearn/decomposition/pca.py", line 635, in transform
    X = check_array(X)
  File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 407, in check_array
    context))
ValueError: Found array with 0 sample(s) (shape=(0, 10304)) while a minimum of 1 is required.
Please help me
Hi! It's important that your folder structure is set up as the script intends it to be. Have you provided some test images of yourself in a folder?

Do exactly as this says:
You of course also need to set up a folder for the face you want to detect (e.g. yourself). Call that s99 and put 10 pictures (format irrelevant, I used JPG) of yourself into it (92x112, grayscale, different light settings etc).
You also need to have a picture of yourself (or whomever you are testing against) in a separate folder called test_faces.

I'm not sure if it was of any help, but don't hesitate if you still can't get it to work.

viveksonone
Posts: 7
Joined: Sat Mar 12, 2016 9:01 am

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Sun Apr 03, 2016 2:58 pm

File "vivek.py", line 62, in <module>
distances.append((dist, y))
IndexError: list index out of range

what should i do?

nordnet
Posts: 6
Joined: Thu Mar 10, 2016 11:59 pm

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Mon Apr 04, 2016 1:41 am

Please have a look at my previous answers which deals with exactly the same problem. I'm glad to help but you need to provide more context. What have you tried, how does your folder structure look like, are you getting any other error codes etc.

ShahidOmari
Posts: 2
Joined: Mon Apr 04, 2016 8:12 am

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Mon Apr 04, 2016 8:28 am

Hi! I tried the code on my own data set and it worked just fine. But the problem is when I try to use an image of a person not in the data set, the code match the image with one of the persons in the train set but I know it is wrong because the image I used to test is not of one of the people in the train set.
So if anyone can help me how to make the code print a message if the person in the test is not one of the persons in the train set.
I hope I was as clear as possible and thanks in advance :)

viveksonone
Posts: 7
Joined: Sat Mar 12, 2016 9:01 am

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Sat Apr 09, 2016 6:21 am

pi@raspberrypi:~/attendace $ python vivek.py
Traceback (most recent call last):
File "vivek.py", line 62, in <module>
distances.append((dist, y))
IndexError: list index out of range
pi@raspberrypi:~/attendace $

error list index out of range

/attandence/train _faces contaion /s1....upto s10 folders
/attandence/tast_faces contain 10 images from sets10 or my own 10 photos


from sklearn.decomposition import RandomizedPCA
import numpy as np
import glob
import cv2
import math
import os.path
import string

#function to get ID from filename
def ID_from_filename(filename):
part = string.split(filename, '/')
return part[1].replace("s", "")

#function to convert image to right format
def prepare_image(filename):
img_color = cv2.imread(filename)
img_gray = cv2.cvtColor(img_color,cv2.COLOR_BGR2GRAY)
img_gray = cv2.equalizeHist(img_gray)
return img_gray.flat

IMG_RES = 92 * 112 # img resolution
NUM_EIGENFACES = 10 # images per train person
NUM_TRAINIMAGES = 110 # total images in training set

#loading training set from folder train_faces
folders = glob.glob('train_faces/*')

# Create an array with flattened images X
# and an array with ID of the people on each image y
X = np.zeros([NUM_TRAINIMAGES, IMG_RES], dtype='int8')
y = []

# Populate training array with flattened imags from subfolders of train_faces and names
c = 0
for x, folder in enumerate(folders):
train_faces = glob.glob(folder + '/*')
for i, face in enumerate(train_faces):
X[c,:] = prepare_image(face)
y.append(ID_from_filename(face))
c = c + 1

# perform principal component analysis on the images
pca = RandomizedPCA(n_components=NUM_EIGENFACES, whiten=True).fit(X)
X_pca = pca.transform(X)

# load test faces (usually one), located in folder test_faces
test_faces = glob.glob('test_faces/*')


# Create an array with flattened images X
X = np.zeros([len(test_faces), IMG_RES], dtype='int8')

# Populate test array with flattened imags from subfolders of train_faces
for i, face in enumerate(test_faces):
X[i,:] = prepare_image(face)

# run through test images (usually one)
for j, ref_pca in enumerate(pca.transform(X)):
distances = []
# Calculate euclidian distance from test image to each of the known images and save distances
for i, test_pca in enumerate(X_pca):
dist = math.sqrt(sum([diff**2 for diff in (ref_pca - test_pca)]))
distances.append((dist, y))

found_ID = min(distances)[1]

print("Identified (result: "+ str(found_ID) +" - dist - " + str(min(distances)[0]) + ")")

nordnet
Posts: 6
Joined: Thu Mar 10, 2016 11:59 pm

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Sat Apr 09, 2016 12:33 pm

Your folder setup is wrong though viveksonone. I'm not sure if it's a typo just here, but the folder needs to be renamed from tast_faces to test_faces. Also the logic is that the script checks the faces in train_faces against the face in test_faces, if you don't have a folder with faces of the image you want to test against in train_faces, it will not find it. In the test_faces folder you don't really need 10 images of yourself, one is usually enough to test if the script works. so:

train_faces/s1...s10 + a folder with images of faces you want to test against (called e.g s99) containing 10 images, in the correct format (i.e .pgm) and correct resolution (i.e 92 * 112) and of your faces (the algorithm will have problems if you include too much of your t-shirt e.g)

and a folder called test_faces with at least on image of the face you want to test (I'm not sure if this needs to be .pgm but I see that I used it for mine)

Hope this helps.

nordnet
Posts: 6
Joined: Thu Mar 10, 2016 11:59 pm

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Sat Apr 09, 2016 12:40 pm

ShahidOmari wrote:Hi! I tried the code on my own data set and it worked just fine. But the problem is when I try to use an image of a person not in the data set, the code match the image with one of the persons in the train set but I know it is wrong because the image I used to test is not of one of the people in the train set.
So if anyone can help me how to make the code print a message if the person in the test is not one of the persons in the train set.
I hope I was as clear as possible and thanks in advance :)
Hi, good that it worked! What you could do is have it print something if it doesn't match the folder you have put your test images in (e.g doesn't match images in folder s99):

You would have to change these lines:

found_ID = min(distances)[1]
print("Identified (result: "+ str(found_ID) +" - dist - " + str(min(distances)[0]) + ")")

if found_ID == "99":
print("Hello Nordnet")

else: print("That's not you Nordnet…")

That's on example how it can be done.

ShahidOmari
Posts: 2
Joined: Mon Apr 04, 2016 8:12 am

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Tue Apr 26, 2016 6:13 pm

Hi. Thank you for the answer.
I did what you said and it worked for the people saved in the database (e.g. s1, s2, ...).
The problem is with people from outside the database, the code seems to match the image with the closest person in the database and comes out as identified.
I tried to look in the code to find where the problem happens, but I couldn't find it. I'm not that good with matrix in coding, so I couldn'nt understand the code well.
please if you know how I can deal with images from outside the database and mark them as not recognized, it would really help.
sorry for the trouble again,I hope I was clear and the language is good.
thanks in advance, any help is really apreciated as succeeding in solving this problem is very important to me.

owen_a
Posts: 58
Joined: Sun Sep 09, 2012 3:31 pm
Location: United Kingdom

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Sat Jun 25, 2016 3:52 pm

I think a really good example would be to modify this code to use the PiCamera so it does the recognition on the fly instead of taking an image from the local directory and checking it.

srinipi
Posts: 1
Joined: Thu Jun 30, 2016 5:56 pm

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Fri Jul 01, 2016 7:02 pm

Hi,

I tried to run this example on my Raspberry pi
my folder structure is this way :
~/opencv/find_faces.py --> copy pasted as it is from this tutorial
~/opencv/test_faces --> I have copied s1 folders 10.pgm and renamed it to 10 copies of .pgm
~/opencv/train_faces --- > Here i have copied the folders s1 to s40

I end up getting following error:

pi@raspberrypi:~/opencv $ python find_faces.py
Traceback (most recent call last):
File "find_faces.py", line 38, in <module>
X[c,:] = prepare_image(face)
ValueError: could not broadcast input array from shape (120000) into shape (10304)
pi@raspberrypi:~/opencv $ ls


How do i fix it? any help is appreciated

Thanks in Advance
~Srinivas

enliteneer
Posts: 1
Joined: Tue Aug 02, 2016 4:30 am

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Tue Aug 02, 2016 5:02 am

Hi Nordnet,

I have a 640x480 usb webcam on the beaglebone black running OpenCV 2.4.9 on Jessie 8.4.. OpenCV builds fine (C++ and python binding).

Any tips on face recognition using a BBB?

For example, can I use my own database of headshots for training the face recognition algorithm? What kind of performance should I expect? Would a higher resolution camera result in better accuracy, at the expense of frames per second?

What's the difference in algorithms - Neural Networks, EigenFaces, FisherFaces, etc? When to use each?

Do you know of any video tutorial that goes over Face Recognition (not Detection)?
For example, this video looks good but has no sound: https://www.youtube.com/watch?v=yJvuZc1Ap3I

Other resources seem to use cv1
https://www.youtube.com/watch?v=yFSPnu6_TZY
https://github.com/benhoff/face-recognizer-gui

Lars_vS
Posts: 1
Joined: Fri Feb 10, 2017 8:15 am

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Fri Feb 10, 2017 8:20 am

Hello,

Thanks for the great tutorial! I got everything working except for one last thing (a kind of critical thing):

When I scan a new picture of me, it returns a distance of 0 with the wrong match. When I upload one of the exact pictures in my S99 folder (pictures of me for comparison) it returns a distance of 4+ with the wrong match.

I'm working on a Raspberry Pi 3 Model B and got it less than a week ago (so it runs the newest software).

Anyone any idea what I did wrong?

Praveen kumar101
Posts: 1
Joined: Tue Feb 13, 2018 2:52 pm

Re: [Tutorial] Simple Face Recognition (OpenCV, Python)

Wed Feb 14, 2018 5:15 pm

can u help me this one. i am doing a project on this...
can anyone guide me to create a database on raspberry pi....

Return to “Graphics, sound and multimedia”