Raspberry Pi - picamera Python openCV2 motion tracking

[pageauc]

motion-track.py Demo
Raspberry Pi - picamera python openCV2 real time motion tracking
motion-track.py is a raspberry pi camera python openCV2 (computer vision) motion tracking demonstration program.
written by Claude Pageau
code available at https://github.com/pageauc/motion-track

Here is YouTube video showing the demo code running on a Raspberry Pi B2
http://youtu.be/09JS7twPBsQ

It will detect motion in the field of view and use openCV2 to compare images and calculate a threshold image and related contours. It will determine the largest contour and return it's x,y coordinates and contour area. Area may be useful for possible crude robot ranging eg very large area is nearby moving object. I will implement similar code in a RPI robotics project, but thought the demo code would be useful for other users as a starting point or as part of an existing project.

I did quite a bit of searching on the internet, github, etc, but could not find a similar python picamera implementation that returns x,y coordinates of the most dominate moving object in the frame although some came close.

Some of this code is based on a YouTube tutorial by Kyle Hounslow using C here -
https://www.youtube.com/watch?v=X6rPdRZzgjg

Prerequisites

Requires a Raspberry Pi computer running an up-to-date raspbian distro and a RPI camera module installed and configured. The dependencies below may be required depending on your previous installs.

Code: Select all

cd ~
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python-opencv python-picamera
mkdir motion-track
cd motion-track    
wget https://raw.github.com/pageauc/motion-track/master/motion-track.py

start up a desktop GUI session and run the code from IDLE or open a terminal console and run

Code: Select all

python ./motion-track.py

I have set debug=True but you should set window_on=True to see the opencv status windows displayed on your local RPI monitor. It will display camera motion images window with a circle marking x,y motion coordinates, as well as the threshold images window. Set window_on=False if you are running from an SSH session otherwise you will get an error.

Good Luck Claude ...

[pageauc]

Cleaned up the motion-track.py code in the repository https://github.com/pageauc/motion-track
- Added window_on variable to turn opencv windows on or off. This allows debug messages to be displayed in an SSH session without the need to run from a GUI desktop.
- added variable to control the size of circle
- set lower area of contours to avoid very small contours and possibly a false positive
- Also cleaned up a bit of logic.

I tested program with low light conditions and it works surprisingly well. You can also see at the start of the video where it caught our black and tan chihuahua investigate who I was talking to. I noticed the target circle near my feet and then realized she was in the frame but hard to see due to her colour.
Let me know if the code is useful
Regards Claude ...

[pageauc]

change the camera settings variables in the program code using nano

Code: Select all

CAMERA_HFLIP = False
CAMERA_VFLIP = False

This will flip the camera image. Camera position differences is a common problem. That is why I added these variables. I have camera's mounted in both positions so the variables come in handy.

[pageauc]

I read that there are still some library compatibility issues with python3 so this might be an issue. I will try some testing with python3. I try to write code for python 3 print statements or convert using

Code: Select all

2to3 -w filename.py

where filename.py is a file wildcard format or the name of python2 program to convert. The python3 print statements use brackets that are backward compatible with newer python2.

OpenCV3 is still Alpha and may not be stable and not sure of any syntax differences. Would be interested to know what FPS processing is achieved with CV3 in window_on=True and window_on=False settings on B2 and B+.

Thanks

[pageauc]

I am trying to compile opencv3 on RPI B2 but having a few issues. What steps did you use to compile opencv3 on RPI?
I am using instructions here.
http://www.linuxcircle.com/2015/05/18/o ... erry-pi-2/
but getting the following error when installing core dependencies.

Code: Select all

pi@rpi-b2-1 ~ $ sudo apt-get install cmake python3-dev python3.2-dev python3-numpy gcc gcc-c++ build-essential cmake-curses-gui
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package gcc-c+
E: Couldn't find any package by regex 'gcc-c+'

Did a package search but no joy

Code: Select all

apt-cache search gcc-c

if I search for gcc, c+, Etc they all return a long list of package names
Any help would be appreciated.
Claude ....

[pageauc]

How to Install and Compile OpenCV3 on RPI and test using motion tracking

These are the detailed steps I used to successfully compile opencv3 on my RPI 2. After all this my testing indicated no speed improvement as far as I could see, but your mileage may vary depending on what opencv code you use. In my case no improvement noticed. Python3 is part of the default RPI build so just did normal update prior to compiling opencv3

step 1 - update raspbian

Code: Select all

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade

Note that dist-upgrade may not be required but I did it anyway since I have enough room on my SD card

step2 - Install/Update required build tools and dependencies then compile opencv3

Followed instructions per link below, but changed the first command to delete the gcc-c++ entry
http://www.linuxcircle.com/2015/05/18/o ... erry-pi-2/

These are the commands I used to compile opencv3

Code: Select all

cd ~
sudo apt-get install cmake python3-dev python3.2-dev python3-numpy gcc  build-essential cmake-curses-gui
sudo apt-get -y install pkg-config libpng12-0 libpng12-dev libpng++-dev libpng3 libpnglite-dev zlib1g-dbg zlib1g zlib1g-dev pngtools libtiff5-dev libtiff5 libtiffxx0c2 libtiff-tools libeigen3-dev
sudo apt-get -y install libjpeg8 libjpeg8-dev libjpeg8-dbg libjpeg-progs ffmpeg libavcodec-dev libavcodec53 libavformat53 libavformat-dev libgstreamer0.10-0-dbg libgstreamer0.10-0 libgstreamer0.10-dev libxine1-ffmpeg libxine-dev libxine1-bin libunicap2 libunicap2-dev swig libv4l-0 libv4l-dev libpython3.2 libgtk2.0-dev
git clone --branch 3.0.0-rc1 --depth 1 https://github.com/Itseez/opencv.git
cd opencv
mkdir release
cd release
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=$(python3 -c "import sys; print(sys.prefix)") -D PYTHON_EXECUTABLE=$(which python3) ..
make -j4
sudo make install

step 3 - Verify installation
Here is the test to verify I have both opencv2 and opencv3 under python3

Code: Select all

]pi@dawn-robot ~/motion-track $ python
>>> from cv2 import __version__
>>> __version__
'2.4.1'
>>> quit()
pi@dawn-robot ~/motion-track $ python3
Python 3.2.3 (default, Mar  1 2013, 11:53:50)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from cv2 import __version__
>>> __version__
'3.0.0-rc1'

step 4 - Test code
Conclusion Does not seem to improve speed over python2 per my testing below
I did get the same findCountours error as mentioned above so I created a separate motion3-track.py and changed the findContour command per below. That was the only change
from

Code: Select all

contours,hierarchy = cv2.findContours(thresholdimage,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)

to

Code: Select all

thresholdimage,contours,hierarchy = cv2.findContours(thresholdimage,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)

will post the slightly revised code on my github repo if you are interested
Tested motion-track.py using python2 and opencv2. Speed was approx 1.25 fps
Tested motion3-track.py using python3 and opencv3. Speed was about the same (no improvement as far as I can see)

Code: Select all

pi@dawn-robot ~/motion-track $ ./motion-track.py
Loading Please Wait ....
Initializing Camera ....
press ctrl-c to quit
Start Motion Tracking ....
total_Contours= 2  Motion at cx= 47 cy= 83   biggest_area: 35x 28=  639
Processing at 1.33 FPS last 10 frames
total_Contours= 4  Motion at cx= 97 cy= 40   biggest_area:192x238=30066
total_Contours= 7  Motion at cx= 49 cy=221   biggest_area: 97x 21= 1279
total_Contours= 2  Motion at cx=130 cy= 40   biggest_area:258x238=35887
total_Contours=12  Motion at cx= 65 cy= 19   biggest_area:128x110= 8229
total_Contours= 4  Motion at cx= 70 cy= 34   biggest_area:139x200=17323
total_Contours= 1  Motion at cx=109 cy= 35   biggest_area:217x209=24427
total_Contours= 2  Motion at cx=109 cy= 40   biggest_area:216x238=34207
total_Contours= 5  Motion at cx= 92 cy=147   biggest_area:182x110=12177
Processing at 1.25 FPS last 10 frames
total_Contours= 2  Motion at cx= 43 cy= 92   biggest_area: 14x 14=  129
total_Contours= 1  Motion at cx= 50 cy= 87   biggest_area: 26x 20=  223
total_Contours= 1  Motion at cx= 48 cy= 93   biggest_area: 23x 15=  190
total_Contours= 1  Motion at cx= 51 cy= 93   biggest_area: 28x 14=  223
total_Contours= 1  Motion at cx= 52 cy= 90   biggest_area: 29x 19=  341
total_Contours= 1  Motion at cx= 53 cy= 88   biggest_area: 27x 22=  412
total_Contours= 1  Motion at cx= 49 cy= 88   biggest_area: 29x 21=  364
total_Contours= 2  Motion at cx= 43 cy= 87   biggest_area: 12x 13=   83

pi@dawn-robot ~/motion-track $ ./motion3-track.py
Loading Please Wait ....
motion3-track.py Initializing Camera ....
press ctrl-c to quit
Start Motion Tracking ....
Processing at 1.34 FPS last 10 frames
total_Contours= 7  Motion at cx= 72 cy= 34   biggest_area:142x200=14533
total_Contours= 8  Motion at cx=132 cy= 35   biggest_area:262x204=27061
total_Contours=13  Motion at cx=132 cy= 60   biggest_area:262x175=18085
total_Contours= 9  Motion at cx=129 cy= 68   biggest_area:256x165=16880
total_Contours= 9  Motion at cx=129 cy= 35   biggest_area:257x206=22325
total_Contours= 2  Motion at cx=107 cy= 34   biggest_area:213x202=19147
total_Contours= 3  Motion at cx=107 cy= 34   biggest_area:213x202=22261
total_Contours= 6  Motion at cx=133 cy= 85   biggest_area:264x163=25308
total_Contours= 6  Motion at cx=133 cy= 85   biggest_area:264x163=25283
Processing at 1.25 FPS last 10 frames
total_Contours= 9  Motion at cx=133 cy=111   biggest_area:265x146=20328
total_Contours= 9  Motion at cx=133 cy=111   biggest_area:265x146=20243
total_Contours= 7  Motion at cx=133 cy=110   biggest_area:264x148=19651
total_Contours=15  Motion at cx=133 cy=110   biggest_area:264x148=19159
total_Contours= 1  Motion at cx= 59 cy= 90   biggest_area:  9x 12=   64

[pageauc]

motion_hotspot_game.py

I added a demo motion tracking game called hotspot.
https://github.com/pageauc/motion-track ... ot-game.py
It is pretty basic and could be a possible starting point for a better or different game. The game tracks motion (fingers work good) and you try to put center of motion inside a hotspot square in the middle of the screen or follow directions on the console. If you move your fingers a bit you can keep the motion going and if you are inside the hotspot for a period of time you will get a success message and a score. The hotspot then shrinks and you try again. Just a fun thing I was playing around with but thought others might be interested. The game plays in SSH session or in a GUI desktop window if the variable window_on=True.

Other games like Red Light, Green Light are also possible. Might give that a try. Might be a picademy idea for students.

Let me know what you think and if you design something or mod this motion game.

Regards Claude ..

[pageauc]

Added bigwindow_on variable to allow resizing the GUI desktop motion window. This makes it larger to allow better viewing on large displays. The window can be resized to anything but I have set them to multiples of the camera image width and height. The game is not very sophisticated but then again it is only a demo proof of concept.

Comments are welcome.

[pageauc]

Version 0.7 release of Faster FPS Motion Tracking Demo available
I have updated the motion_track.py to use picamera video port frame capture. This greatly increases the framerate of the loop. Note this is a demo program that you can use for other purposes. It can be broken down into functions that return x,y coordinates of movement. I use this method so a robot can track/find moving objects (largest contour) that are in the camera frame and take appropriate actions like pan/tilt, turn or move, look for face, etc. Tracking co ordinate information will need to be smoothed due to multiple moving objects, lighting, etc.
See the github repo for details https://github.com/pageauc/motion-track
or YouTube demo at https://youtu.be/09JS7twPBsQ

Also see Speed Track for a real life application at https://github.com/pageauc/motion-track ... peed-track
with YouTube demo at https://youtu.be/eRi50BbJUro
Regards Claude ...