Dylanli
Posts: 28
Joined: Mon May 24, 2021 6:03 am

Pedestrian recognition based on Raspberry Pi

Thu Jul 01, 2021 9:48 am

First, install the c++ version of OpenCV on the Raspberry Pi, use the CSI camera on the board to capture images, and then use HOG features and SVM to detect pedestrians in them. Not sure if the raspberry pi has enough arithmetic power?

I would like to write this program in.

Code: Select all

#include <iostream>
#include <fstream>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/objdetect/objdetect.hpp>
#include <opencv2/ml/ml.hpp>
using namespace std;
using namespace cv;
#define PosSamNO 1114 //number of positive samples
#define NegSamNO 12180 //number of negative samples
#define TRAIN true//whether to train, true means retrain, false means read the SVM model from the xml file
#define CENTRAL_CROP false //true: when training, the 96*160 INRIA positive sample images are cropped out of the middle 64*128 size human body
//HardExample: the number of negative samples. If HardExampleNO is greater than 0, it means that after processing the initial negative sample set, continue to process the HardExample negative sample set.
// Must be set to 0 when HardExample is not used, because this value is used when initializing the dimensionality of the eigenvector matrix and the eigenclass matrix.
#define HardExampleNO 433
 
//Inherited from the CvSVM class, because when generating the detection subparameters used in setSVMDetector(), the decision_func parameter of the trained SVM is used.
// But by looking at the CvSVM source code, we can see that the decision_func parameter is a protected type variable, which cannot be accessed directly, but can only be accessed through a function after inheritance
class MySVM : public CvSVM  
{  
public:  
    // Get the alpha array in the decision function of the SVM  
    double * get_alpha_vector()  
    {  
        return this->decision_func->alpha;  
    }  
    //get the rho parameter of the SVM's decision function, i.e., the offset  
    float get_rho()  
    {  
        return this->decision_func->rho;  
    }  
}; 
 

int main()
{
    // detect window(64,128),block size(16,16),block step(8,8),cell size(8,8),histogram bin number 9
    HOGDescriptor hog(Size(64,128),Size(16,16),Size(8,8),Size(8,8),9);//HOG detector, used to calculate the HOG descriptor
    int DescriptorDim;//the number of dimensions of the HOG descriptor, determined by the image size, detection window size, block size, and the number of histogram bins in the cell
    MySVM svm;//SVM classifier
    //If TRAIN is true, retrain the classifier
    if(TRAIN)
    {
        string ImgName;//Image name (absolute path)
        ifstream finPos("d:/hogmit/pos/pos.txt");//List of filenames of positive sample images
        //ifstream finPos("PersonFromVOC2012List.txt");//filename list of positive sample images
        ifstream finNeg("d:/hogmit/neg/neg.txt");//filename list of negative sample images
        Mat sampleFeatureMat;//Matrix of all training sample feature vectors, the number of rows is equal to the number of all samples, the number of columns is equal to the number of HOG descriptor dimensions    
        Mat sampleLabelMat;//the category vector of training samples, the number of rows is equal to the number of all samples, the number of columns is equal to 1; 1 means someone, -1 means no one
 
        //read the positive sample images in turn to generate HOG descriptors
        for(int num=0; num<PosSamNO && getline(finPos,ImgName); num++)
        {
            cout<<"Processing:"<<ImgName<<endl;
            //ImgName = "D:\\DataSet\\PersonFromVOC2012\\" + ImgName;//add the pathname of the positive sample
            ImgName = "d:/hogmit/pos/" + ImgName;//plus the pathname of the positive sample
            Mat src = imread(ImgName);//read the image
            if(CENTRAL_CROP)
                src = src(Rect(16,16,64,128));//Crop the 96*160 INRIA positive sample image to 64*128, i.e. cut off 16 pixels from the top, bottom, left and right
            //resize(src,src,Size(64,128));
            vector<float> descriptors;//HOG describes the subvector
            hog.compute(src,descriptors,Size(8,8));//compute HOG descriptors, detect window move step(8,8)
            //
            cout<<"Descriptor dimension:"<<descriptors.size()<<endl;
            // initialize the eigenvector matrix and the category matrix when processing the first sample, because the eigenvector matrix can only be initialized if the dimensionality of the eigenvectors is known
            if( 0 == num )
            {
                DescriptorDim = descriptors.size();//the number of dimensions of the HOG descriptors
                //initialize a matrix of feature vectors of all training samples, with the number of rows equal to the number of all samples and the number of columns equal to the dimension of the HOG descriptor sampleFeatureMat
                sampleFeatureMat = Mat::zeros(PosSamNO+NegSamNO+HardExampleNO, DescriptorDim, CV_32FC1);
                //initialize the category vector of training samples, the number of rows is equal to the number of all samples, the number of columns is equal to 1; 1 means occupied, 0 means unoccupied
                sampleLabelMat = Mat::zeros(PosSamNO+NegSamNO+HardExampleNO, 1, CV_32FC1);
            }
            //Copy the computed HOG descriptors to the sampleFeatureMat
            for(int i=0; i<DescriptorDim; i++)
                sampleFeatureMat.at<float>(num,i) = descriptors[i];//the i-th element of the feature vector of the numth sample
            sampleLabelMat.at<float>(num,0) = 1;//positive sample category is 1, someone
        }
        //read negative sample images in turn to generate HOG descriptors
        for(int num=0; num<NegSamNO && getline(finNeg,ImgName); num++)
        {
            cout<<"Processing:"<<ImgName<<endl;
            ImgName = "d:/hogmit/neg/" + ImgName;//add the pathname of the negative sample
            Mat src = imread(ImgName);//read the image
            //resize(src,img,Size(64,128));
            vector<float> descriptors;//HOG description subvector
            hog.compute(src,descriptors,Size(8,8));//compute HOG descriptors, detect window move step(8,8)
            //cout<<"Descriptor dimension:"<<descriptors.size()<<endl;
            //Copy the computed HOG descriptors to the sampleFeatureMat of the sample feature matrix
            for(int i=0; i<DescriptorDim; i++)
                sampleFeatureMat.at<float>(num+PosSamNO,i) = descriptors[i];//the i-th element of the feature vector of the PosSamNO+numth sample
            sampleLabelMat.at<float>(num+PosSamNO,0) = -1;//Negative sample category is -1, no one
        }
        //process HardExample negative samples
        if(HardExampleNO > 0)
        {
            ifstream finHardExample("d:/hogmit/hard/hard.txt");//list of file names of HardExample negative samples
            //read HardExample negative sample images in turn to generate HOG descriptors
            for(int num=0; num<HardExampleNO && getline(finHardExample,ImgName); num++)
            {
                cout<<"Processing:"<<ImgName<<endl;
                ImgName = "d:/hogmit/hard/" + ImgName;//add the pathname of the HardExample negative sample
                Mat src = imread(ImgName);//read the image
                //resize(src,img,Size(64,128));
                vector<float> descriptors;//HOG description subvector
                hog.compute(src,descriptors,Size(8,8));//compute HOG descriptors, detect window move step(8,8)
                //cout<<"Descriptor dimension:"<<descriptors.size()<<endl;
                //Copy the computed HOG descriptors to the sampleFeatureMat of the sample feature matrix
                for(int i=0; i<DescriptorDim; i++)
                    sampleFeatureMat.at<float>(num+PosSamNO+NegSamNO,i) = descriptors[i];//the i-th element of the feature vector of the PosSamNO+numth sample
                sampleLabelMat.at<float>(num+PosSamNO+NegSamNO,0) = -1;//Negative sample category is -1, no one
            }
        }
        //output the HOG feature vector matrix of the samples to the file
        ofstream fout("d:/xlw/SampleFeatureMat.txt");
        for(int i=0; i<PosSamNO+NegSamNO; i++)
        {
            fout<<i<<endl;
            for(int j=0; j<DescriptorDim; j++)
            { fout<<sampleFeatureMat.at<float>(i,j)<<" ";
            }
            fout<<endl;
        }
        // train the SVM classifier
        // iteration termination condition, stop iteration when 1000 iterations or error is less than FLT_EPSILON
        CvTermCriteria criteria = cvTermCriteria(CV_TERMCRIT_ITER+CV_TERMCRIT_EPS, 1000, FLT_EPSILON);
        //SVM parameters: SVM type is C_SVC; linear kernel function; relaxation factor C=0.01
        CvSVMParams param(CvSVM::C_SVC, CvSVM::LINEAR, 0, 1, 0, 0, 0.01, 0, 0, 0, 0, criteria);
        cout<<"Start training SVM classifier"<<endl;
        double time0=static_cast<double>(getTickCount());
        svm.train(sampleFeatureMat,sampleLabelMat, Mat(), Mat(), param);//training classifier
        time0=((double)getTickCount()-time0)/getTickFrequency();
        cout<<"Training completed"<<endl;
        cout<<"Time spent in training:"<<time0<<endl;
        svm.save("d:/xlw/SVM_HOG_mit_inria(1114pos+12180neg+433hard).xml");//save the trained SVM model as an xml file
    }
    else //if TRAIN is false, read the trained classifier from the XML file
    {
        cout<<"fail train"<<endl;
        //svm.load("SVM_HOG_2400PosINRIA_12000Neg_HardExample(false less misses more).xml");//read the trained SVM model from the XML file
        svm.load("d:/LBP/SVM_Model.xml");
    }/*************************************************************************************************
    The XML file obtained after the training of the linear SVM has an array called support vector, another array called alpha, and a floating point number called rho;
    Multiplying the alpha matrix with the support vector, note that alpha*supportVector, will result in a column vector. After that, add an element rho at the end of the column vector.
    In this way, you get a classifier that directly replaces the default classifier for pedestrian detection in opencv (cv::HOGDescriptor::setSVMDetector()).
    You can then use the classifier trained with your training samples for pedestrian detection.
    ***************************************************************************************************/
    DescriptorDim = svm.get_var_count();// the dimensionality of the feature vector, i.e. the dimensionality of the HOG descriptor
    int supportVectorNum = svm.get_support_vector_count();// the number of support vectors
    cout<<"Number of support vectors:"<<supportVectorNum<<endl;
    Mat alphaMat = Mat::zeros(1, supportVectorNum, CV_32FC1);//alpha vector, length is equal to the number of support vectors
    Mat supportVectorMat = Mat::zeros(supportVectorNum, DescriptorDim, CV_32FC1);//support vector matrix
    Mat resultMat = Mat::zeros(1, DescriptorDim, CV_32FC1);//the result of multiplying the alpha vector by the support vector matrix
    //Copy the support vector data into the supportVectorMat matrix
    for(int i=0; i<supportVectorNum; i++)
    {
        const float * pSVData = svm.get_support_vector(i);//return the data pointer of the i-th support vector
        for(int j=0; j<DescriptorDim; j++)
        {
            //cout<<pData[j]<<" ";
            supportVectorMat.at<float>(i,j) = pSVData[j];
        }
    }
    // copy the data of the alpha vector into the alphaMat
    double * pAlphaData = svm.get_alpha_vector();//return the alpha vector from the SVM's decision function
    for(int i=0; i<supportVectorNum; i++)
    {
        alphaMat.at<float>(0,i) = pAlphaData[i];
    }
    //compute -(alphaMat * supportVectorMat), put the result in resultMat
    //gemm(alphaMat, supportVectorMat, -1, 0, 1, resultMat);//don't know why the negative sign is added?
    resultMat = -1 * alphaMat * supportVectorMat;
    //Get the final setSVMDetector(const vector<float>& detector) parameter for the available detectors
    vector<float> myDetector;
    // copy the data from resultMat to the array myDetector
    for(int i=0; i<DescriptorDim; i++)
    {
        myDetector.push_back(resultMat.at<float>(0,i));
    }
    // Finally add the offset rho to get the detector
    myDetector.push_back(svm.get_rho());
    cout<<"Number of detector dimensions:"<<myDetector.size()<<endl;
    // Set the detector of the HOGDescriptor
    HOGDescriptor myHOG;
    myHOG.setSVMDetector(myDetector);
    //myHOG.setSVMDetector(HOGDescriptor::getDefaultPeopleDetector());
    //Save the detector parameters to a file
    ofstream fout("d:/xlw/HOGDetectorForOpenCV.txt");
    for(int i=0; i<myDetector.size(); i++)
    {
        fout<<myDetector[i]<<endl;
    }
 
    /************** read in image for HOG pedestrian detection ******************/
    Mat src = imread("d:/timg.png");
    //Mat src = imread("2007_000423.jpg");
    //Mat src = imread("1.png");
    vector<Rect> found, found_filtered;//array of rectangular boxes
    cout<<"Performing multi-scale HOG body detection"<<endl;
    myHOG.detectMultiScale(src, found, 0, Size(8,8), Size(32,32), 1.05, 2);//perform multi-scale pedestrian detection on the image
     //src is the input image to be detected; found is the list of detected target regions; parameter 3 is the threshold value calculated internally for the pedestrian target, which is the distance from the detected feature to the SVM classification hyperplane;
    // Parameter 4 is the distance of each move of the sliding window. It must be an integer multiple of the block movement; parameter 5 is the size of the image expansion; parameter 6 is the scale factor, i.e., the proportion of the test image that increases with each size scaling.
    // Parameter 7 is the group threshold, i.e. the correction factor. When a target is detected by more than one window, this parameter plays a regulating role at this time, and a value of 0 means no regulating role.
    cout<<"Number of rectangular boxes found:"<<found.size()<<endl;
    //find all the rectangles that are not nested and put them into found_filtered, if they are nested, then take the largest rectangle outside and put it into found_filtered
    for(int i=0; i < found.size(); i++)
    {
        Rect r = found[i];
        int j=0;
        for(; j < found.size(); j++)
            if(j ! = i && (r & found[j]) == r)
                break;
        if( j == found.size())
            found_filtered.push_back(r);
    }
    // draw rectangular box, because hog detects a rectangular box than the actual human body box is slightly larger, so here need to do some adjustments
    for(int i=0; i<found_filtered.size(); i++)
    {
        Rect r = found_filtered[i];
        r.x += cvRound(r.width*0.1);
        r.width = cvRound(r.width*0.8);
        r.y += cvRound(r.height*0.07);
        r.height = cvRound(r.height*0.8);
        rectangle(src, r.tl(), r.br(), Scalar(0,255,0), 3);
    }
    imwrite("d:/SVM/ImgProcessed3.jpg",src);
    namedWindow("src",0);
    imshow("src",src);
    waitKey();//Note: imshow must be followed by waitKey, otherwise the image cannot be displayed
 
    /****************** reads in a single 64*128 test image and classifies its HOG descriptors *********************/
    //// reads a test image (64*128 size) and calculates its HOG descriptors
    //Mat testImg = imread("person014142.jpg");
    //Mat testImg = imread("noperson000026.jpg");
    //vector<float> descriptor;
    //hog.compute(testImg,descriptor,Size(8,8));//compute HOG descriptor, detect window move step(8,8)
    //Mat testFeatureMat = Mat::zeros(1,3780,CV_32FC1);//Test the feature vector matrix of the sample
    //Copy the computed HOG descriptors into the testFeatureMat matrix
    // for(int i=0; i<descriptor.size(); i++)
    // testFeatureMat.at<float>(0,i) = descriptor[i];
    // classify the feature vector of the test image with the trained SVM classifier
    //int result = svm.predict(testFeatureMat);//returns the class label
    //cout<<"Classification result:"<<result<endl;
 
    system("pause");
}

Dylanli
Posts: 28
Joined: Mon May 24, 2021 6:03 am

Re: Pedestrian recognition based on Raspberry Pi

Thu Jul 01, 2021 9:50 am

Maybe even show captured pedestrians on the display? I don't know if that can be done.

[moderator: after being flagged I removed a link which I consider a n unnecessary and obfuscated spamlink, and locked this post to make it impossible to edit it back, if you do post it elsewhere I will consider you to be a spammer, and will ban you!]

User avatar
B.Goode
Posts: 13760
Joined: Mon Sep 01, 2014 4:03 pm
Location: UK

Re: Pedestrian recognition based on Raspberry Pi

Thu Jul 01, 2021 9:52 am

Dylanli wrote:
Thu Jul 01, 2021 9:48 am
First, install the c++ version of OpenCV on the Raspberry Pi, use the CSI camera on the board to capture images, and then use HOG features and SVM to detect pedestrians in them. Not sure if the raspberry pi has enough arithmetic power?

I would like to write this program in.

Code: Select all

# Code suppressed from this response
}


You already have an RPi4 board. Why not simply try it and learn from your own experience?

dbrion06
Posts: 777
Joined: Tue May 28, 2019 11:57 am

Re: Pedestrian recognition based on Raspberry Pi

Thu Jul 01, 2021 10:03 am

Not sure if the raspberry pi has enough arithmetic power?
It has at least in detection... maybe you skip one frame out of 2...
for training: depends on your database size; as everything is in RAM, you might be limited... but xml files can be copied from a powerful PC... if you have
(detection and training are 2 different issues)
Do you hope it will work with grey level images/videos (can be much faster)

Dylanli
Posts: 28
Joined: Mon May 24, 2021 6:03 am

Re: Pedestrian recognition based on Raspberry Pi

Wed Jul 07, 2021 2:18 am

Thank you for your replies, I will consult more information and practice myself!

User avatar
Gavinmc42
Posts: 6621
Joined: Wed Aug 28, 2013 3:31 am

Re: Pedestrian recognition based on Raspberry Pi

Wed Jul 07, 2021 6:35 am

Isn't there laws about putting "capture" pedestrians on display?

The Pi4 should have enough power, especially with big heatsink and overclocked?

Guess it depends on your software, Yolo or other Binary NNs work faster.
Reducing resolution down to 300x300 or so helps too.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

dbrion06
Posts: 777
Joined: Tue May 28, 2019 11:57 am

Re: Pedestrian recognition based on Raspberry Pi

Wed Jul 07, 2021 7:59 am

yolo works *very* slowly in detection on a Pi (it detects a lot of things, not "only" pedestrians). In training, who dared?

Before DNN existed, car industry spent a lot of money to detect pedestrians, trees (vertically shaped; can harm cars) with the latest and greatest embedded cameras and CPUs -in the 1990s-2000s- . This led in the 2000s to HOG+SVM: are reasonably fast (at least in detction; for training, I do not know). seems OP is using HOG (detects shapes without DNNs) ; training HOG is another matter (his/her post is ambiguous).

BTW: once an xml file (can be elaborated on a very powerful machine) is loaded, it becomes a binary (specific, with right indianness, the RPi, say) and one just has to capture pictures, without leaving program....

Return to “C/C++”