Steve's Tech Talk

Face Detection with dotImage and OpenCV

We had a question in our forums about face detection via OpenCV.  I had looked at OpenCV a few weeks ago and had a small amount of time to burn, so I decided that this would be a fun project to tackle.  The API for OpenCV is sane and their internal image format appears to be similar to what we use in dotImage, so getting AtalaImage objects to play nicely with OpenCV looked straight forward.

Wrapping OpenCV is not novel – several people have done OpenCV wrappers for .NET.  I thought it would be nice to have a springboard for playing.

OpenCV is delivered via dlls on Windows.  I could’ve chosen to use P/Invoke to get at the API functions I needed and to make the appropriate struct wrappers, but I thought that creating the wrapper in C++/CLI made more sense and was less work – the interop is built into C++/CLI.  I think the resulting code is easier to follow.  I also chose to use C++/CLI over managed C++ the former feels better to write.  The syntax is much cleaner.

Then I created a C# WinForms application to test out the C++/CLI assembly.  I used our own ImageViewer control for display.

I tried to keep things simple.  I chose to create a single class to act as a face detector.  It would be constructed with the Haar Classifier data and provide one method for finding faces:

// OpenCVFaceDetection.h
// class definition for a face detector
#pragma once
using namespace System;
using namespace System::Drawing;
using namespace System::Collections::Generic;
using namespace Atalasoft::Imaging;
namespace OpenCVFaceDetection {
    public ref class FaceDetector : public IDisposable
    {
    private:
        IntPtr _cascade;
    public:
        FaceDetector(String ^pathToCascade);
        ~FaceDetector();
        !FaceDetector();
        IList<Rectangle> ^Detect(AtalaImage ^image);
    };
}

I’m not entirely fond of using a path for the constructor.  I’d prefer to use a Stream object instead, but there is no facility in OpenCV for custom streaming.  To get custom streaming, I’d have to rewrite their loader entirely.  Not impossible, but also not in my constraints.  I chose to keep the cascade data hanging around for the lifetime of the object under the assumption that the same face detector object would be used again and again.

Removing error checking, the actual code is pleasantly brief:

IList<Rectangle> ^FaceDetector::Detect(AtalaImage ^image)
{
    AtalaImage ^grayImage;
    // force to 8-bit gray.
    if (image->PixelFormat != PixelFormat::Pixel8bppGrayscale) {
        grayImage = image->GetChangedPixelFormat(PixelFormat::Pixel8bppGrayscale);
    }
    else {
        grayImage = image;
    }
    CvMemStorage *storage = 0;
    IplImage cvImage;
    try {
        storage = ::cvCreateMemStorage();
        // convert to a cvImage - does NOT allocate or copy memory
        InitIplImage(grayImage, &cvImage);
        // get the cascade
        CvHaarClassifierCascade *cvCascade = static_cast<CvHaarClassifierCascade *>(_cascade.ToPointer());
        // get the faces
        CvSeq *faces = ::cvHaarDetectObjects(&cvImage, cvCascade, storage, 1.1, 2, CV_HAAR_DO_CANNY_PRUNING, cvSize(40, 40));
        // build up the .NET list
        List<Rectangle> ^rects = gcnew List<Rectangle>();
        for (int i=0; i < faces->total; i++) {
            CvRect *cvr = (CvRect *)cvGetSeqElem(faces, i);
            rects->Add(Rectangle(cvr->x, cvr->y, cvr->width, cvr->height));
        }
        return rects;
    }
    __finally {
        if (storage)
            ::cvReleaseMemStorage(&storage);
        if (grayImage != image)
            delete grayImage;
    }
}

It comes down to converting to source image to gray, creating an image struct from that, firing up the Haar detector, and finally converting rectangles over.  Error checking in the actual code is about 1/4 of the total code.

When hooked into the display application, this is what you get:

In FaceFront, you’ll notice that I scale the source image down.  I found with some playing that the object detector doesn’t really like large images.  It literally finds faces in the woodwork.

To use this, you will need to download and install OpenCV.  The code as is, links to their stub libraries and includes their headers.  You may need to modify the OpenCVFaceDetection project settings to point to your installation.  You will also need the xml training data.  I used haarcascade_frontalface_default.xml in FaceFront, but you can change that as you see fit.

Download and enjoy.