The first step in working towards Friendly Bot is to get some code working that can capture an image from a webcam and detect a face in it. I knew that OpenCV had facilities for doing facial recognition, but I was hoping to avoid some of the documentation pain that I have heard about from other people. I did a quick search for rubygems that wrapped opencv and found spyglass by André Medeiros.
Spyglass makes a serious attempt to simplify the OpenCV API and so far it looks very promising. There is even a great example of doing facial recognition that got me started almost immediately.
Before I could install spyglass I needed to install OpenCV.
$ brew install opencv
The OpenCV classifier needs an xml document that tells it what features to look for. Luckily a bunch of examples come installed with OpenCV.
$ ls /usr/local/Cellar/opencv/18.104.22.168/share/OpenCV/haarcascades/
haarcascade_eye.xml haarcascade_eye_tree_eyeglasses.xml haarcascade_frontalface_alt.xml haarcascade_frontalface_alt2.xml haarcascade_frontalface_alt_tree.xml haarcascade_frontalface_default.xml ...
Now I installed spyglass and made one slight adaptation to the example provided in the gem.
require 'bundler/setup' require 'spyglass' require './lib/face_picker' include Spyglass classifier = CascadeClassifier.new("./haarcascade_frontalface_default.xml") window = GUI::Window.new "Video" cap = VideoCapture.new 0 frame = Image.new loop do cap >> frame rects = classifier.detect(frame, scale_factor: 1.5, min_size: Size.new(30, 30)) rect = rects.sort_by(&:area).last # pick the biggest face found frame.draw_rectangle(rect, Color.new(255, 0, 0)) if rect window.show(frame) break if GUI::wait_key(100) > 0 end
Basically I wanted to avoid having multiple faces in the image so I did a quick “pick the biggest one” and drew a box around it in the frame.
Wait, is it seriously that easy?
Friendly Bot Milestones
Write some code that can detect a face in an image
- Write some artoo code that can control a servo
- Make a webcam mount with servos for pan and tilt
- Write some artoo code that controls the roomba
- Load code onto the Beagle Bone Black that controls the roomba and servos
- Load code onto the Beagle Bone Black that can detect faces seen by the webcam
- Write some glue code on the Beagle Bone Black that turns the roomba and servos to keep a face in the center of the image
- Make the roomba wander until it finds a face to follow (first time we can use the title Friendly Bot)
- Make friendly bot play the “curious” beeps when it finds a face
- Make Friendly Bot recognize family members and do “happy” or “skeptical” beeps