The Challenge of Running Object Detection on Raspberry Pi

Object Detection

The Challenge of Running Object Detection on Raspberry Pi

Sabina Pokhrel / November 4, 2019

Here, I will share my struggle story of running object detection on a Raspberry Pi.

Have you ever been so excited after reading a how-to tutorial or a GitHub readme file, only to discover after following it through that it is not really as simple as they said it would be?

I have had countless such instances and it is a rare moment that I find that some solution that I downloaded or a tutorial that I followed simply worked. That moment though! If you resonate with this and have stories of your own to share, I’m happy to hear about it. (Feel free to leave a comment below).

I love to try out new technologies, and see them working and it was only a year ago, that I was introduced to Raspberry Pi, a small, affordable device, where you can attach cameras and other sensors, and deploy your Python code and see it working instantly. How cool is that?

I immediately ordered one for myself, with a Pi camera, a Raspberry Pi Sense HAT, and with a little casing from Raspberry Pi Australia’s website. I couldn’t wait to play with it.

Setting Up the Raspberry Pi for Object Detection

When I got the Raspberry Pi 3 Ultimate Kit the following week, I had my object detection Python script ready to copy and run on the Pi. I opened the package, took out the Pi and the Pi camera, assembled it, put it into the black case, and connected it to the power source.

I did not have a mouse or a keyboard. I had a TV and an HDMI cable that came within the Raspberry Pi kit, and so my first struggle was to connect it to the internet.

My second one was to copy my Python script into the Pi. After a couple of hours of research and trying different things that people had to say online, I got my Pi connected to the internet, and with VNC viewer, I was now able to remotely connect to it from my laptop and copy over my Python script.

If you are struggling to connect your Pi to the internet using wifi and remotely access it, use this tutorial “Directly Connecting to your Raspberry Pi”. It is very helpful, simple to follow, and more importantly, it works!

Raspberry Pi (Photo by Jeff Loucks on Unsplash)

I would have jumped up and down if my program worked right away. Probably would have shouted from my balcony, telling everyone that it worked. Sadly, it didn’t work. It was complaining about missing libraries, and even tackling one after the other, it never seemed to stop. Somehow I even ended up corrupting pip install in the Pi. I think this is what a lot of other developers face as well; missing dependency, version not supported, incompatible platform, and the list is never-ending.

Error (Image by bartekhdd from Pixabay)

Wouldn’t it be nice if there was a script or a program that I could run and it would install all the required libraries and dependencies for me.

Does it have to be this hard?

The next day, I managed to finally get the object detection code working on a Raspberry Pi that could detect different objects and draw a bounding box around it. I used YOLOv3, which was a state-of-the-art object detection model back then, extremely fast and accurate. I was happy. Yeyy! Finally!

I noticed that the output video, with the bounding box, was lagging, like when you have bad internet connectivity and you are trying to watch a video on YouTube. It was bad!

I was excited that it worked but at the same time was kind of disappointed that the fastest object detection was not fast enough on the Pi. I mean, it would not even make a good demo to show it to my parents or friends.

So my next struggle was to make it run faster on the Pi.

The Struggle of Running Real-Time Object Detection

After tons of research, I found out that I was not the only one struggling with it and that it was not an easy problem to solve. In fact, it was one of the challenges that most developers were talking about when running object detection on an edge device.

“We were able to get up to 0.9 frames per second, which is not fast enough to constitute real-time detection. That said, given the limited processing power of the Pi, 0.9 frames per second is still reasonable for some applications.” — Computer vision expert and blogger Adrian Rosebrock, mentions in one of his blogs.

There were some models like SqueezeNet and MobileNetSSD that developers were talking about that were optimized for Raspberry Pi, and could get somewhat better results. But again, there was a learning curve for that.

I was so frustrated that I had to go through so much trouble to do a little experiment that I wanted. If speed was the problem, how can I expect to use this for a real-time solution? There may be cases where it wouldn’t have to process frames in real-time, and one frame every 10–20 seconds were acceptable, but what about solutions that needed to detect objects more often.

Eventually, I ended up using the darknet implementation of YOLOv3 with a pre-trained Tiny YOLOv3 model. The inference speed was better than what I had, but I was not satisfied with the overall performance of the solution. I had a lot of questions and concerns.

  • Why is it taking up so much space? I cannot run any other code in parallel.
  • Why is the Pi heating up so much? It feels like it will blow up any second.
  • My Pi freezes so often, and I have to restart it. I have no idea why it’s happening or how to fix it.
  • Why the hell is this Pi restarting again and again?

My search for a faster object detection that performed better than YOLOv3 on a Raspberry Pi continued up until three months ago when I found out that Xailient, a startup in Sydney, had out-performed YOLOv3 and Tiny YOLOv3 and had set a new benchmark.

It was great news for me and I knew that somehow I had to get my hands on their object detection model and try it out. Three months later, here I am at Xailient (which is a story for another time), experimenting with the current state-of-the-art real-time object detection on the Pi.

What are some of the challenges that you have faced running object detection on a Raspberry Pi?

Xailient is commercializing breakthrough university research in Artificial Intelligence and Machine Learning. Our technology dramatically reduces the costs of data transmission, storage, and computation associated with extracting useful information from real-time video by processing the way humans think.

Spread the love

About the Author

    Sabina is an AI Specialist and Machine Learning Engineer. She is a Writer and a former Editorial Associate at Towards Data Science.

Trusted by

Xailient’s commercial partners

Press releases

March 14, 2022

Xailient specializes in extremely efficient low-power computer vision. Intel's OpenVINO specializes in maximizing the performance and speed of computer vision AI workloads. OpenVINO improved Xailient FPS 9.5x on Intel hardware to 448 FPS. Together, Xailient-Intel outperforms the comparable MobileNet_SSD by 80x. Even after Intel worked the OpenVINO magic on MobileNet_SSD, Xailient-OpenVINO is 14x faster.

November 29, 2021

Xailient’s Face Recognition enables high-speed edge AI processing with low-power consumption using Sony’s IMX500 – a chip so small it can fit on the tip of your finger.

OnEdge Newsletter

A weekly newsletter with the best hand-picked resources about Edge AI and Computer Vision

OnEdge is a free weekly newsletter that keeps you ahead of the curve on low-powered Edge devices and computer vision AI.


You’ll get insights and resources into:

  • Edge computing use cases.
  • Market trends in Edge computing.
  • Computer Vision AI at the Edge.
  • Machine learning at the Edge.
Cookie Policy