Raspberry Pi video surveillance
3 04 Sep 2017 03:55 by u/kevf4
I've been working on a video surveillance project for the raspberry pi and finally got around to posting the code. It's written in C++ and uses ASIO, OpenGL, and libjpeg-turbo to do a lot of the heavy lifting. Let me know what you think.
16 comments
12 u/Palaver 04 Sep 2017 04:26
Impressive bit of work there. Found it pretty readable at first run through (which surprised me considering how long it has been since I've touched c++).
I would recommend moving to a different git host though, GitHub has banned developers for unpopular speech on a few occasions. Bitbucket is a little better, and still free. Others exist (gitlab is completely open source I believe).
1 u/Xicronic 04 Sep 2017 15:51
I second GitLab. They don't support full free speech, but they are fully open source and have free private repos.
1 u/BentAxel 04 Sep 2017 04:27
Mother Effer. It tracks the objects in the frame? That's some super cool stuff.
1 u/KikeFree 04 Sep 2017 04:54
I made a system for networked H264 cameras, with realtime tracking of multiple targets that works well.
So, some comments from my experience that might be helpful:
Where these things usually fall down, particularly with multiple HD streams is from the load on the CPUs.
I highly recommend reading and decoding the full resolution stream into a framebuffer (looks like you get jpegs) (generally you want the full frames saved for the record if motion is detected anyway), make a lower resolution copy of the frame buffer to run your motion detection/tracking algorithms on (cubic or linear interpolation). Multiple passes of gaussian blur with a running average frame subtracted from the live image, combined with a hole filling algorithm will produce blobs that cover anything moving. You can measure the blob sizes to discard blowing leaves, or massive shadows from moving clouds, and finding the center of the blobs for targeting/tracking purposes is trivial. Use a mask image the same dimensions in concert with your highly optimized gaussian blur algorithm to avoid calculations on pixels outside of your interest.
Reduce your stream buffer size to a minimum to keep tracking lag to a minimum.
I haven't open sourced it, but maybe one day.
1 u/kevf4 [OP] 04 Sep 2017 05:27
You're right about HD processing being an issue. I've worked on a system used for airports and prisons, and they usually have two streams per optical camera. One HD stream that goes to the VMS and another low res stream for object detection + onvif metadata generation. This was just a hobby project for learning and I didn't want to go all-in, otherwise I would have used ffmpeg or opencv.
For blob detection I'm just using a simple depth-first algorithm and culling based on user parameters and a heatmap of negatively scored detections. I tried using random forests for culling, but the algorithm required a bit too much tweaking to get it to work correctly. A few people I work with have researched SVMs and deep learning networks, but those required a massive amount of data that I'm not too interested in collecting.
1 u/KikeFree 04 Sep 2017 05:38
More than one camera can help reject false positives like rain drops or blowing snow.
When it comes down to intelligently rejecting movement, the probable identity of the object needs to be determined, and it's so difficult even a human watching can reject legitimate movement or needs a double-take to figure out wtf it was that changed. For some things thermal imaging can help. I don't think conventional deep learning is the way forward, I lean towards making progress on nested pattern recognition.
1 u/1moar 04 Sep 2017 16:25
Saved, thanks mate.
1 u/hedidnothingwrong 04 Sep 2017 16:35
Hello, I've got a few questions,
Why didn't you use Opencv?
What kind of algorithm do you use? I guess it's probably some background subtraction algorithm, do you have any links (papers)?
Thank you in advance.
1 u/kevf4 [OP] 04 Sep 2017 18:28
The reason I didn't use OpenCV is because I used this project for learning and I wanted to try some new approaches. Also, I wasn't sure how well OpenCV would perform on a Raspberry Pi . I'm using OpenGL shaders for processing when possible.
The function Analytics::update in Analytics.cpp on line 637 gives a high level overview of the detection algorithms. In a nutshell:
A background frame is calculated which is the average of frames over the last few minutes. This may change to a Gaussian distribution to reduce spikes, but so far simple averaging is working well. (see Shaders::incrementShaderSource at Shaders.cpp line 141).
The current frame frame is then subtracted from the background frame, which generates a motion mask. (see Shaders::motionMaskShaderSource at Shaders.cpp line 102).
Blobs are detected within the motion mask by using a depth first search. (see Analytics::_detectObjects at Analytics.cpp line 91).
Blobs are tracked using a simple distanced based approach. (see Analytics::_trackObjects at Analytics.cpp line 284).
When a user "trains" the system it is essentially creating a heatmap with thresholds. If the heatmap has a "hot" spot, this tells the algorithm to ignore alarms in that area. (see ExclusionZones.cpp)
I don't have any documentation or diagrams. This project was never really intended for public use, but I am sharing because it could be useful to someone.
0 u/hedidnothingwrong 04 Sep 2017 19:24
Thanks, it was a very clear explanation.
0 u/brianforward 04 Sep 2017 16:23
It doesn't open...
0 u/ThisIsntMe123 04 Sep 2017 19:19
How is this different from MotionEyeOS or whatever?
1 u/kevf4 [OP] 04 Sep 2017 20:50
I don't have any experience with MotionEyeOS, but it looks very similar. My project (pisentinel) supports SeekThermal cameras, IP MJPEG cameras, has a training mode to prevent false alarms, and supports up to 720p frame processing. I will try to throw together a video sometime this week that breaks down the features.
0 u/ThisIsntMe123 04 Sep 2017 21:14
Optional cloud/ftp backup would be cool.