I have been using the Raspberry Pi products for a couple of years now, and recently purchased a Raspberry Pi 4 (8GB model). As a software developer I have written Go code with two independent forks, the first using GoCV (Golang port or wrapper for the latest version of OpenCV) and also a second fork that does not use GoCV, but rather a webcam driver that utilises V4L2. As of this date (January 2021) my entire tool chain is up to date so I am not using any older versions of anything. the Raspberry Pi is running the latest Raspberry OS (32 bit version as it appears the 64Bit version is still not available except in Beta). I have also tried using Ubuntu MATE and the full blown Ubuntu build for the Raspberry Pi, but I think there may be better hardware support (specifically the CSI port) and hence I have reverted to Raspbian 5.4.83-v7l+
My problem is, no matter what I do I am unable to achieve more than about 4-5 frames per second, and with terrible latency. I have run Raspivid and that appears to be running smoothly, however I am pretty sure it is capturing H.264 compressed video, and this is useless to me for a machine vision project (I need lossless video frames). Nowhere have I been able to find any actual Raspberry Pi specification that clearly states what sort of throughput you can expect to get from the camera (Using the v1 OV5647 device, I have a few of them). I don't know whether I have a software issue or perhaps this is simply the Raspberry Pi tapped out and it is not able to serve up more than 5FPS at 1920*1080 or even 640*480 resolution.
I originally installed GUVCView but could not get it to work regardless of what settings I used. It simply shows a black screen, even when I change the Auto Exposure to manual (as suggested by someone online) and/or set Absolute Exposure to 156 (or anything else). I am using the default image format of YUYV 4:2:2.
I have also tried installing qv4l2 (as per this link https://hackernoon.com/polising-raspber ... a-3z113u18) and used lsmod to determine that I needed to
Code: Select all
sudo modprobe bcm2835-v4l2
The sheer lack of documentation and brevity of online information means that after 2 full days fighting with this, I still have nothing more than a 5FPS and pixelated image in the V4L2 Test Bench as mentioned, or 30FPS of complete black image in my application program, or about 3FPS with huge latency, but good image quality in my Go application. there are plenty webpages and forums telling you how easy it is to get started, and use Raspistill or Raspivid to generate a video, but as soon as you want to get your hands dirty the Raspberry Pi specific information regarding use of the CSI port (or even USB for that matter) dries up rapidly. I have also been consulting more generic information such as the V4L API, but again, without knowing what the actual limitations are for the Raspberry Pi I may be chasing ghosts when it comes to trying to get performance that is simply not possible. Very frustrating, honestly it's like looking up information about car tyres and reading about performance and design of tyres in general, but not knowing whether the tyres I want to buy are only fit for a Ferrari or will they actually work on my Toyota...
Initially I thought it was my non-optimised code, but after spending a couple of days tweaking it and using both a GoCV and a V4L2 approach, both of which are delivering the same bad performance, I am coming to the conclusion that this is a camera setup issue. I have proven now that the majority of the time my code is waiting for the next video frame, and it is not the processing of the frame that is consuming all of the applications resources. As a test I had my application looping around reprocessing the same image, and I can easily achieve several hundred "FPS" but as soon as I enable the code that acquires the image from camera, this performance collapses completely. I have also had to increase GPU memory (in boot config) from 128MB to 256MB, else my program simply locks up when I attempt to process an image of a size approaching 1920x1080.
It's my understanding that the CSI port is directly coupled to the GPU, as opposed to the CPU. Is this correct? It makes sense to me, as then hardware acceleration can be utilized and I would think the GPU to be very capable in terms of rapid image processing, but again, just so little data online to demonstrate this.
Any assistance would be greatly appreciated, especially if there are more knowledgable people out there than me, who are well acquainted with efficient and lossless image capture fro the CSI camera. Thank you