bsdz
Posts: 19
Joined: Mon Dec 30, 2013 5:00 pm

New V4L2 driver and OpenCV

Wed Jan 01, 2014 10:54 pm

Hi

I've managed to set up and use the new driver as shown in original post here. However, I am having problems using the new driver from opencv-python with my RaspberryPi camera board. When I run a simple command line that attempts to capture from the video camera I receive "VIDIOC_QUERYMENU: Invalid argument" errors and I don't see the camera LED turn on (even briefly):

Code: Select all

pi@raspberrypi ~ $ python -c "import cv; cv.CaptureFromCAM(-1)"
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
I've tried checking the dmesg logs and stracing the python script but haven't been able to find out more about what V4L2 calls are failing. Does anyone have any suggestions?

Thanks,
Blair

oldnpastit
Posts: 34
Joined: Wed Dec 04, 2013 7:57 am

Re: New V4L2 driver and OpenCV

Wed Jan 01, 2014 11:44 pm

That's a v4l2 ioctl, described here:

http://linuxtv.org/downloads/v4l-dvb-ap ... yctrl.html

A brief look through the code for the v4l2 bcm2835 driver and it looks like that ioctl is not supported (drivers/media/platform/bcm2835/bcm2835-camera.c).

You would need to add an implementation to the driver. This sounds harder than it actually is - the header file with the relevant function you need to fill in is defined here:

http://lxr.free-electrons.com/ident?i=vidioc_querymenu

bsdz
Posts: 19
Joined: Mon Dec 30, 2013 5:00 pm

Re: New V4L2 driver and OpenCV

Thu Jan 02, 2014 9:35 am

It's been a while since I compiled a kernel let alone dabbled in any driver code. Thanks for the links. I'll definitely take a look. Hopefully I'll be able to cross compile the kernel with mingw or cygwin.
oldnpastit wrote:That's a v4l2 ioctl, described here:

http://linuxtv.org/downloads/v4l-dvb-ap ... yctrl.html

A brief look through the code for the v4l2 bcm2835 driver and it looks like that ioctl is not supported (drivers/media/platform/bcm2835/bcm2835-camera.c).

You would need to add an implementation to the driver. This sounds harder than it actually is - the header file with the relevant function you need to fill in is defined here:

http://lxr.free-electrons.com/ident?i=vidioc_querymenu

oldnpastit
Posts: 34
Joined: Wed Dec 04, 2013 7:57 am

Re: New V4L2 driver and OpenCV

Thu Jan 02, 2014 4:28 pm

Cross-compiling the kernel using cygwin or mingw is going to be very very painful! Your first problem will be building a gcc toolchain that runs under Cygwin and generates ARM6-hf binaries. Even the thought of doing that makes me feel ill!

If you've got a Windows machine, you might be better off putting a virtual machine on it (e.g. VirtualBox). Or build the kernel on your PI directly - it will be slow but still faster than figuring out how to make a Cygwin/armhf cross-compiler.

Let me know if you get any problems with the actual kernel module code - it's pretty easy really.

Luke

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12154
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New V4L2 driver and OpenCV

Thu Jan 02, 2014 5:49 pm

Looking at the v4l2-core code for v4l_querymenu(), you can either do one of three things:

Code: Select all

	if (vfh && vfh->ctrl_handler)
		return v4l2_querymenu(vfh->ctrl_handler, p);
	if (vfd->ctrl_handler)
		return v4l2_querymenu(vfd->ctrl_handler, p);
	if (ops->vidioc_querymenu)
		return ops->vidioc_querymenu(file, fh, p);
If I've read through the core code correctly, we do the first, so the call is supported, just not by an ioctl call all the way into the driver.

This is also confirmed by doing

Code: Select all

v4l2-ctl --list-ctrls-menus
for which the help text says
-L, --list-ctrls-menus
display all controls and their menus [VIDIOC_QUERYMENU]
So that appears to be the same ioctl. We also pass conformance for all our parameters, so that implies we haven't got any obvious nasties in there.

@bsdz I've spent 5 mins trying to find OpenCV for python on Pi, but failed. Could you give some noddy instructions on the packages that I need please?

bsdz
Posts: 19
Joined: Mon Dec 30, 2013 5:00 pm

Re: New V4L2 driver and OpenCV

Thu Jan 02, 2014 6:02 pm

I had a funny feeling most of the code was there already. To install opencv for python try:

Code: Select all

sudo apt-get install libopencv-dev python-opencv
This probably doesn't install a debugable binary. Although I'm not entirely sure about that.
6by9 wrote:Looking at the v4l2-core code for v4l_querymenu(), you can either do one of three things:

Code: Select all

	if (vfh && vfh->ctrl_handler)
		return v4l2_querymenu(vfh->ctrl_handler, p);
	if (vfd->ctrl_handler)
		return v4l2_querymenu(vfd->ctrl_handler, p);
	if (ops->vidioc_querymenu)
		return ops->vidioc_querymenu(file, fh, p);
If I've read through the core code correctly, we do the first, so the call is supported, just not by an ioctl call all the way into the driver.

This is also confirmed by doing

Code: Select all

v4l2-ctl --list-ctrls-menus
for which the help text says
-L, --list-ctrls-menus
display all controls and their menus [VIDIOC_QUERYMENU]
So that appears to be the same ioctl. We also pass conformance for all our parameters, so that implies we haven't got any obvious nasties in there.

@bsdz I've spent 5 mins trying to find OpenCV for python on Pi, but failed. Could you give some noddy instructions on the packages that I need please?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12154
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New V4L2 driver and OpenCV

Thu Jan 02, 2014 6:09 pm

Having just spent a couple more minutes looking at this, the error is "VIDIOC_QUERYMENU: Invalid argument" which is presumably EINVAL. If the thing wasn't implemented then -ENOTTY gets returned and you'd see a different error message.

This implies that some invalid setting or settings are being requested, and the driver is correctly returning an error. Without seeing exactly what was being asked for, it's hard to say what is wrong I'm afraid, and this is all handled in the core code, so we can't even use any debug from the driver.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

bsdz
Posts: 19
Joined: Mon Dec 30, 2013 5:00 pm

Re: New V4L2 driver and OpenCV

Thu Jan 02, 2014 6:20 pm

No worries. I'll have a look at the opencv source code later this evening and see what I can find. Thanks for looking at it.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12154
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New V4L2 driver and OpenCV

Thu Jan 02, 2014 7:15 pm

I've added debug to the core V4L2 code, and it is thinking that V4L2_CID_EXPOSURE_AUTO and V4L2_CID_AUTO_N_PRESET_WHITE_BALANCE values should be "skipped".
It seems to be trying to get the text for values
- V4L2_EXPOSURE_SHUTTER_PRIORITY
- V4L2_EXPOSURE_APERTURE_PRIORITY
- V4L2_WHITE_BALANCE_MANUAL
none of which are supported. I think that is correct if it is trying to draw a menu and finds that some values are invalid, and the code appears to deal with that.

Sorry to be the bearer of bad news, but that seems not to be where your problem lies. It seems to set everything up for a capture at
Width/Height : 64/32
Pixel Format : 'BGR3'
Field : None
Bytes per Line: 192
Size Image : 6144
Colorspace : SRGB
but never starts streaming or passing buffers to be filled. It appears to be something up in OpenCV that is aborting, so we'll have to leave you or others to debug that.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

bsdz
Posts: 19
Joined: Mon Dec 30, 2013 5:00 pm

Re: New V4L2 driver and OpenCV

Wed Feb 05, 2014 10:17 am

I think I've finally found out what is causing the opencv to crash. Opencv uses xioctl to check each control menu item and for some reason the camera controls doesn't set the correct maximum and minimum index on some menu items.

One can see this using v4l2-ctl --list-ctrls-menus

Code: Select all

      auto_exposure (menu)   : min=0 max=3 default=0 value=0
                0: Auto Mode
                1: Manual Mode

      white_balance_auto_preset (menu)   : min=0 max=9 default=1 value=1
                1: Auto
                2: Incandescent
                3: Fluorescent
                4: Fluorescent H
                5: Horizon
                6: Daylight
                7: Flash
                8: Cloudy
                9: Shade
The maximum for auto exposure is set to 3 but there are only 2 sub items. Likewise for white_balance_auto_preset the minimum is set to 0 but the first sub item starts at 1.

I will also cross post this to the main v4l thread.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12154
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New V4L2 driver and OpenCV

Wed Feb 05, 2014 12:53 pm

bsdz wrote:I think I've finally found out what is causing the opencv to crash. Opencv uses xioctl to check each control menu item and for some reason the camera controls doesn't set the correct maximum and minimum index on some menu items.

One can see this using v4l2-ctl --list-ctrls-menus
<snip>
The maximum for auto exposure is set to 3 but there are only 2 sub items. Likewise for white_balance_auto_preset the minimum is set to 0 but the first sub item starts at 1.

I will also cross post this to the main v4l thread.
That is broadly the comment I'd made back in my post of Thu Jan 02, 2014 7:15 pm. This is permitted behaviour in V4L2.

Both of those parameters have an enum v4l2_ctrl_type of V4L2_CTRL_TYPE_MENU. At the driver level at least, then under these conditions the step value is the bitmask of the supported values. The driver also doesn't get a chance to set the minimum value - the control gets added with a call to v4l2_ctrl_new_std_menu (in drivers/media/v4l2-core/v4l2-ctrls.c), which has a prototype of

Code: Select all

struct v4l2_ctrl *v4l2_ctrl_new_std_menu_items(struct v4l2_ctrl_handler *hdl,
                       const struct v4l2_ctrl_ops *ops, u32 id, s32 max,
                       s32 mask, s32 def, const char * const *qmenu)
At the application level looking through v4l2-ctl from v4l2-utils (http://git.linuxtv.org/v4l-utils.git/tr ... s/v4l2-ctl), print_qctrl() in v4l2-ctl-common.cpp is doing:

Code: Select all

        if ((queryctrl->type == V4L2_CTRL_TYPE_MENU ||
             queryctrl->type == V4L2_CTRL_TYPE_INTEGER_MENU) && show_menus) {
                for (i = queryctrl->minimum; i <= queryctrl->maximum; i++) {
                        qmenu.index = i;
                        if (test_ioctl(fd, VIDIOC_QUERYMENU, &qmenu))
                                continue;
                        if (queryctrl->type == V4L2_CTRL_TYPE_MENU)
                                printf("\t\t\t\t%d: %s\n", i, qmenu.name);
                        else
                                printf("\t\t\t\t%d: %lld (0x%llx)\n", i, qmenu.value, qmenu.value);
                }
        }
to iterate over the supported menu options. It appears at first glance that there is no way to find out the skip mask without asking for the menu options and observing a failure.
OpenCV should probably be doing something similar.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

bsdz
Posts: 19
Joined: Mon Dec 30, 2013 5:00 pm

Re: New V4L2 driver and OpenCV

Thu Feb 06, 2014 9:37 pm

Apologies for earlier cross post. I've followed the example as suggested from v4l2-ctl and this fixes the error and the camera capture appears to work as expected.

I've submitted a pull request to opencv here.

magnatag
Posts: 33
Joined: Tue Mar 04, 2014 8:39 pm

Re: New V4L2 driver and OpenCV

Tue Mar 04, 2014 8:42 pm

I just installed opencv 2.4.2 on my rpi with python support. Am I correct to think that I need to upgrade that to 2.4.8.1 to be able to use this V4L2 driver and opencv?

bsdz
Posts: 19
Joined: Mon Dec 30, 2013 5:00 pm

Re: New V4L2 driver and OpenCV

Tue Mar 04, 2014 11:19 pm

The OpenCV V4L2 driver has a bug that breaks on Raspberry Pi (and likely elsewhere too). You can find some patch files on Open CV's bug tracking forum that fix the issue as much as they worked for me. Unfortunately, I couldn't get my head around using git to set up a pull request for the main developers to apply the patch to the main branch so it's unlikely we'll see my patches being applied any time soon :(

magnatag
Posts: 33
Joined: Tue Mar 04, 2014 8:39 pm

Re: New V4L2 driver and OpenCV

Wed Mar 05, 2014 1:10 pm

Just so I'm clear, to install the patch files, I would copy and overwrite the old ones and then run Make again?

bsdz
Posts: 19
Joined: Mon Dec 30, 2013 5:00 pm

Re: New V4L2 driver and OpenCV

Wed Mar 05, 2014 1:24 pm

Close. You need to use cmake and make. Follow the instructions here. This will take a while to compile on your Pi.
magnatag wrote:Just so I'm clear, to install the patch files, I would copy and overwrite the old ones and then run Make again?

User avatar
waveform80
Posts: 371
Joined: Mon Sep 23, 2013 1:28 pm
Location: Manchester, UK
Contact: Website Twitter

Re: New V4L2 driver and OpenCV

Wed Mar 05, 2014 1:58 pm

bsdz wrote:The OpenCV V4L2 driver has a bug that breaks on Raspberry Pi (and likely elsewhere too). You can find some patch files on Open CV's bug tracking forum that fix the issue as much as they worked for me. Unfortunately, I couldn't get my head around using git to set up a pull request for the main developers to apply the patch to the main branch so it's unlikely we'll see my patches being applied any time soon :(
I've managed to fork the OpenCV repository on GitHub and applied your patches against the 2.4 branch. The test suite passes successfully, so if you'd like I could submit this as a pull request on your behalf?
Author of / contributor to a few pi related things (picamera, Sense HAT emulator, gpiozero, piwheels, etc.), and currently a software engineer at Canonical responsible for Ubuntu on the Raspberry Pi.

bsdz
Posts: 19
Joined: Mon Dec 30, 2013 5:00 pm

Re: New V4L2 driver and OpenCV

Wed Mar 05, 2014 2:05 pm

waveform80 wrote:I've managed to fork the OpenCV repository on GitHub and applied your patches against the 2.4 branch. The test suite passes successfully, so if you'd like I could submit this as a pull request on your behalf?
That would be brilliant thanks. I'm still very new to git and couldn't figure it out. :-)

magnatag
Posts: 33
Joined: Tue Mar 04, 2014 8:39 pm

Re: New V4L2 driver and OpenCV

Wed Mar 05, 2014 3:19 pm

That sounds great! I've messed with git hub a bit before and it's definitely not straight forward.

Any advice on how I can point RPi to use my newer OpenCV installation instead of 2.3.1? (http://www.raspberrypi.org/forum/viewto ... 28&t=71288)

Thanks!

bsdz
Posts: 19
Joined: Mon Dec 30, 2013 5:00 pm

Re: New V4L2 driver and OpenCV

Wed Mar 05, 2014 3:31 pm

magnatag wrote:That sounds great! I've messed with git hub a bit before and it's definitely not straight forward.

Any advice on how I can point RPi to use my newer OpenCV installation instead of 2.3.1? (http://www.raspberrypi.org/forum/viewto ... 28&t=71288)

Thanks!
You'll need to set your LD_LIBRARY_PATH to the opencv library folder, something like:

Code: Select all

foo@bar:~$ LD_LIBRARY_PATH=/path/to/opencv/libso/folder python mycode.py

magnatag
Posts: 33
Joined: Tue Mar 04, 2014 8:39 pm

Re: New V4L2 driver and OpenCV

Wed Mar 05, 2014 4:43 pm

I got it all fixed. I was so excited it finally finished making, I didn't run sudo make install make. Now it should be all set. I'm going to apply your patch later today. Thank you for the help!

Joose
Posts: 9
Joined: Fri Mar 21, 2014 11:10 pm

Re: New V4L2 driver and OpenCV

Fri Mar 21, 2014 11:31 pm

I'm using OpenCV 2.4.8 getting this error:

Code: Select all

python -c "import cv2; cv2.VideoCapture(-1)"

HIGHGUI ERROR: V4L/V4L2: VIDIOC_CROPCAP

Joose
Posts: 9
Joined: Fri Mar 21, 2014 11:10 pm

Re: New V4L2 driver and OpenCV

Mon Mar 24, 2014 6:06 pm

Joose wrote:I'm using OpenCV 2.4.8 getting this error:

Code: Select all

python -c "import cv2; cv2.VideoCapture(-1)"

HIGHGUI ERROR: V4L/V4L2: VIDIOC_CROPCAP
Anyone have any ideas? I can't get around this error even with a fresh install of Rasbpiamän and OpenCV

Joose
Posts: 9
Joined: Fri Mar 21, 2014 11:10 pm

Re: New V4L2 driver and OpenCV

Fri Mar 28, 2014 1:20 am

Or am I actually missing some step, setting up the camera or something? What is the correct way to do this?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12154
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New V4L2 driver and OpenCV

Fri Mar 28, 2014 8:10 am

<slightly grumpy>
Put "VIDIOC_CROPCAP" into the little search box at the top right of any forum page and you'll get hits for:
http://www.raspberrypi.org/forum/viewto ... 00#p487300 and http://www.raspberrypi.org/forum/viewto ... 31#p487431
Whilst not giving you an answer to how to resolve the issue (although pondruska implies the failure is non-fatal and that they do still get an image, and even gives you the code he is using), it does give you an explanation as to what the error means.

If you wanted to be really keen, then the source for that bit of OpenCV is at https://github.com/Itseez/opencv/blob/m ... .cpp#L2473, and that appears to confirm that the message is not fatal (it just doesn't populate a couple of fields in a structure).
If your capture is failing, then it sounds like you need to be looking elsewhere - give pondruska's code a try.
</slightly grumpy>
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”