Qt 5.x and camera breaks for firmwares after 66e4e823e1d24f4

[m_rx]

I'm developing a multimedia app under Qt 5.x which uses gstreamer-0.10.
Previewing and capturing still images from the rpi-camera only works for me for firmwares up to and including 66e4e823e1d24f46fcc11a3cd6959a835ad6dc73 , which is 3.10.29 .

i've tried qt 5.1.x and 5.2.x from http://twolife.be/raspbian . same issue: the app halts when trying to access the camera.

i know that after 3.10.29 a lot of v4l2 changes happened.

is there a way to make it work?

an strace seems to indicate that gstreamer can't find a usable/matching capture-mode when trying to open the camera:

Code: Select all

ioctl(27, VIDIOC_TRY_FMT, 0xbe951e94)   = 0
ioctl(27, VIDIOC_ENUM_FRAMEINTERVALS, 0xbe95209c) = 0
ioctl(27, VIDIOC_G_FMT or VT_SENDSIG, 0xbe951f60) = -1 EINVAL (Invalid argument)
... (loads of these) ...
ioctl(27, VIDIOC_TRY_FMT, 0xbe951e94)   = 0
ioctl(27, VIDIOC_ENUM_FRAMEINTERVALS, 0xbe95209c) = 0
ioctl(27, VIDIOC_G_FMT or VT_SENDSIG, 0xbe951f60) = -1 EINVAL (Invalid argument)
ioctl(27, VIDIOC_TRY_FMT, 0xbe951e94)   = 0
ioctl(27, VIDIOC_ENUM_FRAMEINTERVALS, 0xbe95209c) = 0
... (changes to) ...
gettimeofday({1401086957, 725631}, NULL) = 0
gettimeofday({1401086957, 727217}, NULL) = 0
ioctl(27, VIDIOC_G_FMT or VT_SENDSIG, 0xbe951f60) = -1 EINVAL (Invalid argument)
ioctl(27, VIDIOC_TRY_FMT, 0xbe951408)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe951408)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe951408)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe951408)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe951408)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe951408)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe951408)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe951408)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe951408)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe9513f8)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe9513f8)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe9513f8)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe9513f8)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe9513f8)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe9513f8)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe9513f8)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe9513f8)   = 0
ioctl(27, VIDIOC_TRY_FMT, 0xbe9513f8)   = 0
ioctl(27, VIDIOC_ENUM_FRAMEINTERVALS, 0xbe95209c) = 0
ioctl(27, VIDIOC_G_FMT or VT_SENDSIG, 0xbe951f60) = -1 EINVAL (Invalid argument)

the last pattern repeats over and over, the program never recovers.

[6by9]

Gstreamer has issues when the V4L2 driver advertises the supported resolutions as being of type V4L2_FRMSIZE_TYPE_STEPWISE. Two guesses what kernel commit 983dcb1 on Feb 12 does! (committed to main repo on Feb 21) (vidioc_enum_framesizes is the function in question)
That mode means you give a min width, max width, and a step value. Same for height. Any combination of those widths and heights are permitted. For some odd reason Gstreamer apparently tries to create an array containing all the possible combinations. When our widths can range from 16 to 2592, and height from 16 to 1944 with increments of 2 in each case, that is 1.2 million combinations that it is trying to compute!

More info from the Pi forums on http://www.raspberrypi.org/forums/viewt ... 31#p520964, http://www.raspberrypi.org/forums/viewt ... 31#p521235 and a few other posts on that thread (mainly between towolf and I).
And Gstreamer bug report https://bugzilla.gnome.org/show_bug.cgi?id=726521

[m_rx]

That sounds reasonable. I had read about the gstreamer issue but didn't make the connection to Qt ...
Both Qt and gstreamer seem slow/ not very interested in this issue ...

Any way to make it work?
Maybe a "compat" mode for the v4l driver?

[6by9]

That sounds reasonable. I had read about the gstreamer issue but didn't make the connection to Qt ...
Both Qt and gstreamer seem slow/ not very interested in this issue ...

Any way to make it work?
Maybe a "compat" mode for the v4l driver?

It really is Gstreamer rather thn Qt at fault here.
I guess we could do a "gstreamer-broken-framesizses-workaround=1" command-line parameter to just not return a result to vidioc_enum_framesizes (having now worked out how to do those, it is trivial). It does irk to have to do nasty hacks like that, and the gstreamer team have acknowledged that their module is broken. The Pi is one of the few V4L2 drivers to use stepwise mode, so I guess it doesn't matter to most users.

[m_rx]

i completely understand.

BUT, as gstreamer is a central component in video/camera software, the issue will cut off many users from camera/video and/or newer kernels.
for the gstreamer guys the pi is just another platform. for us, gstreamer is _the_ tool for video/camera. (slightly exaggerating)

Come on, don't make me beg ...

[6by9]

BUT, as gstreamer is a central component in video/camera software, the issue will cut off many users from camera/video and/or newer kernels.
for the gstreamer guys the pi is just another platform. for us, gstreamer is _the_ tool for video/camera. (slightly exaggerating)

Well gstreamer is open source too, so there's nothing stopping you fixing gstreamer yourself and pushing it upstream for the benefit of others
I'll sort something today in the office, but it is a nasty hack so may have some sarcastic parameter name

[m_rx]

@fix gstreamer: if i could find the time, i would ...
@sarcastic name: i can well live with that

thanks a million! that will be great!

[6by9]

Done.

If you do:

• -sudo rpi-update to get the latest kernel and firmware
  - download the kernel module from https://github.com/6by9/RPiTest/blob/ma ... 35-v4l2.ko
  - copy it to /lib/modules/3.12.20+/kernel/drivers/media/platform/bcm2835/ on your Pi
  - amend your modprobe line to read modprobe bcm2835-v4l2 gst_v4l2src_is_broken=1

Should be job done.
There should be an extra line in dmesg

Code: Select all

 bcm2835-v4l2: Work-around for gstreamer issue is active.

and if you try v4l2-ctl --list-framesizes=H264 it should respond with "ioctl: VIDIOC_ENUM_FRAMESIZES", when normally it would respond with

Code: Select all

ioctl: VIDIOC_ENUM_FRAMESIZES
        Size: Stepwise 16x16 - 2592x1944 with step 2/2

That is back to where we were in February, and having looked at the gstreamer code briefly (http://cgit.freedesktop.org/gstreamer/g ... 5dd2#n1879) if the ioctl fails then it makes up a set of defaults.

I will be making the pull request to Dom shortly, so it should be picked up automatically with rpi-update fairly soon. In the meantime you'll have to repeat these steps after any rpi-update.

[towolf]

I guess we could do a "gstreamer-broken-framesizses-workaround=1" command-line parameter to just not return a result to vidioc_enum_framesizes (having now worked out how to do those, it is trivial). It does irk to have to do nasty hacks like that, and the gstreamer team have acknowledged that their module is broken. The Pi is one of the few V4L2 drivers to use stepwise mode, so I guess it doesn't matter to most users.

I really love that you did that. I got a bit depressed when I saw that this issue will probably stay unfixed for a while because the guy sounded a bit like they would "get to it eventually".

I'm only interested in maybe 3 resolutions (namely what the sensor supplies), all those intermediate steps are not so useful to me.

[m_rx]

tested and working!

brilliant, brilliant, brilliant!!!
together with https://github.com/raspberrypi/linux/issues/463 this really makes my day!

THANK! YOU!

is there a slight chance you might post here again when it's gone into the mainstream kernel?

[6by9]

is there a slight chance you might post here again when it's gone into the mainstream kernel?

It already has. Because we had a regression, Dom sorted out a release yesterday.
rpi-update https://github.com/Hexxeh/rpi-firmware/ ... a681d8da55 includes this workaround in the V4L2 driver.

[m_rx]

even better!

may i suggest enabling the workaround by placing

Code: Select all

# workaround to keep gstreamer happy with camera resolution reporting
options bcm2835-v4l2 gst_v4l2src_is_broken=1

in /etc/modprobe.d/bcm_v4l2_workaround.conf

i didn't have a modprobe ... line in my setup.

[6by9]

may i suggest enabling the workaround by placing

Code: Select all

# workaround to keep gstreamer happy with camera resolution reporting
options bcm2835-v4l2 gst_v4l2src_is_broken=1

in /etc/modprobe.d/bcm_v4l2_workaround.conf

i didn't have a modprobe ... line in my setup.

Whichever.
I don't have bcm2835-v4l2 in /etc/modules as it does take up some GPU memory just by loading it, so I load it on demand with modprobe (and unload with rmmod). It's also quick and easy when developing to just upload updated .ko, modprobe, test, amend, rmmod, and repeat as necessary. The joys of there being so many ways to skin the same cat