UV4L deletes all /dev/video* and can't pick H.264 stream of the UVC USB camera

[flux_capacitor]

TL;DR: The problem defined below is intended to anyone who knows about raspberry pi + UVC USB cameras + v4l2 + ffmpeg and UV4L. Indeed UV4L deletes all entries /dev/video* created by the camera, recreates its own, and doesn't pick the correct H.264 camera stream. Even the MJPEG stream is caught at wrong lower resolution.

Some context first.
I have a UVC USB camera board from ELP, ref. ELP-USBFHD06H-L37. It has a 1/2.9" Sony IMX 322 sensor and outputs several streams: MJPEG, YUYV and H.264, up to 1920×1080 pixels @ 30 fps:



I plug this camera in a Raspberry Pi 3 Model B (OS version Stretch).
Here are results from various commands entered at prompt:

Code: Select all

$ lsusb
Bus 001 Device 004: ID 05a3:9422 ARC International 

Code: Select all

$ v4l2-ctl --list-devices
H264 USB Camera: USB Camera (usb-3f980000.usb-1.4):
	/dev/video0
	/dev/video1
	/dev/video2
	/dev/video3

The camera also creates these four specific entries "by id":

Code: Select all

/dev/v4l/by-id/usb-Sonix_Technology_Co.__Ltd._H264_USB_Camera_SN0001-video-index0
/dev/v4l/by-id/usb-Sonix_Technology_Co.__Ltd._H264_USB_Camera_SN0001-video-index1
/dev/v4l/by-id/usb-Sonix_Technology_Co.__Ltd._H264_USB_Camera_SN0001-video-index2
/dev/v4l/by-id/usb-Sonix_Technology_Co.__Ltd._H264_USB_Camera_SN0001-video-index3

and these four entries "by path":

Code: Select all

/dev/v4l/by-path/platform-3f980000.usb-usb-0:1.4:1.0-video-index0 
/dev/v4l/by-path/platform-3f980000.usb-usb-0:1.4:1.0-video-index1
/dev/v4l/by-path/platform-3f980000.usb-usb-0:1.4:1.0-video-index2
/dev/v4l/by-path/platform-3f980000.usb-usb-0:1.4:1.0-video-index3

I made a few probe tests with ffmpeg:

Code: Select all

$ ffmpeg -f v4l2 -list_formats all -i /dev/video0
[video4linux2,v4l2 @ 0x11ccc50] Compressed:       mjpeg :          Motion-JPEG : 1920x1080 1280x720 800x600 640x480 640x360 352x288 320x240 1920x1080
[video4linux2,v4l2 @ 0x11ccc50] Raw       :     yuyv422 :           YUYV 4:2:2 : 640x480 800x600 640x360 352x288 320x240 640x480

Code: Select all

$ ffmpeg -f v4l2 -list_formats all -i /dev/video1
[video4linux2,v4l2 @ 0xce4c50] ioctl(VIDIOC_G_INPUT): Inappropriate ioctl for device

Code: Select all

$ ffmpeg -f v4l2 -list_formats all -i /dev/video2
[video4linux2,v4l2 @ 0x201c50] Compressed:        h264 :                H.264 : 1920x1080 1280x720 800x600 640x480 640x360 352x288 320x240 1920x1080

Code: Select all

$ ffmpeg -f v4l2 -list_formats all -i /dev/video3
[video4linux2,v4l2 @ 0x1684c50] ioctl(VIDIOC_G_INPUT): Inappropriate ioctl for device

So it appears that
/dev/video0 corresponds to both the MJPEG HD stream and the YUYV SD stream.
/dev/video2 corresponds to the H.264 HD stream.

When I output a video file (using a Matroska container to be independent of the codec) with ffmpeg from direct copy of input /dev/video0:

Code: Select all

ffmpeg -i /dev/video0 -vcodec copy ~/output-dev-video0.mkv

I get a playable 800×600 2YUY MKV:

Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 800x600, q=2-31, 115200 kb/s, 15 fps, 15 tbr, 1000k tbn, 1000k tbc

Native bitrate in this case is as might be expected insanely high: over 115 Mbps.

BTW I do not know ho to tap into the MJPEG HD stream of same /dev/video0 input. Anyone? But this is a side question, so let's move on.

I do the same with:

Code: Select all

ffmpeg -i /dev/video2 -vcodec copy ~/output-dev-video2.mkv

I get a playable 1920×1080 H.264 MKV:

Code: Select all

Stream #0:0: Video: h264 (Main) (H264 / 0x34363248), yuv420p(progressive), 1920x1080, q=2-31, 30 fps, 30 tbr, 1k tbn, 1000k tbc

Quality of this latest stream is really very good, with a bitrate around 6.5 Mbps. If I want to restraint a bit the bitrate, say to 2 Mbps, I re-encode the stream (instead of just copy it as is) with h264_omx (I have compiled latest ffmpeg with OpenMax GPU acceleration):

Code: Select all

ffmpeg -i /dev/video2 -vcodec h264_omx -b:v 2M ~/output-dev-video2-omx.mkv

And the quality is still very good. So I am quite pleased so far.

Now, in order to stream and display the video in web browsers, at first I tested motion/motionEye, alas with terrible results. Especially because motion is based on MJPEG and not H.264, so it only outputs a laggy 800x600 stream. It managed to catch the 1920×1080 MJPEG stream of the camer once, but systematically failed with a grey screen afterward.

So I decided to give UV4L a shot: https://www.linux-projects.org/uv4l/

This stream server with HTTP/HTTPS GUI has very good records regarding stability, HD stream quality, and low-latency (200-500 ms only).

Like ffmpeg it uses v4l2 (video4linux2). The software comes with uv4l-uvc driver for USB cameras, in addition to uv4l-raspicam (which is more intended to OV-based CSI pi camera boards, those with the ribbon cable).

Raspicam driver is advertised as being compatible with "any video format available from the GPU (yuv420, nv21, yvu420, rgb565, rgb565x, rgb24, bgr24, rgba, bgra, jpeg Still, mjpeg Video, h264)".

No mention of such extended support for the UVC driver though (?)

Whatever, the real issue here is that as soon as I fire UV4L, it first complains that /dev/video0, /dev/video1, /dev/video2 and /dev/video3 already exist (sure, as the camera is plugged in…) then it creates /dev/video4 to pick the camera stream from there instead, and worse it disables (deletes) the first four /dev/video* !!! It also deletes all camera entries in /dev/by-id/ and /dev/by-path

Code: Select all

$ uv4l --device-id 05a3:9422 --config-file=/etc/uv4l/uv4l-uvc.conf
<notice> [core] Trying to loading driver 'uvc' from built-in drivers...
<notice> [core] Loading driver 'uvc' from external plug-in's...
<notice> [driver] Video functionality 'USB Camera' recognized at 05a3:9422
<notice> [core] Device detected!
<warning> [core] Cannot create /dev/video0 because file already exists
<warning> [core] Cannot create /dev/video1 because file already exists
<warning> [core] Cannot create /dev/video2 because file already exists
<warning> [core] Cannot create /dev/video3 because file already exists
<notice> [core] Registering device node /dev/video4

WTF is that?! This means that when UV4L is running, I can no longer access the other streams with ffmpeg at all, or any other software for that matter. Even worse, from /dev/video4 UV4L decides to point to the 1920×1080 MJPEG stream of the camera* and not its H.264 stream! And I didn't find anywhere how to:

1. either tell UV4L to keep original /dev/video2 and tap into it to pick the H.264 stream of the camera
2. or keep this peculiar way of doing things with new /dev/video4, but point the H.264 stream of the camera instead of the MJPEG stream the software arbitrarily chooses (as it comes first in the list I think).

Is there any solution for one of these two possibilities?

[hr]
* I wrote "1920×1080" but this was yesterday. Now it picks the 800×600 MJPEG stream, always. And there is no width, height nor framerate options in the /etc/uv4l/uv4L-uvc.conf!

[RpiName]

WTF is that?! This means that when UV4L is running, I can no longer access the other streams with ffmpeg at all, or any other software for that matter. Even worse, from /dev/video4 UV4L decides to point to the 1920×1080 MJPEG stream of the camera* and not its H.264 stream!

uv4l does not delete anything. uvcvideo and uv4l-uvc are two different drivers for the same devices. When uv4l-uvc is loaded, uvcvideo is unloaded. when this happens the device nodes created by the latter get destroyed. To change the initial video format/resolution in uv4l-uvc you can use the standard v4l2 tools or the uv4l-server REST API. By the way, whenever uv4l streams webrtc, it switches the device to MJPEG.

EDIT: Another possibility is to tell uv4l to pick a specific video device node created by the another driver (uvcvideo in this case) with the --external-driver=yes and e.g. --device-name=video0 options (you must not specify any --driver option in this case).

[flux_capacitor]

uv4l does not delete anything. uvcvideo and uv4l-uvc are two different drivers for the same devices. When uv4l-uvc is loaded, uvcvideo is unloaded. when this happens the device nodes created by the latter get destroyed. To change the initial video format/resolution in uv4l-uvc you can use the standard v4l2 tools or the uv4l-server REST API. By the way, whenever uv4l streams webrtc, it switches the device to MJPEG.

EDIT: Another possibility is to tell uv4l to pick a specific video device node created by the another driver (uvcvideo in this case) with the --external-driver=yes and e.g. --device-name=video0 options (you must not specify any --driver option in this case).

Thanks for your answer, understood. I do not plan to use webrtc, only the HTTP streaming server, and this one should be compatible with H.264.

When the driver uv4l-uvc is loaded (so only /dev/video4 is available) v4l2 tools report two indexes 0 and 1 only, corresponding to MJPEG and YUYV streams respectively:

Code: Select all

$ for d in /dev/video* ; do echo $d ; v4l2-ctl --device=$d -D --list-formats  ; echo '===============' ; done
/dev/video4
Driver Info (not using libv4l2):
	Driver name   : uvc
	Card type     : USB Camera
	Bus info      : usb-1-4
	Driver version: 3.2.21
	Capabilities  : 0x05000001
		Video Capture
		Read/Write
		Streaming
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : MJPEG

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUV 4:2:2 (YUYV)

===============

When the orignal driver uvcvideo is loaded instead, there are 4 entries /dev/video0, /dev/video1, /dev/video2 and /dev/video3. Then v4l2 tools report the following, showing full capabilities of the camera: MJPEG, YUYV, and H.264 streams:

Code: Select all

$ for d in /dev/video* ; do echo $d ; v4l2-ctl --device=$d -D --list-formats  ; echo '===============' ; done
/dev/video0
Driver Info (not using libv4l2):
	Driver name   : uvcvideo
	Card type     : H264 USB Camera: USB Camera
	Bus info      : usb-3f980000.usb-1.4
	Driver version: 4.19.40
	Capabilities  : 0x84A00001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : Motion-JPEG

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2

===============
/dev/video1
Driver Info (not using libv4l2):
	Driver name   : uvcvideo
	Card type     : H264 USB Camera: USB Camera
	Bus info      : usb-3f980000.usb-1.4
	Driver version: 4.19.40
	Capabilities  : 0x84A00001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04A00000
		Streaming
		Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
===============
/dev/video2
Driver Info (not using libv4l2):
	Driver name   : uvcvideo
	Card type     : H264 USB Camera: USB Camera
	Bus info      : usb-3f980000.usb-1.4
	Driver version: 4.19.40
	Capabilities  : 0x84A00001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'H264' (compressed)
	Name        : H.264

===============
/dev/video3
Driver Info (not using libv4l2):
	Driver name   : uvcvideo
	Card type     : H264 USB Camera: USB Camera
	Bus info      : usb-3f980000.usb-1.4
	Driver version: 4.19.40
	Capabilities  : 0x84A00001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04A00000
		Streaming
		Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
===============

As you can see /dev/video2 is the H.264 stream I want to use. Then, if I ask for the details of possibilities of this specific one (resolutions and framerates available) I get:

Code: Select all

$ v4l2-ctl -d 2 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'H264' (compressed)
	Name        : H.264
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.067s (15.000 fps)

So I decide to test your last proposal, using the original uvcvideo driver instead of uv4l-uvc, so the beginning of /dev/uv4l/uv4l-uvc.conf is now:

Code: Select all

#driver = uvc
#video_nr = 4
#auto-video_nr = no
external-driver = yes
device-name = video2

I quit and relaunch uv4l, the output is then:

Code: Select all

$ uv4l --device-id 05a3:9422 --config-file=/etc/uv4l/uv4l-uvc.conf
<notice> [core] Trying to load the the Streaming Server plug-in...
<notice> [server] HTTP/HTTPS Streaming & WebRTC Signalling Server v1.1.125 built on Mar  9 2019
<warning> [server] SSL is not enabled for the Streaming Server. Using unsecure HTTP.
<notice> [core] Streaming Server loaded!
<notice> [server] Web Streaming Server listening on port 8090
<notice> [driver] Using video device /dev/video2

which seems fine so far. I go to the HTTP GUI of UV4L server, then to the Control Panel section, and something new is in there, as I always had MJPEG previously:



But alas, the video stream doesn't work, on any URL. When I try to use /stream/video.mjpeg, I get:

Code: Select all

<warning> [server] Cannot set the MJPEG format

which is normal, as there is no MJPEG stream anymore. But when I try to access /stream/video.h264, I get:

Code: Select all

<warning> [server] Inappropriate ioctl for device

Very annoying…

By the way, switching the entry to video0 in the config file to use the MJPEG stream instead, it doesn't work neither, as it outputs the error:

Code: Select all

<warning> [server] Inappropriate ioctl for device

while attempting to access the incompatible H.264 stream produces the expected error:

Code: Select all

<warning> [server] Cannot set the H264 format

all this, although the MJPEG/YUYV streams are correctly identified in the control panel, like the previous configuration:



Any clue?

[flux_capacitor]

Up
@RpiName do you know why ffmpeg can take the native H.264 stream of the camera from /dev/video2 and export it as an .MP4 file with the flag -vcodec copy without any issue, but UV4L HTTP streaming server using same system driver V4L2 uvcvideo fails to pick this raw H.264 stream from /dev/video2 (blank page in a browser) and reports an "inappropriate ioctl for device"?

[RpiName]

Up
@RpiName do you know why ffmpeg can take the native H.264 stream of the camera from /dev/video2 and export it as an .MP4 file with the flag -vcodec copy without any issue, but UV4L HTTP streaming server using same system driver V4L2 uvcvideo fails to pick this raw H.264 stream from /dev/video2 (blank page in a browser) and reports an "inappropriate ioctl for device"?

A raw H264 stream from the camera cannot be shown in a browser "as is". It needs to be encapsulated in WebRTC. However, UV4L does not do this from a H264 source. Try MJPEG/YUV instead, if available from the device driver. I do not think "inappropriate ioctl for device" is necessarily related to this issue.

A way to get a raw H264 stream from the UV4L server with vlc is shown in a tutorial (although it has been probably tested with .the UV4L driver only)