User avatar
h.larsen
Posts: 12
Joined: Mon Aug 23, 2021 8:09 pm

h264 hardware accelerator - how to install for Bullseye/64b

Sun Nov 20, 2022 9:55 pm

I want to use the hardware encoder for h264 in the PI4 in conjunction with gstreamer and raspberry OS bullseye 64bit

I used to use v4l2h264enc but this I cannot find nor figure out how to install.
I managed (somehow) to install a module: "avenc_h264_omx" as shown below.
Is this the wrapper for the hardware h264 accelerator in rpi? How to get use of the h264 hardware accelerator?

gst-inspect-1.0 | grep =>

Code: Select all

$ gst-inspect-1.0 | grep h264
libav:  avenc_h264_omx: libav OpenMAX IL H.264 video encoder encoder
libav:  avdec_h264: libav H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 decoder
videoparsersbad:  h264parse: H.264 parser
rtp:  rtph264depay: RTP H264 depayloader
rtp:  rtph264pay: RTP H264 payloader
uvch264:  uvch264mjpgdemux: UVC H264 MJPG Demuxer
uvch264:  uvch264src: UVC H264 Source
uvch264:  uvch264deviceprovider (GstDeviceProviderFactory)
typefindfunctions: video/x-h264: h264, x264, 264

pidd
Posts: 4153
Joined: Fri May 29, 2020 8:29 pm
Location: Wirral, UK

Re: h264 hardware accelerator - how to install for Bullseye/64b

Mon Nov 21, 2022 3:09 am

I believe you will already have gstreamer1.0-plugins-ugly installed in which case use x264enc instead of v4l2h264enc

Ignore.
Last edited by pidd on Mon Nov 21, 2022 3:16 pm, edited 1 time in total.

redvli
Posts: 194
Joined: Thu Sep 03, 2020 8:09 am

Re: h264 hardware accelerator - how to install for Bullseye/64b

Mon Nov 21, 2022 9:13 am

OMX is not available anymore in RasPiOS Bullseye. Use a V4L M2M based API/method to use the H264 HW codec in the chip. For gstreamer I don't know, but for ffmpeg decoding or encoding use/force the codec: h264_v4l2m2m

See:
root@raspi:/dev/v4l/by-path# ffmpeg -hide_banner -codecs | grep h264
DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m ) (encoders: libx264 libx264rgb h264_omx h264_v4l2m2m h264_vaapi )

Make sure RasPiOS is updated to 'latest', e.g. kernel 5.15.76-v8+
With older kernels and/or firmware you might experience issues, even lockups, as this method is quite new compared to h264_vaapi on Intel CPU's for example. But I see no problems in Jellyfin for example, which uses encoding via h264_v4l2m2m for older/incompatible codecs.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13531
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: h264 hardware accelerator - how to install for Bullseye/64b

Mon Nov 21, 2022 12:02 pm

GStreamer will use v4l2h264enc for hardware video encode, which is part of gstreamer1.0-plugins-good.

x264enc is software encode.

Anything with avenc in is a wrapper for FFmpeg.
Anything mentioning "omx" or "il" is referencing OpenMax IL, which is deprecated, and not available on a 64bit OS.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
h.larsen
Posts: 12
Joined: Mon Aug 23, 2021 8:09 pm

Re: h264 hardware accelerator - how to install for Bullseye/64b

Mon Nov 21, 2022 10:58 pm

Thank you all so much.
Yes the kernel is the latest - 5.15.76-v8+ (see listing below)
gstreamer1.0-plugins-good (1.18.4-2) is the latest according to apt install but not the latest from source installation (1.20). (see listing below) But as shown in the gstinspect-1.0 listing below it did not install v4l2h264enc. Maybe I will try install from sources.
The x264enc is installed and works with gstreamer but as mentioned it is software encoder. (see listing below)
The ffmpeg encoders gives exactly the same listing as @redvli lists so h264_v4l2m2m is available for ffmpeg.
I am using gstreamer though and would very much like not to switch. Any chance?
Any advice is appreciated.

Relevant listings below.

Code: Select all

heel@pi103:~ $ uname -a
Linux pi103 5.15.76-v8+ #1597 SMP PREEMPT Fri Nov 4 12:16:41 GMT 2022 aarch64 GNU/Linux

Code: Select all

heel@pi103:~ $ sudo apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio            Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
gstreamer1.0-libav is already the newest version (1.18.4-3).
gstreamer1.0-plugins-bad is already the newest version (1.18.4-3).
libgstreamer-plugins-bad1.0-dev is already the newest version (1.18.4-3).
gstreamer1.0-alsa is already the newest version (1.18.4-2).
gstreamer1.0-gl is already the newest version (1.18.4-2).
gstreamer1.0-plugins-base is already the newest version (1.18.4-2).
gstreamer1.0-x is already the newest version (1.18.4-2).
libgstreamer-plugins-base1.0-dev is already the newest version (1.18.4-2).
gstreamer1.0-plugins-ugly is already the newest version (1.18.4-2).
gstreamer1.0-tools is already the newest version (1.18.4-2.1).
libgstreamer1.0-dev is already the newest version (1.18.4-2.1).
gstreamer1.0-gtk3 is already the newest version (1.18.4-2+deb11u1+rpt1).
gstreamer1.0-plugins-good is already the newest version (1.18.4-2+deb11u1+rpt1).
gstreamer1.0-pulseaudio is already the newest version (1.18.4-2+deb11u1+rpt1).
gstreamer1.0-qt5 is already the newest version (1.18.4-2+deb11u1+rpt1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Code: Select all

heel@pi103:~ $ gst-inspect-1.0 | grep 264
libav:  avmux_ipod: libav iPod H.264 MP4 (MPEG-4 Part 14) muxer
libav:  avdec_h264: libav H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 decoder
libav:  avenc_h264_omx: libav OpenMAX IL H.264 video encoder encoder
videoparsersbad:  h264parse: H.264 parser
rtp:  rtph264pay: RTP H264 payloader
rtp:  rtph264depay: RTP H264 depayloader
uvch264:  uvch264deviceprovider (GstDeviceProviderFactory)
uvch264:  uvch264src: UVC H264 Source
uvch264:  uvch264mjpgdemux: UVC H264 MJPG Demuxer
typefindfunctions: video/x-h264: h264, x264, 264
x264:  x264enc: x264enc

Code: Select all

ffmpeg -hide_banner -codecs | grep h264
 DEV.LS h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m ) (encoders: libx264 libx264rgb h264_omx h264_v4l2m2m h264_vaapi )

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13531
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: h264 hardware accelerator - how to install for Bullseye/64b

Tue Nov 22, 2022 7:31 am

Have you edited config.txt to set gpu_mem to 16? That disables the codec hardware acceleration as they then have no memory to work with. GStreamer actually checks for the capabilities of the system before instantiating components, whereas FFmpeg just lists what has been compiled.

Test with a clean install of Raspberry Pi OS.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
h.larsen
Posts: 12
Joined: Mon Aug 23, 2021 8:09 pm

Re: h264 hardware accelerator - how to install for Bullseye/64b

Tue Nov 22, 2022 9:08 pm

Wow - 6by9 I am _very_ impressed. You nailed it!
gpu_mem=16
was the problem.
Now at gpu_mem=160 it loads the v4l2h64enc/dec as it should.
Thank you so much. Lack a thumb's up button though :D

Code: Select all

heel@pi103:~ $ gst-inspect-1.0 | grep 264
video4linux2:  v4l2h264enc: V4L2 H.264 Encoder
video4linux2:  v4l2h264dec: V4L2 H264 Decoder
libav:  avmux_ipod: libav iPod H.264 MP4 (MPEG-4 Part 14) muxer
libav:  avdec_h264: libav H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 decoder
libav:  avenc_h264_omx: libav OpenMAX IL H.264 video encoder encoder
videoparsersbad:  h264parse: H.264 parser
rtp:  rtph264pay: RTP H264 payloader
rtp:  rtph264depay: RTP H264 depayloader
uvch264:  uvch264deviceprovider (GstDeviceProviderFactory)
uvch264:  uvch264src: UVC H264 Source
uvch264:  uvch264mjpgdemux: UVC H264 MJPG Demuxer
typefindfunctions: video/x-h264: h264, x264, 264
x264:  x264enc: x264enc

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 31811
Joined: Sat Jul 30, 2011 7:41 pm

Re: h264 hardware accelerator - how to install for Bullseye/64b

Wed Nov 23, 2022 11:19 am

h.larsen wrote:
Tue Nov 22, 2022 9:08 pm
Wow - 6by9 I am _very_ impressed. You nailed it!
gpu_mem=16
was the problem.
Now at gpu_mem=160 it loads the v4l2h64enc/dec as it should.
Thank you so much. Lack a thumb's up button though :D

Code: Select all

heel@pi103:~ $ gst-inspect-1.0 | grep 264
video4linux2:  v4l2h264enc: V4L2 H.264 Encoder
video4linux2:  v4l2h264dec: V4L2 H264 Decoder
libav:  avmux_ipod: libav iPod H.264 MP4 (MPEG-4 Part 14) muxer
libav:  avdec_h264: libav H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 decoder
libav:  avenc_h264_omx: libav OpenMAX IL H.264 video encoder encoder
videoparsersbad:  h264parse: H.264 parser
rtp:  rtph264pay: RTP H264 payloader
rtp:  rtph264depay: RTP H264 depayloader
uvch264:  uvch264deviceprovider (GstDeviceProviderFactory)
uvch264:  uvch264src: UVC H264 Source
uvch264:  uvch264mjpgdemux: UVC H264 MJPG Demuxer
typefindfunctions: video/x-h264: h264, x264, 264
x264:  x264enc: x264enc
Can I ask why you changed the gpu_mem setting in the first place? We recommend it is left along unless you have a compelling reason to change it - the default should work fine for the majority of use cases.
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

User avatar
jahboater
Posts: 8355
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: h264 hardware accelerator - how to install for Bullseye/64b

Wed Nov 23, 2022 1:47 pm

jamesh wrote:
Wed Nov 23, 2022 11:19 am
Can I ask why you changed the gpu_mem setting in the first place? We recommend it is left along unless you have a compelling reason to change it - the default should work fine for the majority of use cases.
For interest, I always set GPU mem to 16MB when headless on Pi OS Lite, and it always works fine.
If left to the default, it gets set to 64MB on my Pi1 256MB for example, and 48MB is too much to lose!

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 31811
Joined: Sat Jul 30, 2011 7:41 pm

Re: h264 hardware accelerator - how to install for Bullseye/64b

Wed Nov 23, 2022 2:44 pm

jahboater wrote:
Wed Nov 23, 2022 1:47 pm
jamesh wrote:
Wed Nov 23, 2022 11:19 am
Can I ask why you changed the gpu_mem setting in the first place? We recommend it is left along unless you have a compelling reason to change it - the default should work fine for the majority of use cases.
For interest, I always set GPU mem to 16MB when headless on Pi OS Lite, and it always works fine.
If left to the default, it gets set to 64MB on my Pi1 256MB for example, and 48MB is too much to lose!
You know what you are doing. For most people leaving it at the default will be fine, and prevent issues such as the one under discussion.
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

User avatar
h.larsen
Posts: 12
Joined: Mon Aug 23, 2021 8:09 pm

Re: h264 hardware accelerator - how to install for Bullseye/64b

Wed Nov 23, 2022 9:57 pm

The PiB4 is in headless mode so I wanted to conserve some memory space and put the gpu_mem=16.
So this was not a good choice.
I have searched the log files /var/log/kernel, /var/log/messages, /var/log/syslog
and find this line - in for instance syslog - when the encoder is loaded properly when given sufficient ram:

Code: Select all

Nov 23 21:37:34 pi103 kernel: [    6.712524] bcm2835-codec bcm2835-codec: Loaded V4L2 encode_image
I did not find any message that bcm2835-codec cannot load due to lack of resources. Is that correct. If so, then maybe there ought to be a log record.
In any case I am thankful to have the issue solved.

redvli
Posts: 194
Joined: Thu Sep 03, 2020 8:09 am

Re: h264 hardware accelerator - how to install for Bullseye/64b

Thu Nov 24, 2022 8:51 am

jamesh wrote:
Wed Nov 23, 2022 2:44 pm
jahboater wrote:
Wed Nov 23, 2022 1:47 pm
jamesh wrote:
Wed Nov 23, 2022 11:19 am
Can I ask why you changed the gpu_mem setting in the first place? We recommend it is left along unless you have a compelling reason to change it - the default should work fine for the majority of use cases.
For interest, I always set GPU mem to 16MB when headless on Pi OS Lite, and it always works fine.
If left to the default, it gets set to 64MB on my Pi1 256MB for example, and 48MB is too much to lose!
You know what you are doing. For most people leaving it at the default will be fine, and prevent issues such as the one under discussion.
I remember SuSE in the past also had gpu_mem=16 as default for their OpenSuSE OS flavors. Maybe to gain some memory, maybe because they were and are very strict on (software) patents and licensing. There is no h264 in their repositories. So in case a lawyer of a patent troll organization wants to try something, they are safe.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13531
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: h264 hardware accelerator - how to install for Bullseye/64b

Thu Nov 24, 2022 9:02 am

h.larsen wrote:
Wed Nov 23, 2022 9:57 pm
The PiB4 is in headless mode so I wanted to conserve some memory space and put the gpu_mem=16.
So this was not a good choice.
I have searched the log files /var/log/kernel, /var/log/messages, /var/log/syslog
and find this line - in for instance syslog - when the encoder is loaded properly when given sufficient ram:

Code: Select all

Nov 23 21:37:34 pi103 kernel: [    6.712524] bcm2835-codec bcm2835-codec: Loaded V4L2 encode_image
I did not find any message that bcm2835-codec cannot load due to lack of resources. Is that correct. If so, then maybe there ought to be a log record.
In any case I am thankful to have the issue solved.
IIRC you will get messages such as "Failed to initialise VCHI instance (status=N)" and "failed to initialise VCHI instance (ret=N)" logged multiple times as the camera, codecs, ISP, and vcsm-cma all fail to load. Perhaps not totally obvious, but any errors should make you look more closely.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Graphics programming”