dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7175
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Problem with HDMI VC4 audio (channel remapping)

Mon Feb 12, 2024 2:30 pm

The patch to pcm_iec958.c didn't fix the issue.

I have a basic hdmi analyser (Astro VA1809A) which can show some audio details.
After the underrun, as well as the amplitude for each audio channel no longer matching the audio spoken,
it reports an error on the channel status bits.

e.g. I am seeing initially:
0x4 0x82 0x0 0x2 0x2
which I decode to the plausible:

Code: Select all

consumer                      : 0
non-pcm                       : 0
copyright                     : 1
non-linear                    : 0
mode                          : 0
category code                 : Digital/digital converters and signal processing products
L bit                         : 1
source num                    : 0
channel num                   : 0
sampling frequency            : 48000.0 kHz
clock accuracy                : 0
sample word length            : 16 bits
original sampling frequency   : Not indicated
After an underrun:
0x8 0x4 0x1 0x4 0x4
which decodes badly:

Code: Select all

consumer                      : 0
non-pcm                       : 0
copyright                     : 0
non-linear                    : 1
mode                          : 0
category code                 : Broadcast reception of digitally encoded audio signals with or without video signals
L bit                         : 0
source num                    : 1
channel num                   : 0
sampling frequency            : 22050.0 kHz
clock accuracy                : 0
sample word length            : 18 bits
original sampling frequency   : Not indicated
Somewhat interestingly, the wrong "0x8 0x4 0x1 0x4 0x4" seems quite consistent after an underrun -
I might have expected to get a variety of incorrect sequences, depending on exactly where in the buffer the underrun occurred.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7175
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Problem with HDMI VC4 audio (channel remapping)

Mon Feb 12, 2024 2:34 pm

6by9 wrote:
Mon Feb 12, 2024 2:27 pm
It's not being set in https://github.com/raspberrypi/linux/bl ... mi.c#L2421, but potentially could be. It is set by the firmware driver.
It is set here.

phofman
Posts: 86
Joined: Mon Oct 07, 2019 1:37 pm

Re: Problem with HDMI VC4 audio (channel remapping)

Mon Feb 12, 2024 2:49 pm

It's interesting that IIUC the analyzer shows messed-up parts in the HDMI packets which IMO cannot be influenced by the alsa stream entering the driver.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 15860
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Problem with HDMI VC4 audio (channel remapping)

Mon Feb 12, 2024 2:50 pm

dom wrote:
Mon Feb 12, 2024 2:34 pm
6by9 wrote:
Mon Feb 12, 2024 2:27 pm
It's not being set in https://github.com/raspberrypi/linux/bl ... mi.c#L2421, but potentially could be. It is set by the firmware driver.
It is set here.
Doh, finger troubles on my part :( :(
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7175
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Problem with HDMI VC4 audio (channel remapping)

Mon Feb 12, 2024 2:52 pm

phofman wrote:
Mon Feb 12, 2024 2:49 pm
It's interesting that IIUC the analyzer shows messed-up parts in the HDMI packets which IMO cannot be influenced by the alsa stream entering the driver.
I'm not quite following. The csb reported by analyser is decoded directly from the 32-bit iec958 words generated by iec958_subframe:

Code: Select all

	/* set IEC status bits (up to 192 bits) */
	if (iec->status[byte] & mask)
		data |= 0x40000000;
Am I misunderstanding something here?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 15860
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Problem with HDMI VC4 audio (channel remapping)

Mon Feb 12, 2024 2:55 pm

dom wrote:
Mon Feb 12, 2024 2:30 pm
The patch to pcm_iec958.c didn't fix the issue.

I have a basic hdmi analyser (Astro VA1809A) which can show some audio details.
After the underrun, as well as the amplitude for each audio channel no longer matching the audio spoken,
it reports an error on the channel status bits.

e.g. I am seeing initially:
0x4 0x82 0x0 0x2 0x2
which I decode to the plausible:
...
After an underrun:
0x8 0x4 0x1 0x4 0x4
which decodes badly:
You've probably noticed, but that's a one bit shift left in each byte. (Overflow bit on 0x82 getting shifted to the bottom of the next byte)
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

phofman
Posts: 86
Joined: Mon Oct 07, 2019 1:37 pm

Re: Problem with HDMI VC4 audio (channel remapping)

Mon Feb 12, 2024 3:11 pm

I see, those are values from the IEC958 preamble, not bits added by the HDMI transmitter to packets in data islands. My fault, sorry.

Then some byte misalignment at xrun really occurs somewhere in the stream.

Pehaps modifiying that .asoundrc to output to the actual HDMI device could allow capturing the stream entering the driver:

Code: Select all

pcm.testiec {
  type iec958
  slave {
    pcm out
    format IEC958_SUBFRAME_LE
  }
}

pcm.out {
  type file
  file "/tmp/out.raw"
  format "raw"
  slave {
    pcm "hw:vc4hdmi0"
  }
}
The device testiec could be used in the speaker-test check. But finding the "breaking point" will be tedious. Maybe playing zeros with aplay /dev/zero instead of speaker-test would provide better output for analysis whether the "breaking point" comes from userspace or is introduced somewhere further downstream.

But since it was identified as just one bit shift, it's very unlikely from userspace.

phofman
Posts: 86
Joined: Mon Oct 07, 2019 1:37 pm

Re: Problem with HDMI VC4 audio (channel remapping)

Mon Feb 12, 2024 3:31 pm

IIUC one bit shift in the IEC958 status corresponds to one stereo frame shift in the stream.

Perhaps aligning the period size/write granularity) in your player to the 192-frames block (192 x 8ch x 4bytes = 6144 bytes), which has not been tested yet (a different alignment was tested).

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7175
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Problem with HDMI VC4 audio (channel remapping)

Mon Feb 12, 2024 3:59 pm

phofman wrote:
Mon Feb 12, 2024 3:31 pm
IIUC one bit shift in the IEC958 status corresponds to one stereo frame shift in the stream.

Perhaps aligning the period size/write granularity) in your player to the 192-frames block (192 x 8ch x 4bytes = 6144 bytes), which has not been tested yet (a different alignment was tested).
Like this?

Code: Select all

$ speaker-test -D hdmi:vc4hdmi0 -c 8 -p 128000

speaker-test 1.2.8

Playback device is hdmi:vc4hdmi0
Stream parameters are 48000Hz, S16_LE, 8 channels
Using 16 octaves of pink noise
Rate set to 48000Hz (requested 48000Hz)
Buffer size range from 2 to 16384
Period size range from 1 to 8192
Requested period time 128000 us
Periods = 4
was set period_size = 6144
was set buffer_size = 12288
 0 - Front Left
It still goes wrong on an underrun.

phofman
Posts: 86
Joined: Mon Oct 07, 2019 1:37 pm

Re: Problem with HDMI VC4 audio (channel remapping)

Tue Feb 13, 2024 6:41 am

Like this?
Looks OK, just please check the actual period size of the HW device in /proc/asound/vc4hdmi0/pcm0p/sub0/hw_params, to make sure it did not get changed along the chain.

phofman
Posts: 86
Joined: Mon Oct 07, 2019 1:37 pm

Re: Problem with HDMI VC4 audio (channel remapping)

Tue Feb 13, 2024 7:46 am

Maybe the user-space xrun behavior could be checked by tee-ing the stream to file and checking xrun behavior:

.asoundrc:

Code: Select all

pcm.testiec {
  type iec958
  slave {
    pcm out
    format IEC958_SUBFRAME_LE
  }
}

pcm.out {
  type file
  file "/tmp/out.raw"
  format "raw"
  slave {
    pcm "hw:vc4hdmi0"
  }
}
Playback generates 3-bytes format (S24_3LE) where the lowest bytes of samples hold corresponding channel IDs:

Code: Select all

(while true; do for CH in '\x01' '\x02' '\x03' '\x04' '\x05' '\x06' '\x07' '\x08' ; do echo -n -e $CH; echo -n -e '\x00\x00'; done; done ) | aplay -v -c 8 -f S24_3LE  -r 48000  -D testiec

Now the stored /tmp/out.raw should be visibly correct at any line, even after xruns induced by nicing aplay:

Code: Select all

xxd -c 32 -g 1 /tmp/out.raw | head
00000000: 18 00 00 80 24 00 00 80 32 00 00 00 44 00 00 80 52 00 00 00 64 00 00 00 72 00 00 80 84 00 00 80  ....$...2...D...R...d...r.......
00000020: 12 00 00 80 24 00 00 80 32 00 00 00 44 00 00 80 52 00 00 00 64 00 00 00 72 00 00 80 84 00 00 80  ....$...2...D...R...d...r.......
00000040: 12 00 00 80 24 00 00 80 32 00 00 c0 44 00 00 40 52 00 00 00 64 00 00 00 72 00 00 80 84 00 00 80  [email protected].......
00000060: 12 00 00 80 24 00 00 80 32 00 00 00 44 00 00 80 52 00 00 00 64 00 00 00 72 00 00 40 84 00 00 40  ....$...2...D...R...d...r..@...@
00000080: 12 00 00 80 24 00 00 80 32 00 00 00 44 00 00 80 52 00 00 00 64 00 00 00 72 00 00 80 84 00 00 80  ....$...2...D...R...d...r.......
000000a0: 12 00 00 80 24 00 00 80 32 00 00 00 44 00 00 80 52 00 00 00 64 00 00 00 72 00 00 80 84 00 00 80  ....$...2...D...R...d...r.......
000000c0: 12 00 00 80 24 00 00 80 32 00 00 c0 44 00 00 40 52 00 00 00 64 00 00 00 72 00 00 80 84 00 00 80  [email protected].......
000000e0: 12 00 00 80 24 00 00 80 32 00 00 00 44 00 00 80 52 00 00 00 64 00 00 00 72 00 00 80 84 00 00 80  ....$...2...D...R...d...r.......
00000100: 12 00 00 80 24 00 00 80 32 00 00 00 44 00 00 80 52 00 00 00 64 00 00 00 72 00 00 80 84 00 00 80  ....$...2...D...R...d...r.......
00000120: 12 00 00 80 24 00 00 80 32 00 00 00 44 00 00 80 52 00 00 00 64 00 00 00 72 00 00 80 84 00 00 80  ....$...2...D...R...d...r.......
A sample shift should be easily identifiable in this aligned dump, IMO

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7175
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Problem with HDMI VC4 audio (channel remapping)

Tue Feb 13, 2024 4:43 pm

phofman wrote:
Tue Feb 13, 2024 6:41 am
Looks OK, just please check the actual period size of the HW device in /proc/asound/vc4hdmi0/pcm0p/sub0/hw_params, to make sure it did not get changed along the chain.

Code: Select all

$ cat /proc/asound/vc4hdmi0/pcm0p/sub0/hw_params
access: MMAP_INTERLEAVED
format: IEC958_SUBFRAME_LE
subformat: STD
channels: 8
rate: 48000 (48000/1)
period_size: 6144
buffer_size: 12288

oomzay
Posts: 6
Joined: Sun Jan 28, 2024 1:49 pm

Re: Problem with HDMI VC4 audio (channel remapping)

Tue Feb 13, 2024 7:35 pm

To the untrained eye there seems to be a bit of an architectural car-crash going on here...

Why are the HDMI audio transport (N x IEC 60958) details being exposed outside the HDMI driver at all?

Why not deal with this encoding (including x-run handling) within the HDMI driver?

It seems to me that:

1. The HDMI driver has to be IEC 60958 aware anyway if it is to deal with x-runs in any reasonable and safe way.
2. There is no multi-channel IEC 60958 whatever as far as I can see, only 2-channel (I just scanned the 237 page spec.)
3. Relying on some ad-hoc IEC 60958 channel interleaving & blocking "convention" at the interface seems inherently fragile. (is it documented at all?)
4. ...and it assumes the IEC 60958 plug-thing correctly encodes multiple 2-channel IEC 60958 streams - which it almost certainly does not!
5. I have no idea why HDMI spec'd IEC 60958 in the first place as there is no application match at all (i.e. synchronous, stereo only, sample-clocking) and while that is another story - it enforces my rather strong feeling that the audio transport encoding should be hidden away completely with all the other dirty HDMI secrets.

Best wishes.

oomzay
Posts: 6
Joined: Sun Jan 28, 2024 1:49 pm

Re: Problem with HDMI VC4 audio (channel remapping)

Thu Feb 15, 2024 7:36 pm

FWIW: I improved the test script by explicitly pausing the speaker-test process rather than stressing the CPU:

Code: Select all

#!/bin/env bash

# Script to provoke hdmi audio channel swapping bug
#
# Requires:
#    speaker-test

# Arrange to kill speakertest process on CTL-C.

trap "killall background" EXIT

# Run speaker-test in the background

speaker-test -D hdmi:vc4hdmi1,0 -c 8 -r 48000 -p 85000 &

# Capture the background process id so can pause it

bgpid=$!

# Pause the speaker-test process every 10 seconds to force x-run
# Too often = not enough time to be sure of the swap.
# Too seldom = it can take ages for a swap to happen.

while true
do
  sleep 10
  echo Forcing x-run...
  kill -STOP $bgpid
  sleep 0.5
  kill -CONT $bgpid
done

Return to “Interfacing (DSI, CSI, I2C, etc.)”