HarutakaMatsumoto
Posts: 2
Joined: Tue Sep 22, 2020 9:28 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Wed Sep 23, 2020 7:31 am

Thanks.
I'm sorry for not reading latest posts.

HotJollof
Posts: 1
Joined: Tue Dec 10, 2019 9:13 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Fri Sep 25, 2020 2:05 am

Thank you @6by9 for the great information, it has greatly helped me on a project I have been working on where I am taking the feed from a regular camcorder and using araspberrypi4 and a google edgetpu to perform facial recognition on the video feed.

I find myself continuously having to run these three commands every time I need to run my program, and I was wondering if these could be automated in python with the os or subprocess module.

When I run them in sequence like below I get errors like "VIDIOC_QUERY_DV_TIMINGS: failed: Link has been severed"

subprocess.run(['v4l2-ctl', '--set-edid=file=1080P50EDID.txt', '--fix-edid-checksums'], capture_output=True)
subprocess.run(['v4l2-ctl', '--query-dv-timings'])
subprocess.run(['./yavta --capture=10 -n 3 --encode-to=file.h264 -f UYVY -m -T /dev/video0'], shell=True)

However if I only run only one like below, I get the expected results.

subprocess.run(['v4l2-ctl', '--query-dv-timings'])

Any insights into what is causing the issue?
6by9 wrote:
Tue Jul 10, 2018 2:27 pm
*Edited with updated instructions on setting timings*

Finally got there. The drivers and dtoverlays are all merged and released via rpi-update!
Things aren't quite as clean as they could be on the device tree front, but that clean up will come later.

edit First please edit /boot/cmdline.txt and add or edit the "cma=" entry to read "cma=32M", or potentially larger if you want to allocate lots of video buffers. 32M should be sufficient for 3 1080P RGB frames.

After updating, adding

Code: Select all

dtoverlay=tc358743
to /boot/config.txt should be all that is needed on a Pi 3B or 3B+, and you should get a /dev/video0 node (or whatever is the lowest unallocated V4L2 device number).
For a 0, 0W, B+, or 2B, add

Code: Select all

dtoverlay=tc358743,i2c_pins_28_29=1
On a CM or CM3, you need to decide which pins you're connecting up, and then choose either the same as the 3B/3B+ if 44&45, or the 0/0W/B+/2B if using 28&29. Yes you could use 0&1, but I haven't added parameters for that. You can also add "4lane=1" if using the CAM1 connector and have a B102 to be able to achieve 1080P60 (1080P50 UYVY is the maximum available on 2 lanes).
I have not added support for the original A/B as there are too many variations for this round.

As already discussed in this thread, you need to provide an EDID so that the HDMI source knows what resolutions and frame rates are supported. There are example EDID files in https://github.com/6by9/CSI2_device_config for 1080P50 and 1080P60. There is no such thing as a correct EDID for all use cases, so you may want to edit it (not trivial, but there are various programs to help out).
Load the EDID with a command such as

Code: Select all

v4l2-ctl --set-edid=file=1080P50EDID.txt --fix-edid-checksums 
A slight quirk of V4L2 is that whilst the source device will detect the incoming video standard, it does not select it.

Code: Select all

v4l2-ctl --query-dv-timings
will display the detected values. There isn't an easy way to then immediately set the detected mode, and the output resolution is set based on that rather than being able to set the resolution. (it's a trivial app to write - I may see if I can find 5 mins. It'd actually be a useful addition to v4l2-ctl, so I'll see if I can get that upstreamed too).
Run

Code: Select all

v4l2-ctl --set-dv-bt-timings query
and it'll set the current timings to match those that are detected on the input.

The default output format is RGB888 (24bpp), but the device also supports UYVY (YUV422 16bpp). The video_encode component should also now support UYVY, so if you're lucky then the gst-omx plugins should just work (I haven't tried that bit). It still copies the data around a few times, so it's not got all the optimisations possible in there. As noted above, the maximum data rate on 2 lanes is sufficient for 1080P50 as UYVY. In RGB888 mode 1080P30 is likely to be the maximum.
For capture there is also my fork of the yavta test app.

Code: Select all

git clone https://github.com/6by9/yavta
make
./yavta --capture=1000 -n 3 --encode-to=file.h264 -f UYVY -m -T /dev/video0
should encode 100 frames (I'm not sure that counter works) to file.h264.

For audio, add

Code: Select all

dtoverlay=tc358743-audio
and wire it up as LRCK/WFS to GPIO 19, BCK/SCK to GPIO 18, and DATA/SD to GPIO 20. (those are 7, 6 and 5 respectively on the B101/B102 boards).
That should add an audio capture device. Capture sample rate must be read from V4L2. V4L2 also provides an extended control to denote whether audio is present or not. From the command line, "v4l2-ctl --list-ctrls" will print out the respective values (processing that output is not the right way of to do things! Use the proper V4L2 calls). The audio device does work correctly with GStreamer's alsasrc component.

There are also overlays for the ADV7282M and OV5647 (V1.3 Pi camera), but I won't go into those here.

I will be supporting this driver as time permits, so please do report issues. I'd suggest you start a new thread though - this one has got a little on the long side to be readable.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12379
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Fri Sep 25, 2020 11:25 am

HotJollof wrote:
Fri Sep 25, 2020 2:05 am
Thank you @6by9 for the great information, it has greatly helped me on a project I have been working on where I am taking the feed from a regular camcorder and using araspberrypi4 and a google edgetpu to perform facial recognition on the video feed.

I find myself continuously having to run these three commands every time I need to run my program, and I was wondering if these could be automated in python with the os or subprocess module.

When I run them in sequence like below I get errors like "VIDIOC_QUERY_DV_TIMINGS: failed: Link has been severed"

subprocess.run(['v4l2-ctl', '--set-edid=file=1080P50EDID.txt', '--fix-edid-checksums'], capture_output=True)
subprocess.run(['v4l2-ctl', '--query-dv-timings'])
subprocess.run(['./yavta --capture=10 -n 3 --encode-to=file.h264 -f UYVY -m -T /dev/video0'], shell=True)

However if I only run only one like below, I get the expected results.

subprocess.run(['v4l2-ctl', '--query-dv-timings'])

Any insights into what is causing the issue?
Timing.
Setting the EDID causes the bridge chip to deassert and reassert the hotplug line to make the source (your camcorder) reread the EDID. Only once the source has reread the EDID and determined what resolution it should be in can it reconfigure and enable its HDMI output.

Unless you're changing the EDID between runs, you only need to do it once per boot. You really need to leave 1-2 seconds after setting the EDID before expecting there to be any valid input.

Note that "v4l2-ctl --query-dv-timings" only tells you what the timings are, it doesn't set them. yavta has code in that will set them, but you can also do it manually via "v4l2-ctl --set-dv-bt-timings query"
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamiebasil
Posts: 2
Joined: Fri Nov 06, 2020 3:28 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Wed Nov 11, 2020 1:26 am

Tanner1pl wrote:
Sun Aug 18, 2019 7:28 pm
Hello, I have an issue which might be a bug - tested on both Auvidea B101/3b+ and B102/Zero with two blackmagic camers - Pocket 4K and Micro Cinema Camera. I assume hardware works good, as other video sources are captured properly.

My issue - video captured has broken colors - like way too much green and pink.
Colors are fine when I use different camera or any other video source - so B101 and B102 are fine. Also, when I connect these blackmagic cameras to TV - they works good. Also many hdmi cables tested.

Looks more like TC358743 doesn't recognize format/config of video coming from blackmagic - it sends clear 10bit 1080p@25fps 4.2.2 video which can be recorded using external recorder. Looks very similar (if not exactly) like issue with YCrCb vs RGBFull pixel formatting. Has anyone faced this issue and found solution? :)

Code: Select all

raspivid -t 5000 -w 1920 -h 1080 -fps 25 -o test.h264
Zrzut-ekranu-2019-08-18-o-21.12.59.jpg


While image coming out from camera is fine, like here on preview LCD or when I connect monitor/TV to camera.
68800270_360682631267652_8322857960573239296_n.jpg
Hi - I'm having the same "pink image" issue connecting a blackmagic micro cinema camera to a pi 3B via HDMI -> CSI-2 bridge that contains the TC358743 chip. Any success finding a way to get it to work? It seems like the problem is that the video from the blackmagic camera is 10-bit color depth and the pi only reads 8-bit data.

Any ideas on how to down convert the video signal to 8-bits in software or in hardware?

Thanks so much,
Jamie

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12379
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Wed Nov 11, 2020 7:46 am

jamiebasil wrote:
Wed Nov 11, 2020 1:26 am
Tanner1pl wrote:
Sun Aug 18, 2019 7:28 pm
Hello, I have an issue which might be a bug - tested on both Auvidea B101/3b+ and B102/Zero with two blackmagic camers - Pocket 4K and Micro Cinema Camera. I assume hardware works good, as other video sources are captured properly.

My issue - video captured has broken colors - like way too much green and pink.
Colors are fine when I use different camera or any other video source - so B101 and B102 are fine. Also, when I connect these blackmagic cameras to TV - they works good. Also many hdmi cables tested.

Looks more like TC358743 doesn't recognize format/config of video coming from blackmagic - it sends clear 10bit 1080p@25fps 4.2.2 video which can be recorded using external recorder. Looks very similar (if not exactly) like issue with YCrCb vs RGBFull pixel formatting. Has anyone faced this issue and found solution? :)

Code: Select all

raspivid -t 5000 -w 1920 -h 1080 -fps 25 -o test.h264
Zrzut-ekranu-2019-08-18-o-21.12.59.jpg


While image coming out from camera is fine, like here on preview LCD or when I connect monitor/TV to camera.
68800270_360682631267652_8322857960573239296_n.jpg
Hi - I'm having the same "pink image" issue connecting a blackmagic micro cinema camera to a pi 3B via HDMI -> CSI-2 bridge that contains the TC358743 chip. Any success finding a way to get it to work? It seems like the problem is that the video from the blackmagic camera is 10-bit color depth and the pi only reads 8-bit data.

Any ideas on how to down convert the video signal to 8-bits in software or in hardware?
You've quoted against a post referring to using raspivid. Any use of raspivid with a TC358743 is unsupported.
The kernel driver does have some code in there which appears to set up the colour conversion matrix sensibly, so I would have reasonable hopes that it can cope. But also double check your EDID to see whether it advertises support for YCbCr422 or not. Ideally you want it not to.

As advised on the earlier posts, a Kramer presentation switch should do the job for you.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

upanie
Posts: 2
Joined: Tue Nov 24, 2020 10:49 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Tue Nov 24, 2020 11:29 pm

Hi,

I'm trying to make B101 from Auvidea work on RPI4/4G under Ubuntu.
I tried Ubuntu 18.04 and 20.10 with the same results/problems.

Case no. 1
On fresh install of Ubuntu after enabling camera (start_x1=1 in /boot/firmware/config.txt) it sort of works.
I can run ustreamer or get the video using picamera and it works fine.
Fine means that the video content is visible and is perfectly fine :)
But I can't control the b101 module. I can't set EDID, check timings, nothing.
By default, without EDID, B101 works in 720p and the HDMI source can't see it as a HDMI "sink".
No resolution negotiation, nothing. So for most cases it's useless. But at least video works in one configuration.

Case no. 2
So following @6by9 instructions I added overlay to config.txt file and after that /dev/video0 and /dev/video1 vanished.
I found that Ubuntu doesn't come with tc358743 kernel module. So I built it and after loading it video device came back.
And now I can control the module. I can set an EDID, check and set timing, etc.
Any HDMI source can now see a valid video receiver (display), resolution negotiation works fine. Everything looks fine but...
I can't get any video from B101 module.
@6by9 I found your comment here https://github.com/raspberrypi/linux/is ... -624565359
It looks exactly like my case. Do you remember what was the cause?

After making the same steps on Raspbian everything works fine. I can control it and get the video content.
So it has nothing to do with the hardware.

At first I was trying to make it work on Ubuntu 18.04 but then I tried 20.10 with newer kernel and newer firmware but I get the same results.

Now few attachments for Ubuntu 20.10:

Code: Select all

[pi4]
max_framebuffers=2

[all]
arm_64bit=1
kernel=vmlinuz
cmdline=cmdline.txt
initramfs initrd.img followkernel

# Enable the audio output, I2C and SPI interfaces on the GPIO header
dtparam=audio=on
dtparam=i2c_arm=on
dtparam=spi=on

# Enable the FKMS ("Fake" KMS) graphics overlay, enable the camera firmware
# and allocate 128Mb to the GPU memory
dtoverlay=tc358743
dtoverlay=vc4-fkms-v3d
gpu_mem=128
start_x=1

# Comment out the following line if the edges of the desktop appear outside
# the edges of your display
disable_overscan=1

# If you have issues with audio, you may try uncommenting the following line
# which forces the HDMI output into HDMI mode instead of DVI (which doesn't
# support audio output)
#hdmi_drive=2

# If you have a CM4, uncomment the following line to enable the USB2 outputs
# on the IO board (assuming your CM4 is plugged into such a board)
#dtoverlay=dwc2,dr_mode=host

Code: Select all

ubuntu@ubuntu-desktop:~$ vcgencmd version
Sep  2 2020 21:14:24
Copyright (c) 2012 Broadcom
version 4439d2aaa6c376a2d1ef4402f142e1cf4de37c43 (clean) (release) (start_x)
ubuntu@ubuntu-desktop:~$ uname -a
Linux ubuntu-desktop 5.8.0-1007-raspi #10-Ubuntu SMP PREEMPT Thu Nov 5 17:52:40 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

Code: Select all

ubuntu@ubuntu-desktop:~$ lsmod
Module                  Size  Used by
rfcomm                110592  4
algif_hash             20480  1
aes_neon_bs            32768  2
aes_neon_blk           36864  3 aes_neon_bs
crypto_simd            24576  2 aes_neon_bs,aes_neon_blk
cryptd                 32768  2 crypto_simd
algif_skcipher         20480  1
af_alg                 32768  6 algif_hash,algif_skcipher
hci_uart              163840  1
btqca                  20480  1 hci_uart
btrtl                  28672  1 hci_uart
btbcm                  24576  1 hci_uart
btintel                32768  1 hci_uart
bnep                   36864  2
vc4                   294912  9
drm_kms_helper        282624  3 vc4
snd_soc_core          294912  1 vc4
snd_compress           36864  1 snd_soc_core
btsdio                 20480  0
ac97_bus               16384  1 snd_soc_core
joydev                 32768  0
snd_bcm2835            36864  2
pl2303                 28672  0
snd_pcm_dmaengine      20480  1 snd_soc_core
input_leds             16384  0
bluetooth             737280  35 btrtl,btqca,btsdio,btintel,hci_uart,btbcm,bnep,rfcomm
usbserial              65536  1 pl2303
snd_pcm_oss            69632  0
snd_mixer_oss          36864  1 snd_pcm_oss
ecdh_generic           16384  1 bluetooth
ecc                    32768  1 ecdh_generic
snd_pcm               155648  6 vc4,snd_bcm2835,snd_compress,snd_pcm_oss,snd_soc_core,snd_pcm_dmaengine
brcmfmac              434176  0
snd_seq_dummy          16384  0
snd_seq_oss            61440  0
brcmutil               28672  1 brcmfmac
tc358743               45056  1
cfg80211              921600  1 brcmfmac
cec                    77824  3 drm_kms_helper,vc4,tc358743
v3d                    90112  3
snd_seq_midi           20480  0
bcm2835_unicam         49152  0
snd_seq_midi_event     16384  2 snd_seq_midi,snd_seq_oss
gpu_sched              45056  1 v3d
v4l2_fwnode            36864  2 bcm2835_unicam,tc358743
snd_rawmidi            53248  1 snd_seq_midi
raspberrypi_hwmon      16384  0
v4l2_dv_timings        40960  2 bcm2835_unicam,tc358743
crct10dif_ce           20480  1
snd_seq               102400  6 snd_seq_midi,snd_seq_oss,snd_seq_midi_event,snd_seq_dummy
bcm2835_codec          49152  0
bcm2835_isp            36864  0
v4l2_mem2mem           49152  1 bcm2835_codec
bcm2835_v4l2           49152  0
snd_seq_device         20480  4 snd_seq,snd_seq_midi,snd_seq_oss,snd_rawmidi
bcm2835_mmal_vchiq     49152  3 bcm2835_codec,bcm2835_v4l2,bcm2835_isp
videobuf2_dma_contig    24576  3 bcm2835_codec,bcm2835_unicam,bcm2835_isp
snd_timer              49152  2 snd_seq,snd_pcm
videobuf2_vmalloc      20480  1 bcm2835_v4l2
videobuf2_memops       20480  2 videobuf2_vmalloc,videobuf2_dma_contig
videobuf2_v4l2         36864  5 bcm2835_codec,bcm2835_unicam,bcm2835_v4l2,v4l2_mem2mem,bcm2835_isp
videobuf2_common       65536  6 bcm2835_codec,videobuf2_v4l2,bcm2835_unicam,bcm2835_v4l2,v4l2_mem2mem,bcm2835_isp
snd                   131072  15 snd_seq,snd_seq_device,snd_seq_oss,snd_bcm2835,snd_timer,snd_compress,snd_pcm_oss,snd_soc_core,snd_pcm,snd_rawmidi,snd_mixer_oss
videodev              348160  9 bcm2835_codec,v4l2_fwnode,videobuf2_v4l2,bcm2835_unicam,bcm2835_v4l2,videobuf2_common,tc358743,v4l2_mem2mem,bcm2835_isp
mc                     73728  8 videodev,bcm2835_codec,videobuf2_v4l2,bcm2835_unicam,videobuf2_common,tc358743,v4l2_mem2mem,bcm2835_isp
rpivid_mem             16384  0
fb_sys_fops            20480  1 drm_kms_helper
vc_sm_cma              49152  2 bcm2835_mmal_vchiq,bcm2835_isp
syscopyarea            16384  1 drm_kms_helper
sysfillrect            16384  1 drm_kms_helper
sysimgblt              16384  1 drm_kms_helper
uio_pdrv_genirq        16384  0
uio                    28672  1 uio_pdrv_genirq
sch_fq_codel           20480  7
ppdev                  24576  0
lp                     24576  0
parport                40960  2 lp,ppdev
drm                   663552  11 gpu_sched,drm_kms_helper,v3d,vc4
ip_tables              45056  0
x_tables               57344  1 ip_tables
autofs4                61440  2
btrfs                1597440  0
blake2b_generic        20480  0
xor                    20480  1 btrfs
xor_neon               16384  1 xor
hid_generic            16384  0
usbhid                 69632  0
raid6_pq              114688  1 btrfs
libcrc32c              16384  1 btrfs
dm_mirror              28672  0
dm_region_hash         28672  1 dm_mirror
dm_log                 20480  2 dm_region_hash,dm_mirror
i2c_mux_pinctrl        16384  0
i2c_mux                16384  1 i2c_mux_pinctrl
spidev                 24576  0
xhci_pci               24576  0
xhci_pci_renesas       24576  1 xhci_pci
phy_generic            20480  0

Code: Select all

ubuntu@ubuntu-desktop:~$ v4l2-ctl --query-dv-timings
        Active width: 1280
        Active height: 720
        Total width: 1650
        Total height: 750
        Frame format: progressive
        Polarities: -vsync -hsync
        Pixelclock: 74250000 Hz (60.00 frames per second)
        Horizontal frontporch: 0
        Horizontal sync: 370
        Horizontal backporch: 0
        Vertical frontporch: 0
        Vertical sync: 30
        Vertical backporch: 0
        Standards:
        Flags:
ubuntu@ubuntu-desktop:~$ v4l2-ctl --set-dv-bt-timings query
BT timings set
ubuntu@ubuntu-desktop:~$ v4l2-ctl -V
Format Video Capture:
        Width/Height      : 1280/720
        Pixel Format      : 'RGB3' (24-bit RGB 8-8-8)
        Field             : None
        Bytes per Line    : 3840
        Size Image        : 2764800
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             :
ubuntu@ubuntu-desktop:~$ v4l2-ctl -v pixelformat=UYVY
ubuntu@ubuntu-desktop:~$ v4l2-ctl --log-status

Status Log:

   [ 7577.741201] unicam fe801000.csi: =================  START STATUS  =================
   [ 7577.742861] tc358743 10-000f: -----Chip status-----
   [ 7577.743466] tc358743 10-000f: Chip ID: 0x00
   [ 7577.744071] tc358743 10-000f: Chip revision: 0x00
   [ 7577.744076] tc358743 10-000f: Reset: IR: 1, CEC: 1, CSI TX: 0, HDMI: 0
   [ 7577.744079] tc358743 10-000f: Sleep mode: off
   [ 7577.744083] tc358743 10-000f: Cable detected (+5V power): yes
   [ 7577.744737] tc358743 10-000f: DDC lines enabled: yes
   [ 7577.745255] tc358743 10-000f: Hotplug enabled: yes
   [ 7577.745859] tc358743 10-000f: CEC enabled: no
   [ 7577.745863] tc358743 10-000f: -----Signal status-----
   [ 7577.745866] tc358743 10-000f: TMDS signal detected: yes
   [ 7577.745869] tc358743 10-000f: Stable sync signal: yes
   [ 7577.745873] tc358743 10-000f: PHY PLL locked: yes
   [ 7577.745876] tc358743 10-000f: PHY DE detected: yes
   [ 7577.752510] tc358743 10-000f: Detected format: 1280x720p60.00 (1650x750)
   [ 7577.752516] tc358743 10-000f: horizontal: fp = 0, -sync = 370, bp = 0
   [ 7577.752521] tc358743 10-000f: vertical: fp = 0, -sync = 30, bp = 0
   [ 7577.752524] tc358743 10-000f: pixelclock: 74250000
   [ 7577.752529] tc358743 10-000f: flags (0x0):
   [ 7577.752533] tc358743 10-000f: standards (0x0):
   [ 7577.752538] tc358743 10-000f: Configured format: 1280x720p60.00 (1650x750)
   [ 7577.752541] tc358743 10-000f: horizontal: fp = 0, -sync = 370, bp = 0
   [ 7577.752545] tc358743 10-000f: vertical: fp = 0, -sync = 30, bp = 0
   [ 7577.752549] tc358743 10-000f: pixelclock: 74250000
   [ 7577.752553] tc358743 10-000f: flags (0x0):
   [ 7577.752556] tc358743 10-000f: standards (0x0):
   [ 7577.752559] tc358743 10-000f: -----CSI-TX status-----
   [ 7577.752564] tc358743 10-000f: Lanes needed: 1
   [ 7577.752567] tc358743 10-000f: Lanes in use: 1
   [ 7577.753172] tc358743 10-000f: Waiting for particular sync signal: no
   [ 7577.753775] tc358743 10-000f: Transmit mode: no
   [ 7577.754378] tc358743 10-000f: Receive mode: no
   [ 7577.754981] tc358743 10-000f: Stopped: no
   [ 7577.754984] tc358743 10-000f: Color space: YCbCr 422 16-bit
   [ 7577.755497] tc358743 10-000f: -----HDMI status-----
   [ 7577.755501] tc358743 10-000f: HDCP encrypted content: no
   [ 7577.755505] tc358743 10-000f: Input color space: RGB limited range
   [ 7577.756018] tc358743 10-000f: AV Mute: off
   [ 7577.756537] tc358743 10-000f: Deep color mode: 8-bits per channel
   [ 7577.759011] tc358743 10-000f: HDMI infoframe: Auxiliary Video Information (AVI), version 2, length 13
   [ 7577.759017] tc358743 10-000f:     colorspace: RGB
   [ 7577.759022] tc358743 10-000f:     scan mode: No Data
   [ 7577.759026] tc358743 10-000f:     colorimetry: ITU709
   [ 7577.759030] tc358743 10-000f:     picture aspect: 16:9
   [ 7577.759035] tc358743 10-000f:     active aspect: Same as Picture
   [ 7577.759039] tc358743 10-000f:     itc: IT Content
   [ 7577.759043] tc358743 10-000f:     extended colorimetry: xvYCC 601
   [ 7577.759047] tc358743 10-000f:     quantization range: Limited
   [ 7577.759051] tc358743 10-000f:     nups: Unknown Non-uniform Scaling
   [ 7577.759055] tc358743 10-000f:     video code: 4
   [ 7577.759059] tc358743 10-000f:     ycc quantization range: Limited
   [ 7577.759063] tc358743 10-000f:     hdmi content type: Graphics
   [ 7577.759068] tc358743 10-000f:     pixel repeat: 0
   [ 7577.759072] tc358743 10-000f:     bar top 0, bottom 0, left 0, right 0
   [ 7577.759077] unicam fe801000.csi: -----Receiver status-----
   [ 7577.759080] unicam fe801000.csi: V4L2 width/height:   1280x720
   [ 7577.759084] unicam fe801000.csi: Mediabus format:     0000200f
   [ 7577.759088] unicam fe801000.csi: V4L2 format:         59565955
   [ 7577.759092] unicam fe801000.csi: Unpacking/packing:   0 / 0
   [ 7577.759095] unicam fe801000.csi: ----Live data----
   [ 7577.759099] unicam fe801000.csi: Programmed stride:      0
   [ 7577.759102] unicam fe801000.csi: Detected resolution: 0x0
   [ 7577.759106] unicam fe801000.csi: Write pointer:       00000000
   [ 7577.759110] unicam fe801000.csi: ==================  END STATUS  ==================

Code: Select all

ubuntu@ubuntu-desktop:~$ vcgencmd get_camera
supported=1 detected=0
ubuntu@ubuntu-desktop:~$ raspistill -o picure.jpg
mmal: Cannot read camera info, keeping the defaults for OV5647
mmal: mmal_vc_component_create: failed to create component 'vc.ril.camera' (1:ENOMEM)
mmal: mmal_component_create_core: could not create component 'vc.ril.camera' (1)
mmal: Failed to create camera component
mmal: main: Failed to create camera component
mmal: Camera is not detected. Please check carefully the camera module is installed correctly
I tried even change firmware by hand but it then the OS didn't even boot :D

Anyone any idea?
And again please don't suggest hardware issue because on Raspbian it works perfectly fine.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12379
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Wed Nov 25, 2020 1:40 pm

upanie wrote:
Tue Nov 24, 2020 11:29 pm
I'm trying to make B101 from Auvidea work on RPI4/4G under Ubuntu.
I tried Ubuntu 18.04 and 20.10 with the same results/problems.

Case no. 1
On fresh install of Ubuntu after enabling camera (start_x1=1 in /boot/firmware/config.txt) it sort of works.
I can run ustreamer or get the video using picamera and it works fine.
Fine means that the video content is visible and is perfectly fine :)
But I can't control the b101 module. I can't set EDID, check timings, nothing.
By default, without EDID, B101 works in 720p and the HDMI source can't see it as a HDMI "sink".
No resolution negotiation, nothing. So for most cases it's useless. But at least video works in one configuration.
All use of raspivid/still/Picamera with a tc358743 device is unsupported.
upanie wrote:Case no. 2
So following @6by9 instructions I added overlay to config.txt file and after that /dev/video0 and /dev/video1 vanished.
I found that Ubuntu doesn't come with tc358743 kernel module. So I built it and after loading it video device came back.
And now I can control the module. I can set an EDID, check and set timing, etc.
Any HDMI source can now see a valid video receiver (display), resolution negotiation works fine. Everything looks fine but...
I can't get any video from B101 module.
@6by9 I found your comment here https://github.com/raspberrypi/linux/is ... -624565359
It looks exactly like my case. Do you remember what was the cause?
My problem was a dead board or possibly part broken FFC from board to Pi.
What commands have you used to determine that your B101 is dead?
upanie wrote:After making the same steps on Raspbian everything works fine. I can control it and get the video content.
So it has nothing to do with the hardware.

At first I was trying to make it work on Ubuntu 18.04 but then I tried 20.10 with newer kernel and newer firmware but I get the same results.
<snip>

Code: Select all

ubuntu@ubuntu-desktop:~$ v4l2-ctl --query-dv-timings
        Active width: 1280
        Active height: 720
        Total width: 1650
        Total height: 750
        Frame format: progressive
        Polarities: -vsync -hsync
        Pixelclock: 74250000 Hz (60.00 frames per second)
        Horizontal frontporch: 0
        Horizontal sync: 370
        Horizontal backporch: 0
        Vertical frontporch: 0
        Vertical sync: 30
        Vertical backporch: 0
        Standards:
        Flags:
ubuntu@ubuntu-desktop:~$ v4l2-ctl --set-dv-bt-timings query
BT timings set
ubuntu@ubuntu-desktop:~$ v4l2-ctl -V
Format Video Capture:
        Width/Height      : 1280/720
        Pixel Format      : 'RGB3' (24-bit RGB 8-8-8)
        Field             : None
        Bytes per Line    : 3840
        Size Image        : 2764800
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             :
ubuntu@ubuntu-desktop:~$ v4l2-ctl -v pixelformat=UYVY
ubuntu@ubuntu-desktop:~$ v4l2-ctl --log-status

Status Log:

   [ 7577.741201] unicam fe801000.csi: =================  START STATUS  =================
   [ 7577.742861] tc358743 10-000f: -----Chip status-----
   [ 7577.743466] tc358743 10-000f: Chip ID: 0x00
   [ 7577.744071] tc358743 10-000f: Chip revision: 0x00
   [ 7577.744076] tc358743 10-000f: Reset: IR: 1, CEC: 1, CSI TX: 0, HDMI: 0
   [ 7577.744079] tc358743 10-000f: Sleep mode: off
   [ 7577.744083] tc358743 10-000f: Cable detected (+5V power): yes
   [ 7577.744737] tc358743 10-000f: DDC lines enabled: yes
   [ 7577.745255] tc358743 10-000f: Hotplug enabled: yes
   [ 7577.745859] tc358743 10-000f: CEC enabled: no
   [ 7577.745863] tc358743 10-000f: -----Signal status-----
   [ 7577.745866] tc358743 10-000f: TMDS signal detected: yes
   [ 7577.745869] tc358743 10-000f: Stable sync signal: yes
   [ 7577.745873] tc358743 10-000f: PHY PLL locked: yes
   [ 7577.745876] tc358743 10-000f: PHY DE detected: yes
   [ 7577.752510] tc358743 10-000f: Detected format: 1280x720p60.00 (1650x750)
   [ 7577.752516] tc358743 10-000f: horizontal: fp = 0, -sync = 370, bp = 0
   [ 7577.752521] tc358743 10-000f: vertical: fp = 0, -sync = 30, bp = 0
   [ 7577.752524] tc358743 10-000f: pixelclock: 74250000
   [ 7577.752529] tc358743 10-000f: flags (0x0):
   [ 7577.752533] tc358743 10-000f: standards (0x0):
   [ 7577.752538] tc358743 10-000f: Configured format: 1280x720p60.00 (1650x750)
   [ 7577.752541] tc358743 10-000f: horizontal: fp = 0, -sync = 370, bp = 0
   [ 7577.752545] tc358743 10-000f: vertical: fp = 0, -sync = 30, bp = 0
   [ 7577.752549] tc358743 10-000f: pixelclock: 74250000
   [ 7577.752553] tc358743 10-000f: flags (0x0):
   [ 7577.752556] tc358743 10-000f: standards (0x0):
   [ 7577.752559] tc358743 10-000f: -----CSI-TX status-----
   [ 7577.752564] tc358743 10-000f: Lanes needed: 1
   [ 7577.752567] tc358743 10-000f: Lanes in use: 1
   [ 7577.753172] tc358743 10-000f: Waiting for particular sync signal: no
   [ 7577.753775] tc358743 10-000f: Transmit mode: no
   [ 7577.754378] tc358743 10-000f: Receive mode: no
   [ 7577.754981] tc358743 10-000f: Stopped: no
   [ 7577.754984] tc358743 10-000f: Color space: YCbCr 422 16-bit
   [ 7577.755497] tc358743 10-000f: -----HDMI status-----
   [ 7577.755501] tc358743 10-000f: HDCP encrypted content: no
   [ 7577.755505] tc358743 10-000f: Input color space: RGB limited range
   [ 7577.756018] tc358743 10-000f: AV Mute: off
   [ 7577.756537] tc358743 10-000f: Deep color mode: 8-bits per channel
   [ 7577.759011] tc358743 10-000f: HDMI infoframe: Auxiliary Video Information (AVI), version 2, length 13
   [ 7577.759017] tc358743 10-000f:     colorspace: RGB
   [ 7577.759022] tc358743 10-000f:     scan mode: No Data
   [ 7577.759026] tc358743 10-000f:     colorimetry: ITU709
   [ 7577.759030] tc358743 10-000f:     picture aspect: 16:9
   [ 7577.759035] tc358743 10-000f:     active aspect: Same as Picture
   [ 7577.759039] tc358743 10-000f:     itc: IT Content
   [ 7577.759043] tc358743 10-000f:     extended colorimetry: xvYCC 601
   [ 7577.759047] tc358743 10-000f:     quantization range: Limited
   [ 7577.759051] tc358743 10-000f:     nups: Unknown Non-uniform Scaling
   [ 7577.759055] tc358743 10-000f:     video code: 4
   [ 7577.759059] tc358743 10-000f:     ycc quantization range: Limited
   [ 7577.759063] tc358743 10-000f:     hdmi content type: Graphics
   [ 7577.759068] tc358743 10-000f:     pixel repeat: 0
   [ 7577.759072] tc358743 10-000f:     bar top 0, bottom 0, left 0, right 0
   [ 7577.759077] unicam fe801000.csi: -----Receiver status-----
   [ 7577.759080] unicam fe801000.csi: V4L2 width/height:   1280x720
   [ 7577.759084] unicam fe801000.csi: Mediabus format:     0000200f
   [ 7577.759088] unicam fe801000.csi: V4L2 format:         59565955
   [ 7577.759092] unicam fe801000.csi: Unpacking/packing:   0 / 0
   [ 7577.759095] unicam fe801000.csi: ----Live data----
   [ 7577.759099] unicam fe801000.csi: Programmed stride:      0
   [ 7577.759102] unicam fe801000.csi: Detected resolution: 0x0
   [ 7577.759106] unicam fe801000.csi: Write pointer:       00000000
   [ 7577.759110] unicam fe801000.csi: ==================  END STATUS  ==================
That all looks fine for setting stuff up, but what command have you used to request the device to stream?
upanie wrote:

Code: Select all

ubuntu@ubuntu-desktop:~$ vcgencmd get_camera
supported=1 detected=0
ubuntu@ubuntu-desktop:~$ raspistill -o picure.jpg
mmal: Cannot read camera info, keeping the defaults for OV5647
mmal: mmal_vc_component_create: failed to create component 'vc.ril.camera' (1:ENOMEM)
mmal: mmal_component_create_core: could not create component 'vc.ril.camera' (1)
mmal: Failed to create camera component
mmal: main: Failed to create camera component
mmal: Camera is not detected. Please check carefully the camera module is installed correctly
"vcgencmd get_camera" is irrelevant when using the kernel drivers.
raspistill is unsupported with the firmware drivers, and irrelevant with the kernel drivers. DON'T USE IT.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

upanie
Posts: 2
Joined: Tue Nov 24, 2020 10:49 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Wed Nov 25, 2020 10:09 pm

6by9 wrote: What commands have you used to determine that your B101 is dead?
My board isn't dead for sure. It works fine in Raspbian.
6by9 wrote: That all looks fine for setting stuff up, but what command have you used to request the device to stream?
1. my custom software based on picamera - but since you said it won't work then let's forget about it.
2. gstreamer

Code: Select all

gst-launch-1.0 v4l2src ! "video/x-raw,framerate=30/1,format=UYVY" ! v4l2h264enc extra-controls="controls,h264_profile=4,h264_level=10,video_bitrate=256000;" ! video/x-h264,profile=high ! h264parse ! queue ! matroskamux ! filesink location=foo.mkv
.
3. ustreamer: https://github.com/pikvm/ustreamer

Code: Select all

./ustreamer --format=uyvy --encoder=HW --workers=3 --persistent --dv-timings --drop-same-frames=30 --host=0.0.0.0
Ubuntu:
1. works when tc358734 not used. If used, then you know... unsupported :)
2. when tc358734 used it produces empty file (size: 0 bytes)
3. works when tc358734 not used. If used then it can't read video stream.

Raspbian:
1. didn't check
2. gstreamer doesn't work:

Code: Select all

pi@raspberrypi:~ $ gst-launch-1.0 v4l2src ! "video/x-raw,framerate=30/1,format=UYVY" ! v4l2h264enc extra-controls="controls,h264_profile=4,h264_level=10,video_bitrate=256000;" ! video/x-h264,profile=high ! h264parse ! queue ! matroskamux ! filesink location=foo.mkv
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0: Failed to process frame.
Additional debug info:
gstv4l2videoenc.c(803): gst_v4l2_video_enc_handle_frame (): /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0:
Maybe be due to not enough memory or failing driver
Execution ended after 0:00:00.095694562
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
3. ustreamer works fine when tc358734 is used.

I din't check what happens without tc358734 driver in Raspbian but since it works with the driver then I don't need to check that.

So to sum up: in Ubuntu nothing works, in Raspbian ustreamer works fine.
But I really need Ubuntu :(

scriffij
Posts: 2
Joined: Fri Nov 27, 2020 8:17 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Fri Nov 27, 2020 8:22 pm

I purchased a CSI-2 to HDMI B101 bridge (38126-4) and when I plug it into my RPi 4 the Pi won't boot. I get no green lights on the Pi as if the SD card is not being read/seen. I have the ribbon cable plugged in with the contacts facing up as described in the documentation. When the bridge is unplugged my Pi boots just fine!

Does anyone have any ideas as to why this might happen and more importantly how to fix ?

Cheers
Jason

berkutta
Posts: 37
Joined: Mon Feb 08, 2021 7:06 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Mon Feb 08, 2021 7:15 pm

First thanks for this very informational thread. I have tried to get a TC358743 running and output a h.264 stream.

With the following Pipeline I was able to encode the HDMI source to h.264. But the CPU Load is very high as it needs to convert the Color Space:

Code: Select all

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, width=800, height=600, framerate=30/1 ! videoconvert ! omxh264enc ! video/x-h264, profile=baseline, stream-format=byte-stream ! appsink name="appsink"

To get the CPU Load down I've tried to build gstreamer 1.16 and gst-omx with this branch https://github.com/6by9/gst-omx/commits/pi which should allow to fed UYVY Color Space directly into the Encoder. Everything built fine after some struggle. But the Target Pipeline exits with some OpenMAX Error.

This Pipeline still works:

Code: Select all

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,framerate=60/1,colorimetry=bt601 ! videoconvert ! video/x-raw, format=I420 ! omxh264enc ! video/x-h264, profile=baseline, stream-format=byte-stream ! appsink name="appsink"

But the Pipeline without color space conversion fails with some weird OpenMAX Error:

Code: Select all

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,framerate=60/1,colorimetry=bt601 ! omxh264enc ! video/x-h264, profile=baseline, stream-format=byte-stream ! appsink name="appsink"
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, framerate=(fraction)60/1, colorimetry=(string)bt601, width=(int)800, height=(int)600, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, framerate=(fraction)60/1, colorimetry=(string)bt601, width=(int)800, height=(int)600, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, framerate=(fraction)60/1, colorimetry=(string)bt601, width=(int)800, height=(int)600, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, framerate=(fraction)60/1, colorimetry=(string)bt601, width=(int)800, height=(int)600, interlace-mode=(string)progressive
ERROR: from element /GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0: Could not write to resource.
Additional debug info:
gstomxvideoenc.c(2847): gst_omx_video_enc_handle_frame (): /GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0:
Failed to write input into the OpenMAX buffer
Execution ended after 0:00:00.049699865
Setting pipeline to NULL ...
Freeing pipeline ...

@6by9 Do I maybe need to configure something in config.txt or similiar?

berkutta
Posts: 37
Joined: Mon Feb 08, 2021 7:06 pm

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Wed Feb 10, 2021 5:21 pm

I have now switched to the v4l2h264enc which is way easier to use. So ignore the previous post :)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12379
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Wed Feb 10, 2021 5:37 pm

berkutta wrote:
Wed Feb 10, 2021 5:21 pm
I have now switched to the v4l2h264enc which is way easier to use. So ignore the previous post :)
OpenMax is being deprecated, and is 32bit only.
V4L2 exists and works in both 32 and 64bit kernels. If you can use it then do so. It is also more efficient in the way it handles image buffers.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fd_
Posts: 105
Joined: Thu Oct 25, 2018 7:35 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Mar 11, 2021 11:10 am

Hi,
I'm trying to build a custom application for a Pi Zero that streams the HDMI input of the TC358743 to a tablet as a H264 stream, while simultaneously displaying it on the Pi's HDMI output. The goal is to be able to use the tablet as the display for yet another Raspberry Pi without requiring any special software setup (VNC server).
In order to obtain the desired level of performance on the somewhat limited processing power of the Pi Zero, I'm writing my own application based on the excellent yavta sample by @6by9.

While the whole system works in principle, the Pi Zero's SoC gets quite hot from running my application. Since the CPU usage is only around 10-20%, I assume this is because the setup is quite taxing on the VPU. Now I'm wondering if there's some way to reduce the load on the VPU to decrease SoC temperatures (I'm planning to put this in a closed box eventually). I think @6by9 mentioned something about unnecessary colour conversion in yavta at some point, is there some room for getting rid of some overhead?

Any help is highly appreciated!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12379
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Mar 11, 2021 11:52 am

fd_ wrote:
Thu Mar 11, 2021 11:10 am
Hi,
I'm trying to build a custom application for a Pi Zero that streams the HDMI input of the TC358743 to a tablet as a H264 stream, while simultaneously displaying it on the Pi's HDMI output. The goal is to be able to use the tablet as the display for yet another Raspberry Pi without requiring any special software setup (VNC server).
In order to obtain the desired level of performance on the somewhat limited processing power of the Pi Zero, I'm writing my own application based on the excellent yavta sample by @6by9.

While the whole system works in principle, the Pi Zero's SoC gets quite hot from running my application. Since the CPU usage is only around 10-20%, I assume this is because the setup is quite taxing on the VPU. Now I'm wondering if there's some way to reduce the load on the VPU to decrease SoC temperatures (I'm planning to put this in a closed box eventually). I think @6by9 mentioned something about unnecessary colour conversion in yavta at some point, is there some room for getting rid of some overhead?
The video_encoder will take the UYVY format that is produced directly. The HVS (display engine) won't, which is why there is an ISP component in the processing pipeline.
Whilst video_encode could have been fed the source V4L2 buffer in parallel to the ISP being used for the display, it was easier to feed both encoder and display off the same output.

That said, it wouldn't take much to remove the ISP and render side:
- all the current setup of the isp input should now apply to the video_encoder input.
- drop the isp output and render input configuration.
- drop the existing encoder input setup as it's been superceded.
- drop the isp output buffer setup
- enable_mmal_ip now wants to work on the encoder input (not isp input)
- buffers get sent to the encoder input, not isp.

I'd suggest you remove the definitions for the isp and render variables, because then the compiler will tell you if you've left any behind.
You'll probably get warnings/errors over buffers_to_isp and isp_output_callback being defined but not used, so those can be deleted too.
Do ensure that you're now using the isp_ip_cb function as the callback on the encoder input port, otherwise buffers will likely disappear on you.

Good luck.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fd_
Posts: 105
Joined: Thu Oct 25, 2018 7:35 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Mar 11, 2021 5:15 pm

6by9 wrote:
Thu Mar 11, 2021 11:52 am
The video_encoder will take the UYVY format that is produced directly. The HVS (display engine) won't, which is why there is an ISP component in the processing pipeline.
Whilst video_encode could have been fed the source V4L2 buffer in parallel to the ISP being used for the display, it was easier to feed both encoder and display off the same output.

That said, it wouldn't take much to remove the ISP and render side:
- all the current setup of the isp input should now apply to the video_encoder input.
- drop the isp output and render input configuration.
- drop the existing encoder input setup as it's been superceded.
- drop the isp output buffer setup
- enable_mmal_ip now wants to work on the encoder input (not isp input)
- buffers get sent to the encoder input, not isp.

I'd suggest you remove the definitions for the isp and render variables, because then the compiler will tell you if you've left any behind.
You'll probably get warnings/errors over buffers_to_isp and isp_output_callback being defined but not used, so those can be deleted too.
Do ensure that you're now using the isp_ip_cb function as the callback on the encoder input port, otherwise buffers will likely disappear on you.

Good luck.
Thank you very much for your hints!

For my application, I’d like to keep the render component (I want rendering + H264 encoding). However, I think I’d be happy with just going up to 1080p30, so can I get rid of the isp in the pipeline by using RGB24 from the TC358743? Would that still reduce the load on the VPU?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12379
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Mar 11, 2021 5:26 pm

fd_ wrote:
Thu Mar 11, 2021 5:15 pm
For my application, I’d like to keep the render component (I want rendering + H264 encoding). However, I think I’d be happy with just going up to 1080p30, so can I get rid of the isp in the pipeline by using RGB24 from the TC358743? Would that still reduce the load on the VPU?
The VPU itself shouldn't be doing a lot as it's almost all handed off to hardware blocks. The hardware blocks are obviously going to be working fairly hard though, particularly the ISP as it is used once for the conversion, and a second time as part of video_encode.

Using RGB24 from the Toshiba should be possible and allow dropping of the ISP component. You'll need to do a bit of fiddling to send the V4L2 buffer to all the relevant destination components as you can't do it with replicates in isp_output_callback.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fd_
Posts: 105
Joined: Thu Oct 25, 2018 7:35 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Mar 11, 2021 5:30 pm

6by9 wrote:
Thu Mar 11, 2021 5:26 pm
The VPU itself shouldn't be doing a lot as it's almost all handed off to hardware blocks. The hardware blocks are obviously going to be working fairly hard though, particularly the ISP as it is used once for the conversion, and a second time as part of video_encode.

Using RGB24 from the Toshiba should be possible and allow dropping of the ISP component. You'll need to do a bit of fiddling to send the V4L2 buffer to all the relevant destination components as you can't do it with replicates in isp_output_callback.
Fantastic! What do you mean by a bit of fiddling? Can't I essentially do what's currently done to feed the buffers to the ISP, except I'd do it twice, once for the render component and once for the encoder?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12379
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Mar 11, 2021 5:58 pm

fd_ wrote:
Thu Mar 11, 2021 5:30 pm
Fantastic! What do you mean by a bit of fiddling? Can't I essentially do what's currently done to feed the buffers to the ISP, except I'd do it twice, once for the render component and once for the encoder?
Yes, just that. You need two pools of buffer headers and just populate them in the same way.
You can't use the mmal_buffer_header_replicate that isp_output_callback uses as those link the replicas to the original buffer header, and you don't have an original buffer header.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fd_
Posts: 105
Joined: Thu Oct 25, 2018 7:35 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Mar 11, 2021 6:00 pm

6by9 wrote:
Thu Mar 11, 2021 5:58 pm
fd_ wrote:
Thu Mar 11, 2021 5:30 pm
Fantastic! What do you mean by a bit of fiddling? Can't I essentially do what's currently done to feed the buffers to the ISP, except I'd do it twice, once for the render component and once for the encoder?
Yes, just that. You need two pools of buffer headers and just populate them in the same way.
You can't use the mmal_buffer_header_replicate that isp_output_callback uses as those link the replicas to the original buffer header, and you don't have an original buffer header.
Excellent! Thanks so much for your assistance here! Hugely appreciated!

fd_
Posts: 105
Joined: Thu Oct 25, 2018 7:35 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Mar 11, 2021 6:20 pm

6by9 wrote:
Thu Mar 11, 2021 5:58 pm
Yes, just that. You need two pools of buffer headers and just populate them in the same way.
You can't use the mmal_buffer_header_replicate that isp_output_callback uses as those link the replicas to the original buffer header, and you don't have an original buffer header.
Well, won't I get a problem because I have two MMAL buffers (one for render, one for encode) linked to the same V4L2 buffer? Or can I just keep reference to both MMAL buffers from the V4L2 buffer and pray they'll have the same fill cycles?

Could I just replace the ISP with a video splitter that just copies without format conversion so I don't have to change the buffer management? Also, what happens if the ISP has the same input and output format? Can it be effectively turned into a simple splitter?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12379
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Thu Mar 11, 2021 7:03 pm

fd_ wrote:
Thu Mar 11, 2021 6:20 pm
Well, won't I get a problem because I have two MMAL buffers (one for render, one for encode) linked to the same V4L2 buffer? Or can I just keep reference to both MMAL buffers from the V4L2 buffer and pray they'll have the same fill cycles?
They should, although there may be some oddness if the display rate is slower than the incoming rate (two updates in the same vblank period will result in the buffers coming back in a different order).
fd_ wrote:Could I just replace the ISP with a video splitter that just copies without format conversion so I don't have to change the buffer management? Also, what happens if the ISP has the same input and output format? Can it be effectively turned into a simple splitter?
vc.ril.video_splitter will copy the data from input to output ports. It is not a zero cost operation.
Providing the same buffer handle to two components (via separate buffer headers) is a zero cost operation.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fd_
Posts: 105
Joined: Thu Oct 25, 2018 7:35 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Fri Mar 12, 2021 5:44 pm

6by9 wrote:
Thu Mar 11, 2021 7:03 pm
fd_ wrote:
Thu Mar 11, 2021 6:20 pm
Well, won't I get a problem because I have two MMAL buffers (one for render, one for encode) linked to the same V4L2 buffer? Or can I just keep reference to both MMAL buffers from the V4L2 buffer and pray they'll have the same fill cycles?
They should, although there may be some oddness if the display rate is slower than the incoming rate (two updates in the same vblank period will result in the buffers coming back in a different order).
I have now removed the ISP, and while it appears to be working in principle, it seems like I do see issues with buffer orders after a few minutes of operation. Basically, I regularly see frames from the past, which becomes particularly apparent during movement on the HDMI input image.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12379
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Fri Mar 12, 2021 5:50 pm

fd_ wrote:
Fri Mar 12, 2021 5:44 pm
I have now removed the ISP, and while it appears to be working in principle, it seems like I do see issues with buffer orders after a few minutes of operation. Basically, I regularly see frames from the past, which becomes particularly apparent during movement on the HDMI input image.
Rather than using mmal_queue_get, you have an index within the V4L2 buffer, and an array of buffers within the MMAL_POOL_T.. Combine one with the other.
I think code in isp_ip_cb which releases the buffers and returns them to V4L2 is safe in that regard anyway.

TBH it should really do that anyway, and that would avoid the "Mismatch in expected buffers. V4L2 gave idx %d, MMAL expecting %d" warning path.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fd_
Posts: 105
Joined: Thu Oct 25, 2018 7:35 am

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Sat Mar 13, 2021 1:11 pm

6by9 wrote:
Fri Mar 12, 2021 5:50 pm
Rather than using mmal_queue_get, you have an index within the V4L2 buffer, and an array of buffers within the MMAL_POOL_T.. Combine one with the other.
I think code in isp_ip_cb which releases the buffers and returns them to V4L2 is safe in that regard anyway.

TBH it should really do that anyway, and that would avoid the "Mismatch in expected buffers. V4L2 gave idx %d, MMAL expecting %d" warning path.
This mostly seems to have done the trick! I have now tested this for a few hours. Although the effect of removing the ISP on SoC temperature are minimal, it seems to have improved overall reliability (see point 3 below). Here are all my findings:

1. Kernel errors
I am regularly getting kernel errors that seem to come from the unicam driver. Luckily, the errors don't seem to influence the operation of my program (or of the OS in general), but I still am afraid they'll have influenced some part of the OS that I just haven't tested yet. I think I have not seen these issues before removing the ISP, though I wonder how the MMAL changes would have affected the V4L2 side. I have included a log of the kernel issues at the bottom of this post. Please let me know if I might just be doing something wrong from user space!

2. Strange artifact at 720p30 from macOS
When connecting the TC358743 to a MacBook and configuring the display to 720p30 in macOS, I get a strange artifact box on the left side of the screen, where a portion of the right side of the frame buffer is replicated, but with a green tint. Images at: https://imgur.com/a/5BoVPNk. This artifact is not visible in a different configuration (eg 1080p30 or 720p60). Please note this is not related to my modifications or the removal of the ISP, as I also see the same issue with a test build of yavta. Do you have any idea what might be causing this?

3. Improved reliability
Before removing the ISP, I was seeing a very annoying issue that made me question my whole project: After running my program for a few minutes (probably around an hour), the H264 encoder seemed to experience phases of struggling to keep up with quick movements in the HDMI input image. It would slow down and only catch up with real time once the HDMI input was a still image without movement. This would happen for a minute or so, before the encoder worked fine for a few minutes, just to show the same issue a few moments later. Probably the ISP can not quite keep up with the double invocation per frame for longer periods of time?

4. Lack of contrast
It seems the format conversion in the ISP leads to a reduction of colours in the output image. Now that I have removed the ISP from the render path, I no longer see the issue on the HDMI output, but the H264 encoded output still does not faithfully represent the input colours. For example, the background colour (this slight red-ish tint) of the Raspberry Pi forums is indistinguishable from the white boxes around posts in the H264 encoded output (which as far as I understand still goes through the ISP as part of the H264 encoding). Is there anything I can do to improve colour accuracy here?

Anyway, I feel I'm very close to the solution I hoped for so thanks again so much @6by9 for your help :) !


Kernel logs:

Code: Select all

[  688.122565] ------------[ cut here ]------------
[  688.122707] WARNING: CPU: 0 PID: 29175 at drivers/media/common/videobuf2/videobuf2-core.c:927 vb2_buffer_done+0x1f0/0x250 [videobuf2_common]
[  688.122712] Modules linked in: aes_arm aes_generic cmac bnep hci_uart btbcm bluetooth ecdh_generic ecc libaes 8021q garp stp llc tc358743 cdc_ether brcmfmac r8152 brcmutil sha256_generic libsha256 cfg80211 rfkill raspberrypi_hwmon i2c_mux_pinctrl i2c_mux bcm2835_unicam v4l2_dv_timings v4l2_fwnode bcm2835_codec(C) i2c_bcm2835 bcm2835_v4l2(C) v4l2_mem2mem bcm2835_isp(C) snd_bcm2835(C) bcm2835_mmal_vchiq(C) videobuf2_dma_contig videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common snd_pcm snd_timer snd videodev mc vc_sm_cma(C) uio_pdrv_genirq uio fixed ip_tables x_tables ipv6
[  688.122867] CPU: 0 PID: 29175 Comm: find Tainted: G         C        5.4.83+ #1379
[  688.122871] Hardware name: BCM2835
[  688.122875] Backtrace: 
[  688.122905] [<c0015640>] (dump_backtrace) from [<c00159b0>] (show_stack+0x20/0x24)
[  688.122920]  r7:0000039f r6:00000009 r5:bf229344 r4:bf22eb08
[  688.122956] [<c0015990>] (show_stack) from [<c078ada4>] (dump_stack+0x20/0x28)
[  688.122985] [<c078ad84>] (dump_stack) from [<c0023f38>] (__warn+0xd0/0x104)
[  688.123002] [<c0023e68>] (__warn) from [<c00242e8>] (warn_slowpath_fmt+0x68/0xc0)
[  688.123013]  r7:00000009 r6:bf229344 r5:0000039f r4:bf22eb08
[  688.123068] [<c0024284>] (warn_slowpath_fmt) from [<bf229344>] (vb2_buffer_done+0x1f0/0x250 [videobuf2_common])
[  688.123085]  r9:00009189 r8:00000000 r7:00000005 r6:cf3c0388 r5:00000000 r4:cdf46400
[  688.123180] [<bf229154>] (vb2_buffer_done [videobuf2_common]) from [<bf30b848>] (unicam_isr+0x108/0x354 [bcm2835_unicam])
[  688.123193]  r9:00009189 r8:00000000 r7:bf312260 r6:cf3c0000 r5:00000000 r4:cf3c0000
[  688.123242] [<bf30b740>] (unicam_isr [bcm2835_unicam]) from [<c006ab28>] (__handle_irq_event_percpu+0x48/0x1dc)
[  688.123254]  r10:d0cd7d30 r9:00000000 r8:0000003f r7:00000000 r6:d145e200 r5:c0b30010
[  688.123259]  r4:ce80b4c0
[  688.123276] [<c006aae0>] (__handle_irq_event_percpu) from [<c006acf4>] (handle_irq_event_percpu+0x38/0x90)
[  688.123290]  r10:cdf27700 r9:d0cd6000 r8:d145a000 r7:00000000 r6:d145e200 r5:c0b30010
[  688.123296]  r4:c0a97028
[  688.123313] [<c006acbc>] (handle_irq_event_percpu) from [<c006ad84>] (handle_irq_event+0x38/0x4c)
[  688.123321]  r6:00000001 r5:c0b30010 r4:d145e200
[  688.123341] [<c006ad4c>] (handle_irq_event) from [<c006eebc>] (handle_level_irq+0xbc/0x12c)
[  688.123348]  r5:c0b30010 r4:d145e200
[  688.123363] [<c006ee00>] (handle_level_irq) from [<c0069ce4>] (generic_handle_irq+0x30/0x44)
[  688.123370]  r5:c0b30010 r4:0000003f
[  688.123386] [<c0069cb4>] (generic_handle_irq) from [<c006a3c4>] (__handle_domain_irq+0x58/0xb8)
[  688.123407] [<c006a36c>] (__handle_domain_irq) from [<c000a0d0>] (bcm2835_handle_irq+0x3c/0x44)
[  688.123419]  r9:d0cd6000 r8:d13dd980 r7:d0cd7e0c r6:ffffffff r5:c0a979e0 r4:d0cd7dd8
[  688.123431] [<c000a094>] (bcm2835_handle_irq) from [<c0009a1c>] (__irq_svc+0x5c/0x7c)
[  688.123438] Exception stack(0xd0cd7dd8 to 0xd0cd7e20)
[  688.123446] 7dc0:                                                       d13dd99c 00000000
[  688.123457] 7de0: 60000093 60000093 cdf27700 cdc13d80 ce8e9498 d13dd99c d13dd980 ce8e96c0
[  688.123469] 7e00: cdf27700 d0cd7e4c 00000000 d0cd7e28 c00eac30 c018eb20 a0000013 ffffffff
[  688.123476]  r5:a0000013 r4:c018eb20
[  688.123494] [<c018eaa4>] (vma_link) from [<c01919b0>] (mmap_region+0x358/0x660)
[  688.123507]  r9:ce8e96c0 r8:cdc13d80 r7:00100073 r6:ce8e9490 r5:b6ef9000 r4:ce8e9498
[  688.123519] [<c0191658>] (mmap_region) from [<c0191fd4>] (do_mmap+0x31c/0x534)
[  688.123532]  r10:cdf27700 r9:000081a4 r8:cf1ddc80 r7:00000073 r6:b6ef9000 r5:00000012
[  688.123536]  r4:00002000
[  688.123555] [<c0191cb8>] (do_mmap) from [<c0173b94>] (vm_mmap_pgoff+0xd0/0x100)
[  688.123567]  r10:cdf2773c r9:00002000 r8:b6ef9000 r7:00000003 r6:00000000 r5:c0a97028
[  688.123572]  r4:d0cd7f24
[  688.123582] [<c0173ac4>] (vm_mmap_pgoff) from [<c018f74c>] (ksys_mmap_pgoff+0xc8/0x110)
[  688.123594]  r10:00000000 r9:d0cd6000 r8:cf1ddc80 r7:00000003 r6:00002000 r5:b6ef9000
[  688.123598]  r4:00000012
[  688.123610] [<c018f684>] (ksys_mmap_pgoff) from [<c018f7c0>] (sys_mmap_pgoff+0x2c/0x34)
[  688.123621]  r8:c00091a4 r7:000000c0 r6:00000002 r5:00000070 r4:00000003
[  688.123632] [<c018f794>] (sys_mmap_pgoff) from [<c0009000>] (ret_fast_syscall+0x0/0x28)
[  688.123637] Exception stack(0xd0cd7fa8 to 0xd0cd7ff0)
[  688.123646] 7fa0:                   00000003 00000070 b6ef9000 00002000 00000003 00000812
[  688.123658] 7fc0: 00000003 00000070 00000002 000000c0 00000003 bebd1344 b6f6d73c bebd12c4
[  688.123665] 7fe0: 00000000 bebd1124 b6f461f8 b6f5a174
[  688.123672] ---[ end trace 62e7afc919399c61 ]---
[ 4332.364834] 8<--- cut here ---
[ 4332.368734] Unable to handle kernel NULL pointer dereference at virtual address 00000004
[ 4332.373403] pgd = 83fe758f
[ 4332.376890] [00000004] *pgd=0d5f7831, *pte=00000000, *ppte=00000000
[ 4332.380462] Internal error: Oops: 17 [#1] ARM
[ 4332.383684] Modules linked in: aes_arm aes_generic cmac bnep hci_uart btbcm bluetooth ecdh_generic ecc libaes 8021q garp stp llc tc358743 cdc_ether brcmfmac r8152 brcmutil sha256_generic libsha256 cfg80211 rfkill raspberrypi_hwmon i2c_mux_pinctrl i2c_mux bcm2835_unicam v4l2_dv_timings v4l2_fwnode bcm2835_codec(C) i2c_bcm2835 bcm2835_v4l2(C) v4l2_mem2mem bcm2835_isp(C) snd_bcm2835(C) bcm2835_mmal_vchiq(C) videobuf2_dma_contig videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common snd_pcm snd_timer snd videodev mc vc_sm_cma(C) uio_pdrv_genirq uio fixed ip_tables x_tables ipv6
[ 4332.396825] CPU: 0 PID: 8358 Comm: bash Tainted: G        WC        5.4.83+ #1379
[ 4332.401285] Hardware name: BCM2835
[ 4332.406142] PC is at ret_fast_syscall+0x0/0x28
[ 4332.411075] LR is at trace_hardirqs_on+0x4c/0x140
[ 4332.416026] pc : [<c0009000>]    lr : [<c00eac30>]    psr: 60000013
[ 4332.420726] sp : cd5e3fa8  ip : 00000000  fp : 00000000
[ 4332.425840] r10: 00000000  r9 : cd5e2000  r8 : c00091a4
[ 4332.431017] r7 : 000000af  r6 : 00000002  r5 : 00000000  r4 : 000fcef8
[ 4332.435867] r3 : abc3d852  r2 : abc3d852  r1 : 00000000  r0 : 00000000
[ 4332.441166] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[ 4332.446574] Control: 00c5387d  Table: 0de90008  DAC: 00000055
[ 4332.451667] Process bash (pid: 8358, stack limit = 0x0ea95940)
[ 4332.457252] Stack: (0xcd5e3fa8 to 0xcd5e4000)
[ 4332.462852] 3fa0:                   000fcef8 00000000 00000002 bed428d4 00000000 00000008
[ 4332.468301] 3fc0: 000fcef8 00000000 00000002 000000af 00000000 00000001 00000000 0176fde0
[ 4332.474194] 3fe0: 0000014c bed42788 000582f4 b6e32180 60000010 00000002 00000000 00000000
[ 4332.480144] Backtrace: no frame pointer
[ 4332.485767] Code: 00000000 00000000 00000000 00000000 (e5ad0008) 
[ 4332.492211] ---[ end trace 62e7afc919399c62 ]---
[ 6456.990036] 8<--- cut here ---
[ 6456.994547] Unable to handle kernel NULL pointer dereference at virtual address 00000110
[ 6456.997918] pgd = a82209ec
[ 6457.001325] [00000110] *pgd=00000000
[ 6457.004666] Internal error: Oops: 17 [#2] ARM
[ 6457.007555] Modules linked in: aes_arm aes_generic cmac bnep hci_uart btbcm bluetooth ecdh_generic ecc libaes 8021q garp stp llc tc358743 cdc_ether brcmfmac r8152 brcmutil sha256_generic libsha256 cfg80211 rfkill raspberrypi_hwmon i2c_mux_pinctrl i2c_mux bcm2835_unicam v4l2_dv_timings v4l2_fwnode bcm2835_codec(C) i2c_bcm2835 bcm2835_v4l2(C) v4l2_mem2mem bcm2835_isp(C) snd_bcm2835(C) bcm2835_mmal_vchiq(C) videobuf2_dma_contig videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common snd_pcm snd_timer snd videodev mc vc_sm_cma(C) uio_pdrv_genirq uio fixed ip_tables x_tables ipv6
[ 6457.020574] CPU: 0 PID: 14847 Comm: bash Tainted: G      D WC        5.4.83+ #1379
[ 6457.024891] Hardware name: BCM2835
[ 6457.029617] PC is at unlink_anon_vmas+0x188/0x1fc
[ 6457.034419] LR is at 0xf498bc
[ 6457.039044] pc : [<c01962e4>]    lr : [<00f498bc>]    psr: 20000013
[ 6457.043788] sp : cd62be60  ip : cde34390  fp : cd62be94
[ 6457.048765] r10: 00000122  r9 : cde1b99c  r8 : cde1b994
[ 6457.053825] r7 : c0bbe334  r6 : 00000122  r5 : 00000100  r4 : cd405320
[ 6457.058579] r3 : cdfd7d94  r2 : 00000003  r1 : 00000000  r0 : d7c49c20
[ 6457.063761] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[ 6457.069081] Control: 00c5387d  Table: 0eb1c008  DAC: 00000055
[ 6457.074073] Process bash (pid: 14847, stack limit = 0x55875840)
[ 6457.079538] Stack: (0xcd62be60 to 0xcd62c000)
[ 6457.085063] be60: cea67f00 cde1b960 cd62be94 cde1b960 cde1b3c0 b6961000 cd62bec8 00002000
[ 6457.090397] be80: 00000000 00000000 cd62bec4 cd62be98 c01882f0 c0196168 b6800000 cf1ef300
[ 6457.096222] bea0: cf1ef300 cf3faa80 c0a97028 cf3faabc ffffe000 00000000 cd62bf34 cd62bec8
[ 6457.102165] bec0: c0190624 c0188274 cf3faa80 00010000 ffffffff cd62befd 00000001 cd653000
[ 6457.107813] bee0: cd653000 00000008 00000008 d7e4ae8c d7e61fb0 d7e61fd4 d7e61ff8 d7e5dd2c
[ 6457.113945] bf00: d7e5f874 d7e5ddbc d7e5e728 1b33c466 00000000 cf3faa80 00000000 ffffe000
[ 6457.120258] bf20: ffffe000 00000000 cd62bf4c cd62bf38 c002021c c0190548 ceb10000 cf3faa80
[ 6457.126272] bf40: cd62bf74 cd62bf50 c0027118 c00201d8 0000081f c07a8868 b697b0f4 1b33c466
[ 6457.132755] bf60: b697aee8 000000f8 cd62bf94 cd62bf78 c002788c c0026d84 b6f4e6d8 b6f4e6d8
[ 6457.139156] bf80: 00000000 000000f8 cd62bfa4 cd62bf98 c002791c c002784c 00000000 cd62bfa8
[ 6457.145647] bfa0: c0009000 c0027908 b6f4e6d8 b6f4e6d8 00000000 00000000 00000000 34517300
[ 6457.152397] bfc0: b6f4e6d8 b6f4e6d8 00000000 000000f8 00000001 00000000 b6f53120 b6f50000
[ 6457.158832] bfe0: 00000444 bed42cd0 b6e34734 b6ea5944 60000010 00000000 00000000 00000000
[ 6457.165734] Backtrace: 
[ 6457.172507] [<c019615c>] (unlink_anon_vmas) from [<c01882f0>] (free_pgtables+0x88/0xdc)
[ 6457.179512]  r10:00000000 r9:00000000 r8:00002000 r7:cd62bec8 r6:b6961000 r5:cde1b3c0
[ 6457.186807]  r4:cde1b960
[ 6457.193739] [<c0188268>] (free_pgtables) from [<c0190624>] (exit_mmap+0xe8/0x194)
[ 6457.201273]  r9:00000000 r8:ffffe000 r7:cf3faabc r6:c0a97028 r5:cf3faa80 r4:cf1ef300
[ 6457.208560] [<c019053c>] (exit_mmap) from [<c002021c>] (mmput+0x50/0xd4)
[ 6457.216311]  r9:00000000 r8:ffffe000 r6:ffffe000 r5:00000000 r4:cf3faa80
[ 6457.223775] [<c00201cc>] (mmput) from [<c0027118>] (do_exit+0x3a0/0xa7c)
[ 6457.231734]  r5:cf3faa80 r4:ceb10000
[ 6457.239441] [<c0026d78>] (do_exit) from [<c002788c>] (do_group_exit+0x4c/0xbc)
[ 6457.247451]  r7:000000f8
[ 6457.255483] [<c0027840>] (do_group_exit) from [<c002791c>] (__wake_up_parent+0x0/0x30)
[ 6457.263631]  r7:000000f8 r6:00000000 r5:b6f4e6d8 r4:b6f4e6d8
[ 6457.272020] [<c00278fc>] (sys_exit_group) from [<c0009000>] (ret_fast_syscall+0x0/0x28)
[ 6457.280374] Exception stack(0xcd62bfa8 to 0xcd62bff0)
[ 6457.288938] bfa0:                   b6f4e6d8 b6f4e6d8 00000000 00000000 00000000 34517300
[ 6457.297561] bfc0: b6f4e6d8 b6f4e6d8 00000000 000000f8 00000001 00000000 b6f53120 b6f50000
[ 6457.306228] bfe0: 00000444 bed42cd0 b6e34734 b6ea5944
[ 6457.315133] Code: e3520000 1affffe8 ebffff6d eaffffe6 (e24bd028) 
[ 6457.324338] ---[ end trace 62e7afc919399c63 ]---
[ 6457.333978] Fixing recursive fault but reboot is needed!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 12379
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HDMI to CSI-2 via TC358743 on kernel 4.1

Sat Mar 13, 2021 2:00 pm

1) The kernel warnings are known about. The driver hasn't released all the buffers it had been passed, but the framework cleans up anyway. It'll get fixed at some point.

2) This is probably the TC358743 FIFO under/over flowing. You've got video coming in at one rate over HDMI, and being read out at a different rate over CSI2 (always 972Mbit/s per data lane).
The chip has a small FIFO with a trigger point for how full it should be before it starts reading out. That threshold is set at https://github.com/raspberrypi/linux/bl ... 43.c#L1969
Some combinations of resolution and framerate do seem to result in issues in the FIFO. Ideally the trigger point would be dynamically computed, but the formulae to do so are under NDA.

3) Pass.

4) Probably the joys of colourspaces. HDMI is generally sent as RGB, however it can either be full range (0-255) or limited range (16-235) depending on whether it is defined as a CEA mode or not. The TC358743 should be set to always convert back out to full range, but there will be minor inaccuracies.
H264 always encodes as YUV, but that can be using BT601 or BT709 primaries (it changes the conversion coefficients from RGB), and also then also limited or full range. The H264 headers should provide the information on how the data should be interpreted when rendered.
If any one of those conversions or bits of metadata is wrong then you'll see slightly incorrect colours. H264 being lossy doesn't help on keeping all the detail anyway.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Graphics, sound and multimedia”