[Solved] Adding the new camera to libcamera - imx258

[6by9]

Sorry, it's taken a couple of days to find the time to test this.
imx258 on a CM3 has just worked fine for me.
I'm trying to find a Pi02 to try it on, but there shouldn't be any significant difference between CM3 and Pi02.

I am seeing very definite speckles on the image in a fairly regular pattern as if the shield pixel correction isn't working. The docs I have from Sony on this sensor imply that the SPC table must be programmed into the OTP, so it's possible that that table is blank.

This is the version w/o PD pixels, then the initial value of the registers are slightly different.
As I shown past writing in this thread, please modify some registers in the initial sequence.
Those regular speckles must be erased.

It looks like you've:
- removed registers 0x3052 and 0x4e21.
- changed 0x7b25 from 0x00 to 0x01
- reset 0x7bcd from 1 to 0 in 2104x1560 mode.

Maybe I need to find out the way to distinguish the version of w/ and w/o PD pixels.

Ah, I hadn't appreciated that there were different versions of the sensor when it came to PD shielded pixels.

This is a mainline driver that we're modifying, so if these image differences are down to PD pixels, then we do need to either add an autodetect based on I2C reads, or an alternate device tree compatible string to distinguish between the two. The register configuration then needs adjusting accordingly.

0x3052 defaults to 0x01 (driver set it to 0x00)
0x4e21 defaults to 0x10 (driver set it to 0x14). This appears to be the critical one for the speckled images.

If you have a helpful Sony contact to query these settings with, then that's great. Otherwise I can go back to our contact and see if he can help.

[sohonomura2020]

Dear 6by9-san

Ah, I hadn't appreciated that there were different versions of the sensor when it came to PD shielded pixels.

This is a mainline driver that we're modifying, so if these image differences are down to PD pixels, then we do need to either add an autodetect based on I2C reads, or an alternate device tree compatible string to distinguish between the two. The register configuration then needs adjusting accordingly.

0x3052 defaults to 0x01 (driver set it to 0x00)
0x4e21 defaults to 0x10 (driver set it to 0x14). This appears to be the critical one for the speckled images.

If you have a helpful Sony contact to query these settings with, then that's great. Otherwise I can go back to our contact and see if he can help.

OK, I'll contact Sony FAE guy and clarify how to distinguish the version w/ and w/o PD,
to handle this difference properly.

And I have not succeeded in showing the IMX258 image on Pi Zero 2W, yet.
I have built the kernel locally with the configuration for Pi3+ (kernel7) on both Pi Zero 2W and Pi4.
What processes will be necessary?

sohonomura2020

[6by9]

Ah, I hadn't appreciated that there were different versions of the sensor when it came to PD shielded pixels.

This is a mainline driver that we're modifying, so if these image differences are down to PD pixels, then we do need to either add an autodetect based on I2C reads, or an alternate device tree compatible string to distinguish between the two. The register configuration then needs adjusting accordingly.

0x3052 defaults to 0x01 (driver set it to 0x00)
0x4e21 defaults to 0x10 (driver set it to 0x14). This appears to be the critical one for the speckled images.

If you have a helpful Sony contact to query these settings with, then that's great. Otherwise I can go back to our contact and see if he can help.

OK, I'll contact Sony FAE guy and clarify how to distinguish the version w/ and w/o PD,
to handle this difference properly.

I had to email our contact at Sony, so I tagged a request over IMX258 to that email. We'll see what they come back with.

And I have not succeeded in showing the IMX258 image on Pi Zero 2W, yet.
I have built the kernel locally with the configuration for Pi3+ (kernel7) on both Pi Zero 2W and Pi4.
What processes will be necessary?

It should be identical other than using the bcm2709_defconfig.
IMX258 support (other than the speckles) is all merged into our kernel. Use "sudo rpi-update" to grab the latest kernel and firmware, and then "dtoverlay=imx258" or "dtoverlay=imx258,vcm" should do the necessary.
Then check if /dev/video0 and /dev/v4l-subdev[0|1] exist.
If not, then try "sudo cat /sys/kernel/debug/v4l2-async/pending_async_subdevices" to see if any V4L2 devices are still pending. You should see a list of devices with nothing pending, eg

Code: Select all

root@raspberrypi:~# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices 
imx219 10-0010:
unicam fe801000.csi:

If anything is pending, then it'll show something like:

Code: Select all

root@raspberrypi:~# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices 
unicam fe801000.csi:
 [fwnode] dev=10-0010, node=/soc/i2c0mux/i2c@1/imx219@10/port/endpoint

(Unicam is waiting for the imx219 device to register, except I blacklisted it)

You can also use "modinfo imx258" to show the module information and ensure it has been built. Likewise "modinfo ad5398_vcm" for the VCM driver.

[sohonomura2020]

I had to email our contact at Sony, so I tagged a request over IMX258 to that email. We'll see what they come back with.

Just got the reply from the Sony FAE team, and it includes the confidential information, I'll send the material to you directly.

[sohonomura2020]

With executing rpi-update, now I can see the image from IMX258 on Rasoi Zero 2W.
Thank you for your advice.

And maybe this is critical point, I have also wrote the driver for IMX577, I can see the image on RPi4,
I modified bcm2711_defconfig to add imx258 and imx577 but did not bcm2709_defconfig.
This might be the reason camera is not detected on RPI3 or Zero 2W, right?

And I have not succeeded in showing the IMX258 image on Pi Zero 2W, yet.
I have built the kernel locally with the configuration for Pi3+ (kernel7) on both Pi Zero 2W and Pi4.
What processes will be necessary?

It should be identical other than using the bcm2709_defconfig.
IMX258 support (other than the speckles) is all merged into our kernel. Use "sudo rpi-update" to grab the latest kernel and firmware, and then "dtoverlay=imx258" or "dtoverlay=imx258,vcm" should do the necessary.
Then check if /dev/video0 and /dev/v4l-subdev[0|1] exist.
If not, then try "sudo cat /sys/kernel/debug/v4l2-async/pending_async_subdevices" to see if any V4L2 devices are still pending. You should see a list of devices with nothing pending, eg

Code: Select all

root@raspberrypi:~# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices 
imx219 10-0010:
unicam fe801000.csi:

If anything is pending, then it'll show something like:

Code: Select all

root@raspberrypi:~# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices 
unicam fe801000.csi:
 [fwnode] dev=10-0010, node=/soc/i2c0mux/i2c@1/imx219@10/port/endpoint

(Unicam is waiting for the imx219 device to register, except I blacklisted it)

You can also use "modinfo imx258" to show the module information and ensure it has been built. Likewise "modinfo ad5398_vcm" for the VCM driver.

sohonomura2020

[6by9]

With executing rpi-update, now I can see the image from IMX258 on Rasoi Zero 2W.
Thank you for your advice.

And maybe this is critical point, I have also wrote the driver for IMX577, I can see the image on RPi4,
I modified bcm2711_defconfig to add imx258 and imx577 but did not bcm2709_defconfig.
This might be the reason camera is not detected on RPI3 or Zero 2W, right?

Almost certainly as the module will not have been built. "modinfo imx577" (or whatever your module is called) should tell you about it if it has been built.
It's more common to use "make menuconfig" or "make xconfig" to go to a UI to choose additional modules. The defconfigs stage normally comes later, although ideally you use "make savedefconfig" to save the currently configured configuration rather than making manual edits (otherwise PhilE complains when the entries are in the wrong order!)

[6by9]

I had to email our contact at Sony, so I tagged a request over IMX258 to that email. We'll see what they come back with.

Just got the reply from the Sony FAE team, and it includes the confidential information, I'll send the material to you directly.

I haven't received anything as yet, and nor has David P.

[sohonomura2020]

I had to email our contact at Sony, so I tagged a request over IMX258 to that email. We'll see what they come back with.

Just got the reply from the Sony FAE team, and it includes the confidential information, I'll send the material to you directly.

I haven't received anything as yet, and nor has David P.

I just sent the email to David P-san

sohonomura2020

[sohonomura2020]

With executing rpi-update, now I can see the image from IMX258 on Rasoi Zero 2W.
Thank you for your advice.

And maybe this is critical point, I have also wrote the driver for IMX577, I can see the image on RPi4,
I modified bcm2711_defconfig to add imx258 and imx577 but did not bcm2709_defconfig.
This might be the reason camera is not detected on RPI3 or Zero 2W, right?

Almost certainly as the module will not have been built. "modinfo imx577" (or whatever your module is called) should tell you about it if it has been built.
It's more common to use "make menuconfig" or "make xconfig" to go to a UI to choose additional modules. The defconfigs stage normally comes later, although ideally you use "make savedefconfig" to save the currently configured configuration rather than making manual edits (otherwise PhilE complains when the entries are in the wrong order!)

Thank you for the information.

By adding IMX577 in bcm2709_defconfig and rebuilt on Raspi4, I succeeded in operating IMX577 camera on Raspi Zero 2W, though it might be incorrect way.
(I modified bcmrpi_defconfig at the same time but not succeeded in displaying the image on Raspi Zero W.)

I'll study the mechanism more.

Best Regards,

sohonomura2020

[6by9]

IMX258 updates to fix the speckles on Soho Enterprises IMX258 module.
(Apologies for managing to attribute the module to Vision Components in the commit text - brain fail!)