6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13013
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: [Solved] Adding the new camera to libcamera - imx258

Tue Jun 21, 2022 1:55 pm

sohonomura2020 wrote:
Tue Jun 21, 2022 6:10 am
6by9 wrote:
Mon Jun 20, 2022 1:16 pm
Sorry, it's taken a couple of days to find the time to test this.
imx258 on a CM3 has just worked fine for me.
I'm trying to find a Pi02 to try it on, but there shouldn't be any significant difference between CM3 and Pi02.

I am seeing very definite speckles on the image in a fairly regular pattern as if the shield pixel correction isn't working. The docs I have from Sony on this sensor imply that the SPC table must be programmed into the OTP, so it's possible that that table is blank.
This is the version w/o PD pixels, then the initial value of the registers are slightly different.
As I shown past writing in this thread, please modify some registers in the initial sequence.
Those regular speckles must be erased.
It looks like you've:
- removed registers 0x3052 and 0x4e21.
- changed 0x7b25 from 0x00 to 0x01
- reset 0x7bcd from 1 to 0 in 2104x1560 mode.
Maybe I need to find out the way to distinguish the version of w/ and w/o PD pixels.
Ah, I hadn't appreciated that there were different versions of the sensor when it came to PD shielded pixels.

This is a mainline driver that we're modifying, so if these image differences are down to PD pixels, then we do need to either add an autodetect based on I2C reads, or an alternate device tree compatible string to distinguish between the two. The register configuration then needs adjusting accordingly.

0x3052 defaults to 0x01 (driver set it to 0x00)
0x4e21 defaults to 0x10 (driver set it to 0x14). This appears to be the critical one for the speckled images.

If you have a helpful Sony contact to query these settings with, then that's great. Otherwise I can go back to our contact and see if he can help.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

sohonomura2020
Posts: 67
Joined: Mon Oct 26, 2020 7:30 am

Re: [Solved] Adding the new camera to libcamera - imx258

Wed Jun 22, 2022 8:41 am

Dear 6by9-san
6by9 wrote:
Tue Jun 21, 2022 1:55 pm

Ah, I hadn't appreciated that there were different versions of the sensor when it came to PD shielded pixels.

This is a mainline driver that we're modifying, so if these image differences are down to PD pixels, then we do need to either add an autodetect based on I2C reads, or an alternate device tree compatible string to distinguish between the two. The register configuration then needs adjusting accordingly.

0x3052 defaults to 0x01 (driver set it to 0x00)
0x4e21 defaults to 0x10 (driver set it to 0x14). This appears to be the critical one for the speckled images.

If you have a helpful Sony contact to query these settings with, then that's great. Otherwise I can go back to our contact and see if he can help.
OK, I'll contact Sony FAE guy and clarify how to distinguish the version w/ and w/o PD,
to handle this difference properly.

And I have not succeeded in showing the IMX258 image on Pi Zero 2W, yet.
I have built the kernel locally with the configuration for Pi3+ (kernel7) on both Pi Zero 2W and Pi4.
What processes will be necessary?

sohonomura2020

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13013
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: [Solved] Adding the new camera to libcamera - imx258

Wed Jun 22, 2022 11:22 am

sohonomura2020 wrote:
Wed Jun 22, 2022 8:41 am
6by9 wrote:
Tue Jun 21, 2022 1:55 pm

Ah, I hadn't appreciated that there were different versions of the sensor when it came to PD shielded pixels.

This is a mainline driver that we're modifying, so if these image differences are down to PD pixels, then we do need to either add an autodetect based on I2C reads, or an alternate device tree compatible string to distinguish between the two. The register configuration then needs adjusting accordingly.

0x3052 defaults to 0x01 (driver set it to 0x00)
0x4e21 defaults to 0x10 (driver set it to 0x14). This appears to be the critical one for the speckled images.

If you have a helpful Sony contact to query these settings with, then that's great. Otherwise I can go back to our contact and see if he can help.
OK, I'll contact Sony FAE guy and clarify how to distinguish the version w/ and w/o PD,
to handle this difference properly.
I had to email our contact at Sony, so I tagged a request over IMX258 to that email. We'll see what they come back with.
sohonomura2020 wrote:And I have not succeeded in showing the IMX258 image on Pi Zero 2W, yet.
I have built the kernel locally with the configuration for Pi3+ (kernel7) on both Pi Zero 2W and Pi4.
What processes will be necessary?
It should be identical other than using the bcm2709_defconfig.
IMX258 support (other than the speckles) is all merged into our kernel. Use "sudo rpi-update" to grab the latest kernel and firmware, and then "dtoverlay=imx258" or "dtoverlay=imx258,vcm" should do the necessary.
Then check if /dev/video0 and /dev/v4l-subdev[0|1] exist.
If not, then try "sudo cat /sys/kernel/debug/v4l2-async/pending_async_subdevices" to see if any V4L2 devices are still pending. You should see a list of devices with nothing pending, eg

Code: Select all

root@raspberrypi:~# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices 
imx219 10-0010:
unicam fe801000.csi:
If anything is pending, then it'll show something like:

Code: Select all

root@raspberrypi:~# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices 
unicam fe801000.csi:
 [fwnode] dev=10-0010, node=/soc/i2c0mux/i2c@1/imx219@10/port/endpoint
(Unicam is waiting for the imx219 device to register, except I blacklisted it)

You can also use "modinfo imx258" to show the module information and ensure it has been built. Likewise "modinfo ad5398_vcm" for the VCM driver.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

sohonomura2020
Posts: 67
Joined: Mon Oct 26, 2020 7:30 am

Re: [Solved] Adding the new camera to libcamera - imx258

Thu Jun 23, 2022 5:24 am

6by9 wrote:
Wed Jun 22, 2022 11:22 am
I had to email our contact at Sony, so I tagged a request over IMX258 to that email. We'll see what they come back with.
Just got the reply from the Sony FAE team, and it includes the confidential information, I'll send the material to you directly.

sohonomura2020
Posts: 67
Joined: Mon Oct 26, 2020 7:30 am

Re: [Solved] Adding the new camera to libcamera - imx258

Thu Jun 23, 2022 6:36 am

With executing rpi-update, now I can see the image from IMX258 on Rasoi Zero 2W.
Thank you for your advice.

And maybe this is critical point, I have also wrote the driver for IMX577, I can see the image on RPi4,
I modified bcm2711_defconfig to add imx258 and imx577 but did not bcm2709_defconfig.
This might be the reason camera is not detected on RPI3 or Zero 2W, right?
6by9 wrote:
Wed Jun 22, 2022 11:22 am
sohonomura2020 wrote:And I have not succeeded in showing the IMX258 image on Pi Zero 2W, yet.
I have built the kernel locally with the configuration for Pi3+ (kernel7) on both Pi Zero 2W and Pi4.
What processes will be necessary?
It should be identical other than using the bcm2709_defconfig.
IMX258 support (other than the speckles) is all merged into our kernel. Use "sudo rpi-update" to grab the latest kernel and firmware, and then "dtoverlay=imx258" or "dtoverlay=imx258,vcm" should do the necessary.
Then check if /dev/video0 and /dev/v4l-subdev[0|1] exist.
If not, then try "sudo cat /sys/kernel/debug/v4l2-async/pending_async_subdevices" to see if any V4L2 devices are still pending. You should see a list of devices with nothing pending, eg

Code: Select all

root@raspberrypi:~# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices 
imx219 10-0010:
unicam fe801000.csi:
If anything is pending, then it'll show something like:

Code: Select all

root@raspberrypi:~# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices 
unicam fe801000.csi:
 [fwnode] dev=10-0010, node=/soc/i2c0mux/i2c@1/imx219@10/port/endpoint
(Unicam is waiting for the imx219 device to register, except I blacklisted it)

You can also use "modinfo imx258" to show the module information and ensure it has been built. Likewise "modinfo ad5398_vcm" for the VCM driver.
sohonomura2020

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13013
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: [Solved] Adding the new camera to libcamera - imx258

Thu Jun 23, 2022 8:24 am

sohonomura2020 wrote:
Thu Jun 23, 2022 6:36 am
With executing rpi-update, now I can see the image from IMX258 on Rasoi Zero 2W.
Thank you for your advice.

And maybe this is critical point, I have also wrote the driver for IMX577, I can see the image on RPi4,
I modified bcm2711_defconfig to add imx258 and imx577 but did not bcm2709_defconfig.
This might be the reason camera is not detected on RPI3 or Zero 2W, right?
Almost certainly as the module will not have been built. "modinfo imx577" (or whatever your module is called) should tell you about it if it has been built.
It's more common to use "make menuconfig" or "make xconfig" to go to a UI to choose additional modules. The defconfigs stage normally comes later, although ideally you use "make savedefconfig" to save the currently configured configuration rather than making manual edits (otherwise PhilE complains when the entries are in the wrong order!)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13013
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: [Solved] Adding the new camera to libcamera - imx258

Thu Jun 23, 2022 11:09 am

sohonomura2020 wrote:
Thu Jun 23, 2022 5:24 am
6by9 wrote:
Wed Jun 22, 2022 11:22 am
I had to email our contact at Sony, so I tagged a request over IMX258 to that email. We'll see what they come back with.
Just got the reply from the Sony FAE team, and it includes the confidential information, I'll send the material to you directly.
I haven't received anything as yet, and nor has David P.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

sohonomura2020
Posts: 67
Joined: Mon Oct 26, 2020 7:30 am

Re: [Solved] Adding the new camera to libcamera - imx258

Thu Jun 23, 2022 1:30 pm

6by9 wrote:
Thu Jun 23, 2022 11:09 am
sohonomura2020 wrote:
Thu Jun 23, 2022 5:24 am
6by9 wrote:
Wed Jun 22, 2022 11:22 am
I had to email our contact at Sony, so I tagged a request over IMX258 to that email. We'll see what they come back with.
Just got the reply from the Sony FAE team, and it includes the confidential information, I'll send the material to you directly.
I haven't received anything as yet, and nor has David P.
I just sent the email to David P-san

sohonomura2020

sohonomura2020
Posts: 67
Joined: Mon Oct 26, 2020 7:30 am

Re: [Solved] Adding the new camera to libcamera - imx258

Fri Jun 24, 2022 5:53 am

6by9 wrote:
Thu Jun 23, 2022 8:24 am
sohonomura2020 wrote:
Thu Jun 23, 2022 6:36 am
With executing rpi-update, now I can see the image from IMX258 on Rasoi Zero 2W.
Thank you for your advice.

And maybe this is critical point, I have also wrote the driver for IMX577, I can see the image on RPi4,
I modified bcm2711_defconfig to add imx258 and imx577 but did not bcm2709_defconfig.
This might be the reason camera is not detected on RPI3 or Zero 2W, right?
Almost certainly as the module will not have been built. "modinfo imx577" (or whatever your module is called) should tell you about it if it has been built.
It's more common to use "make menuconfig" or "make xconfig" to go to a UI to choose additional modules. The defconfigs stage normally comes later, although ideally you use "make savedefconfig" to save the currently configured configuration rather than making manual edits (otherwise PhilE complains when the entries are in the wrong order!)
Thank you for the information.

By adding IMX577 in bcm2709_defconfig and rebuilt on Raspi4, I succeeded in operating IMX577 camera on Raspi Zero 2W, though it might be incorrect way.
(I modified bcmrpi_defconfig at the same time but not succeeded in displaying the image on Raspi Zero W.)

I'll study the mechanism more.

Best Regards,

sohonomura2020

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13013
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: [Solved] Adding the new camera to libcamera - imx258

Mon Jul 04, 2022 12:43 pm

IMX258 updates to fix the speckles on Soho Enterprises IMX258 module.
(Apologies for managing to attribute the module to Vision Components in the commit text - brain fail!)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”