-
- Posts: 42
- Joined: Fri Jun 12, 2020 10:19 pm
Getting Arducam Global Shutter Camera to Work w/ROS
So, I was wanting to preform Visual Inertial Navigation using the OpenVINS library (https://github.com/rpng/open_vins) on an RPI-3B+ running Ubuntu 16.04 and using ROS kinetic. I realised that in order to acheive optimal results, I needed a camera which had global shutter instead of rolling shutter. Which my rpi v1 camera could not do. Since I wanted a relativley cheap monochrome camera (<40$), this lead me to the Arducam Global Shutter Camera (https://www.amazon.com/Arducam-Shutter- ... B07V215NHN), which seemed like a nice fit, so I went along and bought it.
Unfortunatly, when I look at the github for this camera, it does not seem to be interfacing with ros, and more just standard python script. And since im not one for making ROS wrappers unless I have to, I was wondering if there was any way to use this camera and implement it into publishing its data onto ROS topics. It specifically said it was not compatible with normal rpi libraries, so that throws most hope out...
But, on another library for a different product made by the same manufacturer, a USB Camera Hat (https://github.com/ArduCAM/ArduCAM_USB_ ... m_usb2_ros) for the pi, they mentioned support for this camera and linked it to a ROS compatible library. Now, while I do not want to buy a seperate hat just to access ROS, this leads me to beleive it is compatible without the hat entirley. So, I was wondering if there are existing libraries for this, and ive been searching in the wrong place, or if I have to make my own, and how I could do that. Or maybe its just not possible without the hat, then do link me to a better global shutter monochrome camera I could use for my application.
In any case, thank you for reading, and all input is appreciated. =)
EDIT: I have found a library (https://github.com/tiralonghipol/raspicam_stereo) for this camera yet it is intended for another product (https://www.arducam.com/product/b0195-s ... pberry-pi/) made by this manufacturer but it is closer in structure to what I am looking for. It was made as a ros wrapper for a stereo hat on the rpi, not a USB hat, so we got that going for us. There are signifigantly less files in the library so it should be easier to just cut the stereo part out. Only problem is im not really a ROS programmer and cant utilize this, maybe one of you guys could, but I understand how asking some random strangers to do something for you on the internet is a bit hopefull, but it is an update none-the-less.
Unfortunatly, when I look at the github for this camera, it does not seem to be interfacing with ros, and more just standard python script. And since im not one for making ROS wrappers unless I have to, I was wondering if there was any way to use this camera and implement it into publishing its data onto ROS topics. It specifically said it was not compatible with normal rpi libraries, so that throws most hope out...
But, on another library for a different product made by the same manufacturer, a USB Camera Hat (https://github.com/ArduCAM/ArduCAM_USB_ ... m_usb2_ros) for the pi, they mentioned support for this camera and linked it to a ROS compatible library. Now, while I do not want to buy a seperate hat just to access ROS, this leads me to beleive it is compatible without the hat entirley. So, I was wondering if there are existing libraries for this, and ive been searching in the wrong place, or if I have to make my own, and how I could do that. Or maybe its just not possible without the hat, then do link me to a better global shutter monochrome camera I could use for my application.
In any case, thank you for reading, and all input is appreciated. =)
EDIT: I have found a library (https://github.com/tiralonghipol/raspicam_stereo) for this camera yet it is intended for another product (https://www.arducam.com/product/b0195-s ... pberry-pi/) made by this manufacturer but it is closer in structure to what I am looking for. It was made as a ros wrapper for a stereo hat on the rpi, not a USB hat, so we got that going for us. There are signifigantly less files in the library so it should be easier to just cut the stereo part out. Only problem is im not really a ROS programmer and cant utilize this, maybe one of you guys could, but I understand how asking some random strangers to do something for you on the internet is a bit hopefull, but it is an update none-the-less.
-
- Raspberry Pi Engineer & Forum Moderator
- Posts: 13009
- Joined: Wed Dec 04, 2013 11:27 am
- Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
How much kernel hacking are you prepared to do?
There's a kernel driver in mainline Linux for that sensor - https://github.com/torvalds/linux/blob/ ... c/ov7251.c
There's also a Rockchip kernel driver at https://github.com/rockchip-linux/kerne ... c/ov7251.c if you need a second reference.
Combine that with a device tree overlay, and you should be able to have a standard V4L2 device producing either V4L2_PIX_FMT_Y10 or V4L2_PIX_FMT_Y10P.
Taking the imx219 overlay and adapting it for the binding doc for ov7251 shouldn't be too involved, mainly as you don't actually have control over the regulators or clocks.
I had looked at picking up one of these modules, but all the suppliers I'd found wanted stupid amounts for shipping.
There's a kernel driver in mainline Linux for that sensor - https://github.com/torvalds/linux/blob/ ... c/ov7251.c
There's also a Rockchip kernel driver at https://github.com/rockchip-linux/kerne ... c/ov7251.c if you need a second reference.
Combine that with a device tree overlay, and you should be able to have a standard V4L2 device producing either V4L2_PIX_FMT_Y10 or V4L2_PIX_FMT_Y10P.
Taking the imx219 overlay and adapting it for the binding doc for ov7251 shouldn't be too involved, mainly as you don't actually have control over the regulators or clocks.
I had looked at picking up one of these modules, but all the suppliers I'd found wanted stupid amounts for shipping.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
-
- Posts: 42
- Joined: Fri Jun 12, 2020 10:19 pm
Re: Getting Arducam Global Shutter Camera to Work w/ROS
I think its safe to day this problem is out of my league when i dont understand a word you said at all hahaha......
Is there a lowlowlowlowlow level tutorial that could pertain to this? Is there way i can utilise in layman's terms the method you described to modify an existing repositiory to work with arducam? like the ubiquity one (https://github.com/UbiquityRobotics/ras ... tic/launch)? Although your response may workd, I would not know for I would not be able to do that with what I currently know. And if theres no way to make things simpler and that is in fact the only approach, what tutorials do you reccomend to look at? If its alot Im wondering if i could get by using a camera with rolling shutter for SLAM. Either way, thank you for your response! =)
Is there a lowlowlowlowlow level tutorial that could pertain to this? Is there way i can utilise in layman's terms the method you described to modify an existing repositiory to work with arducam? like the ubiquity one (https://github.com/UbiquityRobotics/ras ... tic/launch)? Although your response may workd, I would not know for I would not be able to do that with what I currently know. And if theres no way to make things simpler and that is in fact the only approach, what tutorials do you reccomend to look at? If its alot Im wondering if i could get by using a camera with rolling shutter for SLAM. Either way, thank you for your response! =)
-
- Raspberry Pi Engineer & Forum Moderator
- Posts: 13009
- Joined: Wed Dec 04, 2013 11:27 am
- Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Building the kerrnel is documented at https://www.raspberrypi.org/documentati ... uilding.md
You'll want an rpi-5.4.y tree (not the 4.19 that is currently shipping in apt), and after the "make ... defconfig" line you need to run "make menuconfig" and enable the relevant module ("/" to enter search, enter "OV7251" and hit return, it should list one entry with a "(1)" against it, press 1, and then return to select "M") before running the full make.
I'll see if I can find the time to make up a config change and guessed at overlay. Whether it works or not will need someone with a module to test it.
You'll want an rpi-5.4.y tree (not the 4.19 that is currently shipping in apt), and after the "make ... defconfig" line you need to run "make menuconfig" and enable the relevant module ("/" to enter search, enter "OV7251" and hit return, it should list one entry with a "(1)" against it, press 1, and then return to select "M") before running the full make.
I'll see if I can find the time to make up a config change and guessed at overlay. Whether it works or not will need someone with a module to test it.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
-
- Posts: 42
- Joined: Fri Jun 12, 2020 10:19 pm
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Well Ill be more than glad to be your tester considering i have the camera and an RPI-3B+! Also, I was doing some searching online and I found out a couple of more things. The original library made for this camera (https://github.com/ArduCAM/MIPI_Camera/tree/master/RPI) is not compatible with Ubuntu. Now, I dont know much about this new solution you proposed, but its just extra information if it may be important. For all I know what you're suggesting might not use that library at all. But If i have to be on raspian, that would blow because I already built OpenVINS and an IMU node on this one, but sometimes you just gotta deal with it if thats the case
.
And one last thing, I was checking with cheese and and it returned This along with the fact there isnt a /dev/video0 or /dev/video* folder is getting me woried its not detecting the camera. I used multiple cameras, ribbon cables, and oriented them properly (blue taps facing towards black clip), and still cannot get it to register. Like I said, this could be because its not supposed to, but just in case I wanted to let you know because maybe you can use this information better than I can lol. Truly though, youve been a great help for showing me those interesting libraries, they seem to contain some interesting information like the registers on the device. I hope I havent been toooo much of a pain 

And one last thing, I was checking with cheese and
Code: Select all
vcgencmd get_camera
Code: Select all
supported = 1, detected = 0

-
- Raspberry Pi Engineer & Forum Moderator
- Posts: 13009
- Joined: Wed Dec 04, 2013 11:27 am
- Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
The firmware knows how to communicate with ov5647, imx219, or imx477 sensors and no others. Vcgencmd get_camera will only report on sensors the firmware knows about.
There is an alternative driver (bcm2835_unicam) that is a pure Linux driver for the csi2 receiver. It purely writes the frames to ram and has no ae or awb control loops, but it can easily support any sensor for which there is a Linux driver.
Arducam have gone down the route of writing binary blobs to try and do everything from userspace. Whilst this can work, it has exactly the restrictions that you state of not working on all systems.
There is an alternative driver (bcm2835_unicam) that is a pure Linux driver for the csi2 receiver. It purely writes the frames to ram and has no ae or awb control loops, but it can easily support any sensor for which there is a Linux driver.
Arducam have gone down the route of writing binary blobs to try and do everything from userspace. Whilst this can work, it has exactly the restrictions that you state of not working on all systems.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
-
- Posts: 42
- Joined: Fri Jun 12, 2020 10:19 pm
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Ok, nothing to be much worried about then. And if thats the case, I assume the kernel hacking would be its own seperate thing, and we would not utilise any of the libraries i mentioned that are associated with Arducam's stuff, correct? This is fine, I just wanted clarification on where the solution was heading.
So once I have this done, would it build into a ROS node that I could publish to topics with, or is this a starting point to write my own node? Forgive me, for I am inexperienced with this stuff as you know, and if I have to build the ROS node myself, are there some tutorials you could reccomend that would allow me to take advantage of the kernel hacking we did?
So once I have this done, would it build into a ROS node that I could publish to topics with, or is this a starting point to write my own node? Forgive me, for I am inexperienced with this stuff as you know, and if I have to build the ROS node myself, are there some tutorials you could reccomend that would allow me to take advantage of the kernel hacking we did?
-
- Posts: 42
- Joined: Fri Jun 12, 2020 10:19 pm
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Okay, Update:
-------------------------
Im not sure what the kernel hacking was supposed to do, whether it make it work off-the-bat or be a stepping stone to another solution, im assuming its the ladder because you mentioned more than just doing this, but either way here are some updates. So I wanted to see if installing the driver would make the device appear as "detected" because since I modularized the new sensor it would update its firmware, and it still didnt detect it.
Then I thought, "what if I need to reboot it to start the changes", because I hadn't rebooted it yet, And now my RPI is stuck at rainbow screen. I looked up why this is and apparently its due to kernel errors, So I mustve done something wrong. I flashed my card and did it again and this is still the case. I followed the official rpi directions and yours to a tee without any errors, making sure im using the correct version and branch for my RPI3B+, and it still is stuck at rainbow. So Im wondering if now I should Flash it and try again or wait for you to tell me something because it should act like this until I've reached a step yet to be known. But I trust you, as you seem to know what youre doing and other threads I look at show you replying and helping out OP, so I have no doubt that well get this solved!
-------------------------
Im not sure what the kernel hacking was supposed to do, whether it make it work off-the-bat or be a stepping stone to another solution, im assuming its the ladder because you mentioned more than just doing this, but either way here are some updates. So I wanted to see if installing the driver would make the device appear as "detected" because since I modularized the new sensor it would update its firmware, and it still didnt detect it.
Then I thought, "what if I need to reboot it to start the changes", because I hadn't rebooted it yet, And now my RPI is stuck at rainbow screen. I looked up why this is and apparently its due to kernel errors, So I mustve done something wrong. I flashed my card and did it again and this is still the case. I followed the official rpi directions and yours to a tee without any errors, making sure im using the correct version and branch for my RPI3B+, and it still is stuck at rainbow. So Im wondering if now I should Flash it and try again or wait for you to tell me something because it should act like this until I've reached a step yet to be known. But I trust you, as you seem to know what youre doing and other threads I look at show you replying and helping out OP, so I have no doubt that well get this solved!

-
- Raspberry Pi Engineer & Forum Moderator
- Posts: 13009
- Joined: Wed Dec 04, 2013 11:27 am
- Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Sorry, I can't help with your build not working. The steps do work, but obviously something has gone wrong.
I've created a dtoverlay for you in https://github.com/raspberrypi/linux/pull/3714, but it would be nice if it were tested before merging. I've checked that it probes correctly, but not having hardware I have to remove any I2C reads from the driver to stop it aborting on those.
I'm also going to move this thread to the camera board as I believe HermanSW has the Arducam hardware and may be able to test.
I've created a dtoverlay for you in https://github.com/raspberrypi/linux/pull/3714, but it would be nice if it were tested before merging. I've checked that it probes correctly, but not having hardware I have to remove any I2C reads from the driver to stop it aborting on those.
I'm also going to move this thread to the camera board as I believe HermanSW has the Arducam hardware and may be able to test.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
I did get two ov9281 cameras from Lee, this model:6by9 wrote: ↑Tue Jul 07, 2020 3:35 pmSorry, I can't help with your build not working. The steps do work, but obviously something has gone wrong.
I've created a dtoverlay for you in https://github.com/raspberrypi/linux/pull/3714, but it would be nice if it were tested before merging. I've checked that it probes correctly, but not having hardware I have to remove any I2C reads from the driver to stop it aborting on those.
I'm also going to move this thread to the camera board as I believe HermanSW has the Arducam hardware and may be able to test.
https://www.uctronics.com/arducam-ov928 ... amera.html
I killed one in attempting to use both with Arducam Stereo hat because of lack of soldering skills, but the other works:
viewtopic.php?t=249483#p1529108
So what should I do, which steps?
P.S:
On the topic of this thread, I talked with my son working as student assistant at robotic research institute for 18 months, and he said all their ROS systems are Ububtu based. In theory it would be possible to run ROS on Raspbian, but that is likely to be much harder than to run Ubuntu based.
Can the dtoverlay you created be used under Ubuntu installed on Raspberry Pi4B as well?
https://hermann-sw.github.io/planar_graph_playground
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
-
- Raspberry Pi Engineer & Forum Moderator
- Posts: 13009
- Joined: Wed Dec 04, 2013 11:27 am
- Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Thanks. Sorry, but all the Arducam suppliers I've found want to charge an absolute arm and a leg for shipping, otherwise I would have probably have just bought one.HermannSW wrote: ↑Wed Jul 08, 2020 4:11 pmI did get two ov9281 cameras from Lee, this model:6by9 wrote: ↑Tue Jul 07, 2020 3:35 pmSorry, I can't help with your build not working. The steps do work, but obviously something has gone wrong.
I've created a dtoverlay for you in https://github.com/raspberrypi/linux/pull/3714, but it would be nice if it were tested before merging. I've checked that it probes correctly, but not having hardware I have to remove any I2C reads from the driver to stop it aborting on those.
I'm also going to move this thread to the camera board as I believe HermanSW has the Arducam hardware and may be able to test.
https://www.uctronics.com/arducam-ov928 ... amera.html
I killed one in attempting to use both with Arducam Stereo hat because of lack of soldering skills, but the other works:
viewtopic.php?t=249483#p1529108
So what should I do, which steps?
You need to rebuild the kernel - https://www.raspberrypi.org/documentati ... uilding.md
You'll want to clone https://github.com/6by9/linux/ and checkout the ov7251 branch, not https://github.com/raspberrypi/linux
Please do it on a spare SD card rather than an important one. Building natively on a Pi4 should take around 40mins.
Add "dtoverlay=ov7251" to /boot/config.txt and you should get /dev/video0 (and potentially a /dev/video1) with v4l2-ctl -V listing 640x480 Y10P by default.
Clone https://github.com/6by9/yavta/, run make, and hopefully "yavta c5000 -m /dev/video0" should give you pictures on the HDMI output.
It's just a Linux kernel driver, so anything that builds our kernel should be able to use it.HermannSW wrote:P.S:
On the topic of this thread, I talked with my son working as student assistant at robotic research institute for 18 months, and he said all their ROS systems are Ububtu based. In theory it would be possible to run ROS on Raspbian, but that is likely to be much harder than to run Ubuntu based.
Can the dtoverlay you created be used under Ubuntu installed on Raspberry Pi4B as well?
The driver itself is from the mainline Linux tree, but the drivers are often only ever tested on a single platform and so have quirks. Without a module I've had to make a few guesses on configuration.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
-
- Posts: 42
- Joined: Fri Jun 12, 2020 10:19 pm
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Oh. My. Gosh. I am at a loss for words. You dont know how happy youve made me. Thank you guys so much it seems like you did a lot of work just out of the sheer kindness of your heart. open source code is truly amazing. I dont know if i can be much of a help, but if I get it to work ill let you know so your commit can have more backing. I managed to install the modules too, so I should get back to you very soon. I cant express how greatfull I am. Truly, thank you.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
I built a SD card freshly two days ago, with update+upgrade+rpi-update.6by9 wrote: ↑Wed Jul 08, 2020 4:33 pmYou need to rebuild the kernel - https://www.raspberrypi.org/documentati ... uilding.md
You'll want to clone https://github.com/6by9/linux/ and checkout the ov7251 branch, not https://github.com/raspberrypi/linux
Please do it on a spare SD card rather than an important one. Building natively on a Pi4 should take around 40mins.
Add "dtoverlay=ov7251" to /boot/config.txt and you should get /dev/video0 (and potentially a /dev/video1) with v4l2-ctl -V listing 640x480 Y10P by default.
Clone https://github.com/6by9/yavta/, run make, and hopefully "yavta c5000 -m /dev/video0" should give you pictures on the HDMI output.
Before building kernel I wanted to test whether dtoverlay works fine with v2 camera.
Without
Code: Select all
dtoverlay=imx219
core_freq_min=250
Adding both line to /boot/config.txt and rebooting, I would have expected v2 camera to work with qcam, but unlike HQ camera qcam does not start application window, but opens a "Select Camera" dialog instead:
Code: Select all
pi@raspberrypi:~ $ libcamera/build/src/qcam/qcam
libEGL warning: DRI3: failed to query the version
libEGL warning: DRI2: failed to authenticate
[0:06:39.473625462] [1251] INFO IPAManager ipa_manager.cpp:136 libcamera is not installed. Adding '/home/pi/libcamera/build/src/ipa' to the IPA search path
[0:06:39.477702881] [1251] INFO Camera camera_manager.cpp:283 libcamera v0.0.0+1573-61f4296a-dirty
qt.qpa.xcb: QXcbConnection: XCB error: 1 (BadRequest), sequence: 412, resource id: 408, major code: 155 (Unknown), minor code: 1
Code: Select all
pi@raspberrypi:~ $ ls /dev/video*
/dev/video10 /dev/video11 /dev/video12 /dev/video13 /dev/video14 /dev/video15 /dev/video16
pi@raspberrypi:~ $
https://hermann-sw.github.io/planar_graph_playground
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
-
- Raspberry Pi Engineer & Forum Moderator
- Posts: 13009
- Joined: Wed Dec 04, 2013 11:27 am
- Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
rpi-update doesn't have https://github.com/raspberrypi/linux/pull/3706 built in yet.HermannSW wrote: ↑Thu Jul 09, 2020 9:51 amI built a SD card freshly two days ago, with update+upgrade+rpi-update.
Before building kernel I wanted to test whether dtoverlay works fine with v2 camera.
Withoutin /boot/config.txt raspivid -t 0 works.Code: Select all
dtoverlay=imx219 core_freq_min=250
Adding both line to /boot/config.txt and rebooting, I would have expected v2 camera to work with qcam, but unlike HQ camera qcam does not start application window, but opens a "Select Camera" dialog instead:What can be reason for this?Code: Select all
pi@raspberrypi:~ $ libcamera/build/src/qcam/qcam libEGL warning: DRI3: failed to query the version libEGL warning: DRI2: failed to authenticate [0:06:39.473625462] [1251] INFO IPAManager ipa_manager.cpp:136 libcamera is not installed. Adding '/home/pi/libcamera/build/src/ipa' to the IPA search path [0:06:39.477702881] [1251] INFO Camera camera_manager.cpp:283 libcamera v0.0.0+1573-61f4296a-dirty qt.qpa.xcb: QXcbConnection: XCB error: 1 (BadRequest), sequence: 412, resource id: 408, major code: 155 (Unknown), minor code: 1
Or can I try to build new kernel for ov7251 testing (and ov9281?) without libcamera working for v2 camera connected?Code: Select all
pi@raspberrypi:~ $ ls /dev/video* /dev/video10 /dev/video11 /dev/video12 /dev/video13 /dev/video14 /dev/video15 /dev/video16 pi@raspberrypi:~ $
You've probably got
Code: Select all
[ 5.132117] unicam fe801000.csi: subdevice /soc/i2c0mux/i2c@1/imx219@10: unsupported bus type 2
[ 5.132137] unicam fe801000.csi: Failed to connect subdevs
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Thanks, yes I got those lines:
I know that I successfully used v2 camera with imx219 overlay in the past.
So I will now try to build kernel.
Code: Select all
pi@raspberrypi:~ $ dmesg | grep unicam
[ 4.385822] unicam fe801000.csi: subdevice /soc/i2c0mux/i2c@1/imx219@10: unsupported bus type 2
[ 4.385842] unicam fe801000.csi: Failed to connect subdevs
[ 4.387130] unicam: probe of fe801000.csi failed with error -22
pi@raspberrypi:~ $
So I will now try to build kernel.
https://hermann-sw.github.io/planar_graph_playground
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
-
- Raspberry Pi Engineer & Forum Moderator
- Posts: 13009
- Joined: Wed Dec 04, 2013 11:27 am
- Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
I'd updated the Unicam driver to use the (nearly) upstreamed version, and that does a couple of device tree things in a slightly different way. Unfortunately I missed imx219 in the patch fixing up the overlays, and that's what that PR rectifies.HermannSW wrote: ↑Thu Jul 09, 2020 12:13 pmThanks, yes I got those lines:I know that I successfully used v2 camera with imx219 overlay in the past.Code: Select all
pi@raspberrypi:~ $ dmesg | grep unicam [ 4.385822] unicam fe801000.csi: subdevice /soc/i2c0mux/i2c@1/imx219@10: unsupported bus type 2 [ 4.385842] unicam fe801000.csi: Failed to connect subdevs [ 4.387130] unicam: probe of fe801000.csi failed with error -22 pi@raspberrypi:~ $
So I will now try to build kernel.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Short update -- the build is victim of SD card perfromance.
I started the build more than 2 hours ago, but is still not completed.
I will just wait until build is complete and then test:
I started the build more than 2 hours ago, but is still not completed.
I will just wait until build is complete and then test:
Code: Select all
pi@raspberrypi:~ $ ps -ef | grep make | grep zImage
pi 3317 1154 0 15:13 pts/0 00:00:00 make -j4 zImage modules dtbs
pi@raspberrypi:~ $ date
Thu 09 Jul 2020 05:21:08 PM CEST
pi@raspberrypi:~ $
https://hermann-sw.github.io/planar_graph_playground
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
Re: Getting Arducam Global Shutter Camera to Work w/ROS
The build did finish after 2.5h:6by9 wrote: ↑Wed Jul 08, 2020 4:33 pmThanks. Sorry, but all the Arducam suppliers I've found want to charge an absolute arm and a leg for shipping, otherwise I would have probably have just bought one.
You need to rebuild the kernel - https://www.raspberrypi.org/documentati ... uilding.md
You'll want to clone https://github.com/6by9/linux/ and checkout the ov7251 branch, not https://github.com/raspberrypi/linux
Please do it on a spare SD card rather than an important one. Building natively on a Pi4 should take around 40mins.
Add "dtoverlay=ov7251" to /boot/config.txt and you should get /dev/video0 (and potentially a /dev/video1) with v4l2-ctl -V listing 640x480 Y10P by default.
Clone https://github.com/6by9/yavta/, run make, and hopefully "yavta c5000 -m /dev/video0" should give you pictures on the HDMI output.
Code: Select all
pi@raspberrypi:~/linux $ time make -j4 zImage modules dtbs
...
LD [M] sound/usb/snd-usb-audio.ko
LD [M] sound/usb/snd-usbmidi-lib.ko
real 157m20.607s
user 167m0.616s
sys 25m23.373s
pi@raspberrypi:~/linux $
Code: Select all
INSTALL drivers/media/i2c/ov5647.ko
INSTALL drivers/media/i2c/ov7251.ko
INSTALL drivers/media/i2c/ov7640.ko
INSTALL drivers/media/i2c/ov9281.ko
After adding "dtoverlay=ov7251" to /boot/config.txt and reboot, video 0 and 1 appear:
Code: Select all
pi@raspberrypi:~ $ ls -l /dev/video*
crw-rw----+ 1 root video 81, 7 Jul 9 18:36 /dev/video0
crw-rw----+ 1 root video 81, 8 Jul 9 18:36 /dev/video1
crw-rw----+ 1 root video 81, 4 Jul 9 18:36 /dev/video10
crw-rw----+ 1 root video 81, 5 Jul 9 18:36 /dev/video11
crw-rw----+ 1 root video 81, 6 Jul 9 18:36 /dev/video12
crw-rw----+ 1 root video 81, 0 Jul 9 18:36 /dev/video13
crw-rw----+ 1 root video 81, 1 Jul 9 18:36 /dev/video14
crw-rw----+ 1 root video 81, 2 Jul 9 18:36 /dev/video15
crw-rw----+ 1 root video 81, 3 Jul 9 18:36 /dev/video16
pi@raspberrypi:~ $
As you said, 640x480 Y10P gets reported:
Code: Select all
pi@raspberrypi:~ $ v4l2-ctl -V
Format Video Capture:
Width/Height : 640/480
Pixel Format : 'Y10P' (10-bit Greyscale (MIPI Packed))
Field : None
Bytes per Line : 800
Size Image : 384000
Colorspace : sRGB
Transfer Function : sRGB
YCbCr/HSV Encoding: ITU-R 601
Quantization : Full Range
Flags :
pi@raspberrypi:~ $
I did build yavta, but the command you proposed failed:
Code: Select all
pi@raspberrypi:~/yavta $ ./yavta c5000 -m /dev/video0
Error opening device c5000: No such file or directory (2).
pi@raspberrypi:~/yavta $
I reduced number of frames to capture, still timeout 2 frames before end:
Code: Select all
pi@raspberrypi:~/yavta $ ./yavta --capture=10 -n 3 --encode-to=file.h264 -f UYVY -m -T /dev/video0
We're encoding to file.h264
Device /dev/video0 opened.
Device `unicam' on `platform:fe801000.csi' (driver 'unicam') is a video capture (without mplanes) device.
stride is 0
stride is now 1280
Video format set: Y10P (50303159) 640x480 (stride 1280) field none buffer size 614400
Video format: Y10P (50303159) 640x480 (stride 1280) field none buffer size 614400
Current frame rate: 100/3000
Unsupported encoding
3 buffers requested, V4L2 returned 3 bufs.
length: 614400 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xb3264000.
Importing DMABUF 6 into VCSM...
...done. vcsm_handle 40960
Exported buffer 0 to dmabuf 6, vcsm handle 40960
length: 614400 offset: 614400 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xb31ce000.
Importing DMABUF 7 into VCSM...
...done. vcsm_handle 45056
Exported buffer 1 to dmabuf 7, vcsm handle 45056
length: 614400 offset: 1228800 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xb3138000.
Importing DMABUF 8 into VCSM...
...done. vcsm_handle 49152
Exported buffer 2 to dmabuf 8, vcsm handle 49152
0 (0) [-] none 0 614400 B 1043.300377 1043.309960 93.153 fps ts mono/EoF
1 (1) [-] none 1 614400 B 1043.333706 1043.343286 30.004 fps ts mono/EoF
2 (2) [-] none 2 614400 B 1043.367034 1043.376613 30.005 fps ts mono/EoF
3 (0) [-] none 3 614400 B 1043.400362 1043.410089 30.005 fps ts mono/EoF
4 (1) [-] none 4 614400 B 1043.433691 1043.443281 30.004 fps ts mono/EoF
5 (2) [-] none 5 614400 B 1043.467018 1043.476608 30.006 fps ts mono/EoF
6 (0) [-] none 6 614400 B 1043.500346 1043.509936 30.005 fps ts mono/EoF
7 (1) [-] none 7 614400 B 1043.533675 1043.543264 30.004 fps ts mono/EoF
8 (2) [-] none 8 614400 B 1043.567002 1043.576592 30.006 fps ts mono/EoF
select timeout
pi@raspberrypi:~/yavta $
https://hermann-sw.github.io/planar_graph_playground
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
-
- Raspberry Pi Engineer & Forum Moderator
- Posts: 13009
- Joined: Wed Dec 04, 2013 11:27 am
- Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Typo - missing a hyphen.HermannSW wrote: ↑Thu Jul 09, 2020 4:56 pmI did build yavta, but the command you proposed failed:Code: Select all
pi@raspberrypi:~/yavta $ ./yavta c5000 -m /dev/video0 Error opening device c5000: No such file or directory (2). pi@raspberrypi:~/yavta $
Code: Select all
./yavta -c5000 -m /dev/video0
HermannSW wrote:Then I tried the command from README.md, and it fails after 998 frames with timeout.
I reduced number of frames to capture, still timeout 2 frames before end:What to do next?Code: Select all
pi@raspberrypi:~/yavta $ ./yavta --capture=10 -n 3 --encode-to=file.h264 -f UYVY -m -T /dev/video0 We're encoding to file.h264 Device /dev/video0 opened. Device `unicam' on `platform:fe801000.csi' (driver 'unicam') is a video capture (without mplanes) device. stride is 0 stride is now 1280 Video format set: Y10P (50303159) 640x480 (stride 1280) field none buffer size 614400 Video format: Y10P (50303159) 640x480 (stride 1280) field none buffer size 614400 Current frame rate: 100/3000 Unsupported encoding 3 buffers requested, V4L2 returned 3 bufs. length: 614400 offset: 0 timestamp type/source: mono/EoF Buffer 0/0 mapped at address 0xb3264000. Importing DMABUF 6 into VCSM... ...done. vcsm_handle 40960 Exported buffer 0 to dmabuf 6, vcsm handle 40960 length: 614400 offset: 614400 timestamp type/source: mono/EoF Buffer 1/0 mapped at address 0xb31ce000. Importing DMABUF 7 into VCSM... ...done. vcsm_handle 45056 Exported buffer 1 to dmabuf 7, vcsm handle 45056 length: 614400 offset: 1228800 timestamp type/source: mono/EoF Buffer 2/0 mapped at address 0xb3138000. Importing DMABUF 8 into VCSM... ...done. vcsm_handle 49152 Exported buffer 2 to dmabuf 8, vcsm handle 49152 0 (0) [-] none 0 614400 B 1043.300377 1043.309960 93.153 fps ts mono/EoF 1 (1) [-] none 1 614400 B 1043.333706 1043.343286 30.004 fps ts mono/EoF 2 (2) [-] none 2 614400 B 1043.367034 1043.376613 30.005 fps ts mono/EoF 3 (0) [-] none 3 614400 B 1043.400362 1043.410089 30.005 fps ts mono/EoF 4 (1) [-] none 4 614400 B 1043.433691 1043.443281 30.004 fps ts mono/EoF 5 (2) [-] none 5 614400 B 1043.467018 1043.476608 30.006 fps ts mono/EoF 6 (0) [-] none 6 614400 B 1043.500346 1043.509936 30.005 fps ts mono/EoF 7 (1) [-] none 7 614400 B 1043.533675 1043.543264 30.004 fps ts mono/EoF 8 (2) [-] none 8 614400 B 1043.567002 1043.576592 30.006 fps ts mono/EoF select timeout pi@raspberrypi:~/yavta $
Code: Select all
Unsupported encoding
The alternative to just see whether the thing streams is
Code: Select all
v4l2-ctl --stream-mmap=3 --stream-count=100 --stream-to=/dev/null
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
-
- Raspberry Pi Engineer & Forum Moderator
- Posts: 13009
- Joined: Wed Dec 04, 2013 11:27 am
- Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
OK, yavta updated to include the mono formats
The fact you got
would imply that it got 9 buffers in, and it's probably a lockup in shutdown as you had --capture=10.
Sounding promising anyway. It'll be nice to see some pictures from it though.
The fact you got
Code: Select all
0 (0) [-] none 0 614400 B 1043.300377 1043.309960 93.153 fps ts mono/EoF
1 (1) [-] none 1 614400 B 1043.333706 1043.343286 30.004 fps ts mono/EoF
2 (2) [-] none 2 614400 B 1043.367034 1043.376613 30.005 fps ts mono/EoF
3 (0) [-] none 3 614400 B 1043.400362 1043.410089 30.005 fps ts mono/EoF
4 (1) [-] none 4 614400 B 1043.433691 1043.443281 30.004 fps ts mono/EoF
5 (2) [-] none 5 614400 B 1043.467018 1043.476608 30.006 fps ts mono/EoF
6 (0) [-] none 6 614400 B 1043.500346 1043.509936 30.005 fps ts mono/EoF
7 (1) [-] none 7 614400 B 1043.533675 1043.543264 30.004 fps ts mono/EoF
8 (2) [-] none 8 614400 B 1043.567002 1043.576592 30.006 fps ts mono/EoF
Sounding promising anyway. It'll be nice to see some pictures from it though.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Thanks.
I compiled new yavta, all fine now for ov7251 sensor, no timeout anymore, full frame number stored in file.h264.
Analysis of created file.pts:
Code: Select all
pi@raspberrypi:~/yavta $ !./pts
./ptsanalyze file.pts
creating tstamps.csv
500 frames were captured
majority framerate 30fps
frame delta time[us] distribution
6 33323
62 33324
49 33325
6 33326
76 33327
165 33328
14 33329
16 33330
76 33331
25 33332
2 33333
> after skip frame indices (middle column)
0 frame skips (0%)
average framerate 30fps
pi@raspberrypi:~/yavta $
OK, this is one frame of 500 frame file.h264 captured:Sounding promising anyway. It'll be nice to see some pictures from it though.
Please see the recorded video, there are quite some glitches in generated .h264, especially when hand is moved:
https://stamm-wilbrandt.de/en/forum/file.ov7251.h264
During recording preview window works fine, especially it looks brighter than what gets recorded:
As you can see on smartphone photo scene was really bright, ov7251 parameters as is result in relatively dark images:
Now to ov9281 dtoverlay, after reboot sensor gets reported correctly:
Code: Select all
pi@raspberrypi:~ $ v4l2-ctl -V
Format Video Capture:
Width/Height : 1280/800
Pixel Format : 'Y10P' (10-bit Greyscale (MIPI Packed))
Field : None
Bytes per Line : 1600
Size Image : 1280000
Colorspace : sRGB
Transfer Function : sRGB
YCbCr/HSV Encoding: ITU-R 601
Quantization : Full Range
Flags :
pi@raspberrypi:~ $
But yavta just hangs with timeout, unlike ov7251 first try not even few frames:
Code: Select all
pi@raspberrypi:~/yavta $ ./yavta --capture=500 -n 3 --encode-to=file.h264 -f UYVY -m -T /dev/video0
We're encoding to file.h264
Device /dev/video0 opened.
Device `unicam' on `platform:fe801000.csi' (driver 'unicam') is a video capture (without mplanes) device.
stride is 0
stride is now 1280
Video format set: Y10P (50303159) 1280x800 (stride 1600) field none buffer size 1280000
Video format: Y10P (50303159) 1280x800 (stride 1600) field none buffer size 1280000
Unable to get frame rate: Inappropriate ioctl for device (25).
vc.ril.isp:in:0(Y10P)(0x18b2120)type: video, fourcc: Y10P bitrate: 0, framed: 0 extra data: 0, (nil) width: 1280, height: 800, (0,0,1280,800) pixel aspect ratio: 0/0, frame rate: 0/0
buffers num: 3(opt 1, min 1), size: 1280000(opt 1280000, min: 1280000), align: 0
Created pool of length 3, size 0
Enable encoder....
Create pool of 3 buffers of size 0 for render
Create pool of 3 buffers of size 0 for encode ip
Create pool of 3 buffers of size 1536000 for encode/render
Writing data to file.h264
Create pool of 8 buffers of size 262144
Sent buffer 0x18bca28
Sent buffer 0x18bcc00
Sent buffer 0x18bcdd8
Sent buffer 0x18bcfb0
Sent buffer 0x18bd188
Sent buffer 0x18bd360
Sent buffer 0x18bd538
Sent buffer 0x18bd710
3 buffers requested, V4L2 returned 3 bufs.
length: 1280000 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xb19c7000.
Importing DMABUF 8 into VCSM...
...done. vcsm_handle 73728
Exported buffer 0 to dmabuf 8, vcsm handle 73728
Linking V4L2 buffer index 0 ptr 0x18be430 to MMAL header 0x18b9190. mmal->data 0xC0000003
length: 1280000 offset: 1282048 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xb188e000.
Importing DMABUF 9 into VCSM...
...done. vcsm_handle 77824
Exported buffer 1 to dmabuf 9, vcsm handle 77824
Linking V4L2 buffer index 1 ptr 0x18be4a0 to MMAL header 0x18b9368. mmal->data 0xC0000004
length: 1280000 offset: 2564096 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xb1755000.
Importing DMABUF 10 into VCSM...
...done. vcsm_handle 81920
Exported buffer 2 to dmabuf 10, vcsm handle 81920
Linking V4L2 buffer index 2 ptr 0x18be510 to MMAL header 0x18b9540. mmal->data 0xC0000005
select timeout
pi@raspberrypi:~/yavta $
https://hermann-sw.github.io/planar_graph_playground
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
-
- Raspberry Pi Engineer & Forum Moderator
- Posts: 13009
- Joined: Wed Dec 04, 2013 11:27 am
- Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Thanks HermannSW, that's all the confirmation I wanted on OV7251. It's a driver from mainline with no modifications, so the fact it loads correctly and can stream is good enough for me to ask Phil to merge it. Any further mods can come in the fullness of time. yavta isn't optimised on the recording, it was my hacked together test tool.
I know OV9281 works as I have two modules, one from Innomaker, and one from Vision Components, and it works with both of those. Which Pi variant are you running on, and have you got a sensibly sized CMA heap?
I know OV9281 works as I have two modules, one from Innomaker, and one from Vision Components, and it works with both of those. Which Pi variant are you running on, and have you got a sensibly sized CMA heap?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
4GB Pi4B.
I have "gpu_mem=256" in /boot/config.txt,.
No setting for cma_lwm or cma_hwm.
Do you have example for "sensible sized CMA heap"?
https://hermann-sw.github.io/planar_graph_playground
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbt
https://github.com/Hermann-SW/memrun
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/en/Raspberry_camera.html
-
- Posts: 42
- Joined: Fri Jun 12, 2020 10:19 pm
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Wait, So if the commit is merged, does that mean if i want to use the camera on other raspberry pi's all i have to do is along with cloning and making the yavta repo and then im good to go? That'd be amazing if its true.
Code: Select all
sudo rpi-update
-
- Raspberry Pi Engineer & Forum Moderator
- Posts: 13009
- Joined: Wed Dec 04, 2013 11:27 am
- Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
Re: Getting Arducam Global Shutter Camera to Work w/ROS
Cma != gpu_mem.
Pi4 with FKMS enabled will have a cma heap of 256MB by default, which is plenty for most camera use cases. You can check with "dmesg | grep cma"
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.