ROS node for camera module

[MacHack]

Hi everybody,
This is my first topic here and I think it might be useful for some people.
During the last 2 days, I've been developing a ROS node for the camera module of the RPi. For those who don't know what ROS is, you can learn more at http://www.ros.org.

The goal of this node is to grab images from the camera module and publish them on a topic.
To save bandwith, images are compressed in jpeg format.
The source code is mainly based on the RaspiVid software where the H264 encoder has been replaced by a JPEG encoder.
So, what can you expect from this piece of software :
- bugs
- jpeg images publish in a topic
- resolutions up to 1920x1080
- framerate up to 30 fps
- bugs
- low latency
- configurable quality of the jpeg encoder
- smaller FOV as it uses the video mode which does not use the full sensor.

You can find the source code at https://github.com/fpasteau/raspicam_node

There are still some bugs and memory leaks in the software that I need to patch.
The source code is available so you can try it for yourself. You can report bugs to me through this thread, but I don't promise anything.

[danishbizkit]

Hi MacHack,

Thanks for the source!

Is this node supposed to be running on a RPi or on a another device that receives the video stream over LAN/WIFI/USB from the RPi camera?

[MacHack]

This node is supposed to be running on the RPi itself.
Then you can get the images through the ROS topic on another computer.

I've recently added the necessary code for camera info management.
This way it's really easy to calibrate the camera module using the camera_calibration node and remove the distortion from the images.

[Fr4gg0r]

Have you tested 1920*1080 at 30Hz?
Gstreamer won't do that resolution at 30Hz

I recently created a node to interface servoblaster via ros, a wiki or common github repo for ros nodes designated for rpi would be nice.

Edit:
Ok, you never said it would reach 30Hz at 1920*1080 -_-

Edit2:
Took a look at your code, looks very fine.
I think I am going to fork it when I find some time and try some stuff out.

[MacHack]

Theoratically, the node can output 1920*1080 jpeg images at 30 Hz. In fact, when debugging the node, I managed to save in ram 1920*1080 jpeg images at 30 Hz. However, at this high resolution/framerate, the bottleneck is the network bandwidth. A 1920*1080 jpeg is around 1.2 MB, meaning a 36 MB/s bandwith needed to transmit the stream. On the RPI, the ethernet is limited to 100Mb/s meaning around 10MB/s which is not enough.
The only way to stream 1080p/30Hz stream would be to use a different encoder. But AFAIK, the ros image-transport stack doe not support H264 stream, that's why I used JPEG.

I will try to check if theora encoding works, but as I haven't seen anybody using it, I'm not too confident about it.

I recently created a node to interface servoblaster via ros, a wiki or common github repo for ros nodes designated for rpi would be nice.

For sure, a common github repo for all RPI ros node would be very usefull.

[Maximus5684]

Holy *bleep*, you're the man (or woman)! I've been trying to do this for MONTHS.

I'll also work with the node and see if I can make any improvements.

THANKS AGAIN!

*EDIT: Nevermind on the suggestion that was here. I'm blind.

[cesarverdes]

Dear MacHack,

I think that it will be a good idea to transfer the video from the Raspberry Pi to the PC using a more optimised codec and create a ROS node into the PC to change the format and publish it to jpeg.

The used bandwith could be critical over a Wifi link. What do you think?

César

[MacHack]

Dear MacHack,

I think that it will be a good idea to transfer the video from the Raspberry Pi to the PC using a more optimised codec and create a ROS node into the PC to change the format and publish it to jpeg.

The used bandwith could be critical over a Wifi link. What do you think?

César

I think it should already be possible to do so using gstreamer with h264 codec and the gscam node from ROS.
You would get a lower bandwith but a higher latency.
Anyway, there might be problems using wifi. If some packets get dropped, you would get a lot of bad frames due to the codec.

I haven't tried it as in my use case, I need minimal latency.
I use this ROS node with these parameters :
resolution : 640x480
framerate : 90fps
Jpeg quality : 10
I have a bandwith of around 1-2 MB/s with nearly no latency.

[adt47]

MacHack,

I am attempting to use this node, but it does not appear to be publishing any data even though the camera is operating (red led is on).

When I run "rosrun image_view image_view image:=/camera/image _image_transport:=compressed" on the pi, I receive the following message: "[rospack] Error: stack/package image_view not found". I can run rqt_image_view, but in the pull down menu there are no topics/messages available.

If I run the image_view command on a separate pc with the pi declared as the master, image_view opens, but no images are displayed.

Any help would be appreciated.

Thanks,
adt47

edit: It is publishing data. I used "rostopic hz /camera/image/compressed", and it is publishing data at about 30hz.

[adt47]

MacHack,

I am attempting to use this node, but it does not appear to be publishing any data even though the camera is operating (red led is on).

When I run "rosrun image_view image_view image:=/camera/image _image_transport:=compressed" on the pi, I receive the following message: "[rospack] Error: stack/package image_view not found". I can run rqt_image_view, but in the pull down menu there are no topics/messages available.

If I run the image_view command on a separate pc with the pi declared as the master, image_view opens, but no images are displayed.

Any help would be appreciated.

Thanks,
adt47

edit: It is publishing data. I used "rostopic hz /camera/image/compressed", and it is publishing data at about 30hz.

Ok, the node works. I can now run image_view on the pi, and I see the feed from the camera. It has about 1.5-2 seconds of lag.

I still cannot see the feed when I run image_view on my desktop. roscore is running on the desktop.

image_view now works on the pc. I added the hostnames and IP addresses to etc/hosts.

Now how do I get a different node to subscribe. I keep getting the message: It looks like you are trying to subscribe directly to a transport-specific image topic 'camera/image/compressed', in which case you will likely get a connection error.

[Ch4rly]

Hi,

first of all thanks for sharing this ROS node! The basic functionality is working pretty well.

Although I have problems using RaspiCamControl to change some parameters. I would like to disable AWB and set an exposure mode for faster shutter speeds. What I tried was to set this in the defaults (no luck) and setting the parameters just before capture starts with individual function-calls. The functions return success, but nothing seems to change, I also tried rotation without any success.
Any help how to fix this would be highly appreciated!

Thanks in advance!
Charly

[ptksmith]

MacHack,

I am attempting to use this node, but it does not appear to be publishing any data even though the camera is operating (red led is on).

When I run "rosrun image_view image_view image:=/camera/image _image_transport:=compressed" on the pi, I receive the following message: "[rospack] Error: stack/package image_view not found". I can run rqt_image_view, but in the pull down menu there are no topics/messages available.

If I run the image_view command on a separate pc with the pi declared as the master, image_view opens, but no images are displayed.

Any help would be appreciated.

Thanks,
adt47

edit: It is publishing data. I used "rostopic hz /camera/image/compressed", and it is publishing data at about 30hz.

Ok, the node works. I can now run image_view on the pi, and I see the feed from the camera. It has about 1.5-2 seconds of lag.

I still cannot see the feed when I run image_view on my desktop. roscore is running on the desktop.

image_view now works on the pc. I added the hostnames and IP addresses to etc/hosts.

Now how do I get a different node to subscribe. I keep getting the message: It looks like you are trying to subscribe directly to a transport-specific image topic 'camera/image/compressed', in which case you will likely get a connection error.

Hi, adt47,
I have the same problem with image_view that it shows "[rospack] Error: stack/package image_view not found".
Could you please tell me how do you run the image_view on the pi?
Thanks
ptksmith

[sandPi]

I am running Ubuntu 14.04 on my RPi.
I am getting an error when I try to compile this node,

/usr/include/interface/vcos/vcos_types.h:38:33: fatal error: vcos_platform_types.h: No such file or directory
#include "vcos_platform_types.h"
^
compilation terminated.
In file included from /usr/include/interface/vcos/vcos_assert.h:149:0,
from /usr/include/interface/vcos/vcos.h:114,
from /home/ubuntu/catkin_ws/src/raspicam/src/RaspiCLI.cpp:46:
/usr/include/interface/vcos/vcos_types.h:38:33: fatal error: vcos_platform_types.h: No such file or directory
#include "vcos_platform_types.h"

Please let me know what is the fix for this.

[DougieLawson]

git clone https://github.com/raspberrypi/userland
cd userland
./buildme

[oramirez]

git clone https://github.com/raspberrypi/userland
cd userland
./buildme

Actually I just did a very similar install and the problem was not the lack of userland, but a problem with where things are installed.

Code: Select all

sudo apt-get install libraspberrypi-bin-nonfree libraspberrypi0 libraspberrypi-dev libraspberrypi-bin ros-indigo-image-transport  ros-indigo-theora-image-transport

For me raspivid and all the dependencies where in /usr. Here is a patch on the cmakelist of the raspicam_node

Code: Select all

diff --git a/CMakeLists.txt b/CMakeLists.txt
index b197adb..5e18d9c 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -64,8 +64,8 @@ include_directories(include
   ${catkin_INCLUDE_DIRS}
   /home/pi/userland
   /opt/vc/include
-  /opt/vc/include/interface/vcos/pthreads
-  /opt/vc/include/interface/vmcs_host/linux
+  /usr/include/interface/vcos/pthreads
+  /usr/include/interface/vmcs_host/linux
 )
 
 ## Declare a cpp library
@@ -88,25 +88,25 @@ include_directories(include
  target_link_libraries(raspicam_node
    ${catkin_LIBRARIES}
 raspicamcontrol raspicli   
-/opt/vc/lib/libbcm_host.so
-/opt/vc/lib/libvcos.so
-/opt/vc/lib/libmmal.so
-/opt/vc/lib/libmmal_core.so
-/opt/vc/lib/libmmal_util.so
-/opt/vc/lib/libmmal_vc_client.so
-/opt/vc/lib/libvchostif.a
+/usr/lib/libbcm_host.so
+/usr/lib/libvcos.so
+/usr/lib/libmmal.so
+/usr/lib/libmmal_core.so
+/usr/lib/libmmal_util.so
+/usr/lib/libmmal_vc_client.so
+/usr/lib/libvchostif.a
 )
 
  target_link_libraries(raspicam_raw_node
    ${catkin_LIBRARIES}
 raspicamcontrol raspicli   
-/opt/vc/lib/libbcm_host.so
-/opt/vc/lib/libvcos.so
-/opt/vc/lib/libmmal.so
-/opt/vc/lib/libmmal_core.so
-/opt/vc/lib/libmmal_util.so
-/opt/vc/lib/libmmal_vc_client.so
-/opt/vc/lib/libvchostif.a
+/usr/lib/libbcm_host.so
+/usr/lib/libvcos.so
+/usr/lib/libmmal.so
+/usr/lib/libmmal_core.so
+/usr/lib/libmmal_util.so
+/usr/lib/libmmal_vc_client.so
+/usr/lib/libvchostif.a
 )
 
 #############

[DougieLawson]

On Raspbian there's a lot of symbolic links from /usr/bin/xxx stuff to /opt/vc/xxx stuff.