Video capture software - Ubuntu Arm to Rpi?

[Steep]

This is a beginners question really, I have a video camera I use with my telescope ( a Point Grey Chameleon ) for which Point Grey have written capture software for the Beagle board. I'd like to try and make it work on my RPi, what would I need to do to convert it from Ubuntu Arm7 to Debian Arm?

I have no Linux system except the RPi but could if needed install on a spare PC.

[ghans]

If it is open source , it should just be a configure / make away.
Else the company will have to pay a programmer / linux guy to port it over , and then think about
how to get back this additional expense (from customers like you perhaps ?) .

ghans

[Steep]

Thanks, I guess my first port of call should be PG, I'll try them.

[ghans]

Was it open source anyway ?

ghans

[trbotha]

The Point Grey API (Flycapture) is not open source and I have querried them whether they can make it available on ARMv6, but no luck. However, Point Grey USB cams use IIDC standard used for 1394 (firewire) cameras. I have succesfully obtained images from two Point Grey chameleon cameras simultaneously using the Rpi and the libdc1394 library which is open source.

[ghans]

Good to know , but i don't understand the second part of your post:
Is there a USB Device "category" or standard for FireWire
cameras ?


ghans

[trbotha]

There is no USB device standard for Firewire interface. However the Point Grey USB cams dont use the standard Webcam interface, they are compliant with IIDC 1934 DCAM interface. The libdc1394 library thus uses libusb (specific versions are required for both libraries latest libs work fine) to communicate with the USB Point Grey Cameras. Thus, any command issued via the lidc1394 API is transmitted via USB and the Point Grey USB cameras understands this command. There may be some commands that are specific to the 1394 bus that are not capable on the USB bus. One I believe is the synching of the cameras using the bus. However, the Point Grey cameras both support Software and Hardware triggers for synching. Thus, this is only applicable to Point Grey USB cameras (there may be others check the list of IIDC DCAM compliant hardware). Any other USB cameras should be compliant with the same standard used on most webcams.

[rasmus25]

Hi, I am also considering the Chameleon+RPi combo. Were you able to get any video out of the camera or just still frames? The RPi USB interface seems to be a bit slow. I wonder if 15 FPS at 800x600 would be doable on Pi?

Some day I would like to use Pi's hardware encoding capabilities to record hours of video from the Point Grey Chameleon.

[trbotha]

I can capture video as well. I currently capture two chameleon cams using as fast as possible sofware synching (required for stereo vision) so not consistent fps streaming per say. However, I can currently capture two chameleons at 640x480 grayscale resolution at around 25-30fps. The biggest problem I had was not the USB speed, but rather the SD memory card write speed since I started out writing raw images to the SD card and that was the limit. I make use of hardware H264 encodingto speed up capturing. I transform the gray to raw rgb (since the encoder only except rgb and yuv input streams no gray as yet) then hardware encode to h264 and add the h264 header afterwards. The limit is then really the conversion from gray to rgb and other slight cpu computing part. If you were to use colour the slowest bit will most likely be the bayer image to rgb conversion. Not sure if bayer inputs are accpeted by the harware encoding.

[ghans]

Can you share with us how you realised it ?
How did you use OpenMAX and how did you interface
the camera in your setup ? Did you create custom software ?

ghans

[trbotha]

I'm willing to share my code, since alot of it based on other people's work. However, I'll just write some comments before I do so. I used OMX based on the example hello_video just wrote a c++ class for it and for the point grey cams. Some of the code is hardware coded since I only spend a few hours on the code. The camera class needs alot of change settings code since I mainly plugged the cameras into my laptop set them up with flycapture and saved the settings on the cameras. It should be easy to add functions to change exposure etc.. And again I'm using software triggering not streaming but this should be an easy mod aswell.

[rasmus25]

Good job! I would be very much interested in the code. I am not in a hurry, as I haven't ordered the camera yet, but it is encouraging to know that the idea has potential. I guess the color conversion could not be avoided anyways, even with typical webcams the color format of raw images is not quite what encoders expect.

[trbotha]

I have uploaded the source code to https://github.com/trbotha/PointGrey_RPi

Again note that the code is not complete. For the code to be used the camera needs to be setup to work in software trigger mode, the prototype function is provided for this but is commented out. The camera can be setup using Flycapture software from PointGrey simply save the changes on the first memmory channel of the camera. I will add code later to setup the camera from the source code when I review the program later, since I will use the program later in a project, for now people can play around with the code if they want. In main.cpp the program only capture 100 frames using a for loop simply change this to a while loop with some kind of keyhit function to exit.

[trbotha]

I had time to work on the code again, the cameras can be set into streaming mode aswell, the shutter opening time can also be specified. Other settings can easily be added as well using the dc1394 library api. The code also auto sets up the amount of cameras on the bus ie.e 1 or 2 cameras. More may be possible depending on bus limitations however only upto two have been tested thus far. Two chaemleons cameras were tested at streaming 30fps succesfully, this I believe is the limit of the cameras as well. I have not tested colour as yet, but bayer conversion algorithms are available from the dc1394 library as well.

[rasmus25]

I received my camera yesterday and experimented with color video. Results are good.

trbotha, I forked your code to https://github.com/rasmus25/PointGrey_RPi and modified it to get color video out. I used libdc1394 debayerization functions to get RGB images. Current code is hard-coded to receive 1280x960 bayer images from camera and use downsample debayerization to get a 640x480 image out of it, which is then encoded in GPU. I used downsampling, because it seems to me that this is the easiest way to do color pixel binning (using Chameleons built-in pixel binning produces grayscale output, not bayer). Framerate is around 17.5, as expected from streaming 1280x960 video over USB2.0, CPU usage is around 70%. This is probably because debayerization runs on CPU. It should not be too hard to write an OpenGL shader that does downsampling debayerization on the GPU.

[RomanG]

Hi trbotha,

I have Point Grey Research FireFly MV FMVU-03MTM mono camera and I would like to get it going on pi.
Was very happy to find this thread (the only one dealing with PointGrey cameras on rpi).

Trying to compile your code on my rpi (Raspbian 3.6.11+ #371 PREEMPT) I run in to some missing libraries (ilclient and camwire).

Using your code, how can be video stream from the camera captured?
Just saved to the file or can it be also continuous live stream?

Thanks,
Roman

[RomanG]

I compiled and installed camwire and then I was able to compile your GPU-Encode.
When I run it as a regular user, I get Segmentation fault error:

Code: Select all

pi@RPi /usr/src/PointGrey_RPi $ ./GPU-Encode
No File name supplied will use deafult = file*.mp4
string is shutter
shutter is 10.000000
libdc1394 warning: usb: Failed to open device for config ROM
libdc1394 warning: Failed to get config ROM from usb device
libdc1394 error: No cameras found.

Segmentation fault

When I run it as root, the GPU-encode is able to query camera for parameters and then displays prompt, Press S to start and stop recording

Code: Select all

pi@RPi /usr/src/PointGrey_RPi $ sudo ./GPU-Encode
No File name supplied will use deafult = file*.mp4
string is shutter
shutter is 10.000000
Using camera with GUID 49712223531685979
Shutter value of cam 0 is 480
Get Camera Features
------ Features report ------
OP   - one push capable
RC   - readout capable
O/OC - on/off capable
AC   - auto capable
MC   - manual capable
ABS  - absolute capable
-----------------------------
Brightness:
        RC  MC  (active is: MAN)  AC  (active is: MAN)
        min: 1 max 255
        current value is: 134
Exposure:
        O/OC  MC  (active is: MAN)  AC  (active is: MAN)
        Feature: OFF  min: 7 max 62
        current value is: 15
Sharpness:
        NOT AVAILABLE
White Balance:
        NOT AVAILABLE
Hue:
        NOT AVAILABLE
Saturation:
        NOT AVAILABLE
Gamma:
        RC  O/OC  MC  (active is: MAN)
        Feature: OFF  min: 0 max 1
        current value is: 0
Shutter:
        RC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        min: 1 max 2048
        current value is: 480
        absolute settings:
         value: 0.029988
         min: 0.000062
         max: 0.127948
Gain:
        RC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        min: 16 max 64
        current value is: 64
        absolute settings:
         value: 12.041193
         min: 0.000000
         max: 12.041193
Iris:
        NOT AVAILABLE
Focus:
        NOT AVAILABLE
Temperature:
        NOT AVAILABLE
Trigger:
        RC  O/OC
        Feature: ON
        AvailableTriggerModes: 384 387
        AvailableTriggerSources: 576 579 580
        Polarity Change Capable: True
        Current Polarity: NEG
        current mode: 384

        current source: 580
Trigger Delay:
        RC  O/OC  MC  (active is: MAN)  ABS
        Feature: OFF  min: 0 max 4095
        current value is: 0
        absolute settings:
         value: 0.000000
         min: 0.000000
         max: 65.000000
White Shading:
        NOT AVAILABLE
Frame Rate:
        RC  O/OC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        Feature: OFF  min: 466 max 2258
        current value is: 480
        absolute settings:
         value: 30.000000
         min: 4.604834
         max: 30.430416
Zoom:
        NOT AVAILABLE
Pan:
        RC  O/OC  MC  (active is: AUTO)  AC  (active is: AUTO)
        Feature: ON  min: 0 max 112
        current value is: 56
Tilt:
        NOT AVAILABLE
Optical Filter:
        NOT AVAILABLE
Capture Size:
        NOT AVAILABLE
Capture Quality:
        NOT AVAILABLE
 nSlice = 480
 nStride = 640
Bitrate is = 5000000
encode to idle...
enabling port buffers for 200...
enabling port buffers for 201...
encode to executing...

Filename is file_Cam0.h264

Press S to start and stop recording.

When I press S, the session hangs (I am doing this trough ssh).
I have to restart the session and kill the GPU-Encode.

Running it directly on pi yields same results.

The video file *.h264 is created, but it has 0b size.

Any suggestions?

Thanks a lot,
Roman

[trbotha]

Hi RomanG

Sorry for the late reply, If you still having problems try to remove any function in main.cpp with the word "terminal" in it i.e. reset_terminal and set_conio_terminal etc. These functions are used to provide getch characteristics such that carriage return (enter) does not need to be entered after a letter. I have experienced at times that the the program does not get the character input. If you remove these lines that will remove this problem, however, you will then need to type 's' or 'S' and then 'Enter' for both start and stop.

Hope this helps

[RomanG]

Thanks trbotha,

I will try as you suggested.

Roman

[RomanG]

OK, I commented out any functions related to terminal and also any calls to these functions.

When I run GPU-Encode as sudo, I get following:

Code: Select all

pi@RPi /usr/src/PointGrey_RPi $ sudo ./GPU-Encode
No File name supplied will use deafult = file*.mp4
string is shutter
shutter is 10.000000
Using camera with GUID 49712223531685979
Shutter value of cam 0 is 480
Get Camera Features
------ Features report ------
OP   - one push capable
RC   - readout capable
O/OC - on/off capable
AC   - auto capable
MC   - manual capable
ABS  - absolute capable
-----------------------------
Brightness:
        RC  MC  (active is: MAN)  AC  (active is: MAN)
        min: 1 max 255
        current value is: 146
Exposure:
        O/OC  MC  (active is: MAN)  AC  (active is: MAN)
        Feature: OFF  min: 7 max 62
        current value is: 7
Sharpness:
        NOT AVAILABLE
White Balance:
        NOT AVAILABLE
Hue:
        NOT AVAILABLE
Saturation:
        NOT AVAILABLE
Gamma:
        RC  O/OC  MC  (active is: MAN)
        Feature: OFF  min: 0 max 1
        current value is: 0
Shutter:
        RC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        min: 1 max 2048
        current value is: 480
        absolute settings:
         value: 0.029988
         min: 0.000062
         max: 0.127948
Gain:
        RC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        min: 16 max 64
        current value is: 64
        absolute settings:
         value: 12.041193
         min: 0.000000
         max: 12.041193
Iris:
        NOT AVAILABLE
Focus:
        NOT AVAILABLE
Temperature:
        NOT AVAILABLE
Trigger:
        RC  O/OC
        Feature: ON
        AvailableTriggerModes: 384 387
        AvailableTriggerSources: 576 579 580
        Polarity Change Capable: True
        Current Polarity: NEG
        current mode: 384

        current source: 580
Trigger Delay:
        RC  O/OC  MC  (active is: MAN)  ABS
        Feature: OFF  min: 0 max 4095
        current value is: 0
        absolute settings:
         value: 0.000000
         min: 0.000000
         max: 65.000000
White Shading:
        NOT AVAILABLE
Frame Rate:
        RC  O/OC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        Feature: OFF  min: 466 max 2258
        current value is: 480
        absolute settings:
         value: 30.000000
         min: 4.604834
         max: 30.430416
Zoom:
        NOT AVAILABLE
Pan:
        RC  O/OC  MC  (active is: AUTO)  AC  (active is: AUTO)
        Feature: ON  min: 0 max 112
        current value is: 56
Tilt:
        NOT AVAILABLE
Optical Filter:
        NOT AVAILABLE
Capture Size:
        NOT AVAILABLE
Capture Quality:
        NOT AVAILABLE
 nSlice = 480
 nStride = 640
Bitrate is = 5000000
encode to idle...
enabling port buffers for 200...
enabling port buffers for 201...
encode to executing...

Filename is file_Cam0.h264

Press S to start and stop recording.

Pressing S seems to start recording (the CPU load goes up) and pressing S again stops recording (CPU load goes down). It does not hang up the RPi

However, the *.mp4 file has 0 bytes size, same as before.

Roman

[trbotha]

The last problem that it could be is that of the triggering, since I see the default in my code seems to use software triggering. Try and umcomment line 67 and comment line 68 in main.cpp. This will swicth the mode to streaming mode instead of software triggering. The software triggering is not neccessary since I use it to sync two cameras for stereovision and is therefore not needed in your code.

If you wish to use streaming look at rtmp servers, I have personally not used it before, you should be able to use the file being created for the server.

[djibb]

Hi guys,

this works :
"Try and umcomment line 67 and comment line 68 in main.cpp"

I'm able to have a video from pointgrey firefly mv.

Now, I will try to stream it.

[Kozuch]

trbotha, what degree of stereo synchronization can you achieve with the Chameleon cameras? They do have no memory buffer so I guess the you may not be able to get very precise sync as one of the cameras still has to wait untill the first one finishes its read-out over USB? I am very interested in very precise stereo sync and currently am exploring ways how to do that and I read you need a memory frame buffer on the camera is you hook both cameras on the same serial interface.

It is also a bit tricky to measure the sync down to ms level, for rough measurement you can use stopwatch on a smartphone (display refresh 30-60Hz) but for ms measurement you need something like a custom circuit with sequential LEDs which you can time to light for 1ms each for example...

[trbotha]

Point Grey cameras have an internal buffer, generally two frames. The frames are captured at the same time and send later. The synchronization has not been a problem. The bigger problem is the h264 encoding which performs digital image correlation itself and can therefore affect the results you get. I have however coded most of my programs on an x86 PC using mostly hardware synching and MJPEG encoding at the most.

[Kozuch]

trbotha, I am surprised to learn about the 1-2 frame image buffer. I would like to learn more about it - does Point Grey have some official info on this or did you learn it by your experience with the stereo sync only? I can not find any official docs so far...

Also, did you really measure your sync to a 1ms precision? Because if not you may only have got maybe 15ms sync and not know about it...