T-i-m
Posts: 47
Joined: Sun Jun 15, 2014 8:02 pm

Optimize dynamic range for security camera

Sun Jun 15, 2014 8:36 pm

Hi

This forum has been much appreciated over the last weeks when I have set up a Raspberry with camera module as a security system!

Finally I ran into something that isn't covered anywhere I believe. On most modern DSLR cameras there is a special effect that the camera can apply that evens out the light in the photos.

It has diffeent names on different systems:

Sony - Dynamic Range Optimization (DRO)
Nikon - Active D-Lighting
Cannon - Auto Lighting Optimizer (ALO)

But what it does to the raw image before it converts it is to brighten up the dark parts and darken the bright parts. It flatens the picture and isn't always usefull for creative photography but I think it is almost always desirable when you are recording surveilance videos.

A good explanation can be found here:
http://www.avtech.tw/index.php?option=c ... cle&id=279

Does anything like this exist as an effect in the camera module? Or is it planed to come in the future?

Regards

Tim

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 16119
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Optimize dynamic range for security camera

Mon Jun 16, 2014 9:00 am

Two related algorithms - we know them as dynamic range compression (DRC), and high dynamic range (HDR).

DRC is possible quite easily - I'm not sure it isn't already in there. It just cranks up the dark areas and turns down the bright ones.

HDR involves requesting a short and long exposure frame from the sensor and then combining them. This is possible on the Pi for stills captures only - to run at even modest resolutions it requires more processing than time allows to run at higher frame rates. It also needs more memory as it has to hold multiple raw images in memory before processing.
The issue is one of tuning again. Getting the correct settings for the sensor requires a fair amount of time and effort - how long do you go for the long exposure, and how short for the short one to give the best combined image? Also any motion between the two frames (they're read out sequentially) will result in blur.
I suspect that it isn't practical to get that work done for the limited benefit - sorry.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

T-i-m
Posts: 47
Joined: Sun Jun 15, 2014 8:02 pm

Re: Optimize dynamic range for security camera

Mon Jun 16, 2014 2:08 pm

Agreed, I am only interested in DRC which should be straight forward to add. And would be desirable for anyone doing surveillance camera applications.

It doesn't seem to be documented, can I find out if it is "in there already"?

Regards

Tim

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 16119
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Optimize dynamic range for security camera

Mon Jun 16, 2014 4:32 pm

I've just checked the firmware modules list - it isn't built at the moment.
The bad news is also that it too takes a load of tuning parameters that I wasn't aware of. I may be able to copy/paste it from another sensor, but I haven't a clue on exactly what it is setting up and whether that will work.

If I can beat it into shape and released, then MMAL_PARAMETER_DYNAMIC_RANGE_COMPRESSION will control it. That takes a struct MMAL_PARAMETER_DRC_T, with the strength being MMAL_PARAMETER_DRC_STRENGTH_OFF, MMAL_PARAMETER_DRC_STRENGTH_LOW, MMAL_PARAMETER_DRC_STRENGTH_MEDIUM, or MMAL_PARAMETER_DRC_STRENGTH_HIGH.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 33342
Joined: Sat Jul 30, 2011 7:41 pm

Re: Optimize dynamic range for security camera

Mon Jun 16, 2014 5:57 pm

6by9 wrote:I've just checked the firmware modules list - it isn't built at the moment.
The bad news is also that it too takes a load of tuning parameters that I wasn't aware of. I may be able to copy/paste it from another sensor, but I haven't a clue on exactly what it is setting up and whether that will work.

If I can beat it into shape and released, then MMAL_PARAMETER_DYNAMIC_RANGE_COMPRESSION will control it. That takes a struct MMAL_PARAMETER_DRC_T, with the strength being MMAL_PARAMETER_DRC_STRENGTH_OFF, MMAL_PARAMETER_DRC_STRENGTH_LOW, MMAL_PARAMETER_DRC_STRENGTH_MEDIUM, or MMAL_PARAMETER_DRC_STRENGTH_HIGH.
I've got time to take a look at this tomorrow David, if you are busy. I can plumb it in to Raspistill pretty quickly. But the problem will be the tuning - again, can check with the team tomorrow to see what is involved.
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

T-i-m
Posts: 47
Joined: Sun Jun 15, 2014 8:02 pm

Re: Optimize dynamic range for security camera

Mon Jun 16, 2014 9:08 pm

Wow thank you very much for the attention!

I didn't realize it would be so much effort to tune the parameters. Maybe an option could be to not tune the parameters much at all? Just force us users to set them to whatever we want them to be? I am happy to fiddle with the parameters myself if they are available. The usefulness of the feature would make it worth the effort for me to find a good set of parameters.

Thanks anyway for looking into it!

Tim

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 16119
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Optimize dynamic range for security camera

Tue Jun 17, 2014 7:20 am

T-i-m wrote:I didn't realize it would be so much effort to tune the parameters. Maybe an option could be to not tune the parameters much at all? Just force us users to set them to whatever we want them to be? I am happy to fiddle with the parameters myself if they are available. The usefulness of the feature would make it worth the effort for me to find a good set of parameters.
It's effectively quantifying how much to crank up the darks or otherwise manipulate the scene, and a set for each strength setting.
With DRC I suspect there aren't many/any sensor specific bits, so a copy/paste will probably work fairly well, just as long as we have the same version of the algorithm as the place we copy it from.
Sorry, but all the tuning parameters are set up within the GPU, so it is closed source.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 33342
Joined: Sat Jul 30, 2011 7:41 pm

Re: Optimize dynamic range for security camera

Tue Jun 17, 2014 4:29 pm

OK, have got DRC working to some extent. I've check in the change to the firmware, so once Dom does his next release I will also send the Raspicam changes to userland.

They can be seen here, but note, will not work until the firmware changes are released.

https://github.com/JamesH65/userland/tree/drc
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

riklaunim
Posts: 269
Joined: Tue Apr 22, 2014 7:34 pm

Re: Optimize dynamic range for security camera

Tue Jun 17, 2014 4:37 pm

You can also stack multiple frames into one. That increases signal to noise ratio by square root of stacked frames count. That is used in astrophotography.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 33342
Joined: Sat Jul 30, 2011 7:41 pm

Re: Optimize dynamic range for security camera

Thu Jun 19, 2014 3:47 pm

OK, if you rpi-update to get the latest firmware, then download and build a new version of raspistill/vid from userland, it should have a new DRC mode.

Whether it's any good I leave up to you guys...

James
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

ethanol100
Posts: 668
Joined: Wed Oct 02, 2013 12:28 pm

Re: Optimize dynamic range for security camera

Fri Jun 20, 2014 2:13 pm

Can confirm that the new DRC mode is working.

But without using it(i.e., raspistill -o test.jpg), I get

"mmal: mmal_vc_port_parameter_set: failed to set port parameter 64:0:EINVAL
mmal: Argument is invalid"

The Parameter "params->drc_level" is not set to a default value and is empty if not specified.

Additionally could you somewhere write the possible levels "off,low,med,high". As a new item under "Notes"?

Edit: Have send a pull request.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 33342
Joined: Sat Jul 30, 2011 7:41 pm

Re: Optimize dynamic range for security camera

Fri Jun 20, 2014 3:06 pm

ethanol100 wrote:Can confirm that the new DRC mode is working.

But without using it(i.e., raspistill -o test.jpg), I get

"mmal: mmal_vc_port_parameter_set: failed to set port parameter 64:0:EINVAL
mmal: Argument is invalid"

The Parameter "params->drc_level" is not set to a default value and is empty if not specified.

Additionally could you somewhere write the possible levels "off,low,med,high". As a new item under "Notes"?

Edit: Have send a pull request.
Oops. My bad - not enough testing.

The levels are indicated in the documentation on the Raspi website. But I need to put them in the ? help as well....thought I had done that already.....double my bad.
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

ethanol100
Posts: 668
Joined: Wed Oct 02, 2013 12:28 pm

Re: Optimize dynamic range for security camera

Fri Jun 20, 2014 3:28 pm

Sadly I have no time to check/test my changes and I am not sure if it is still working as expected.
I can completely understand the small mistakes, when you are in a hurry.
Thank you for spending your time for the new feature.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 33342
Joined: Sat Jul 30, 2011 7:41 pm

Re: Optimize dynamic range for security camera

Fri Jun 20, 2014 4:05 pm

I've fixed up the issues and submitted a pull request to raspi/userland. A quick check showed all the issues reported are OK now.
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

T-i-m
Posts: 47
Joined: Sun Jun 15, 2014 8:02 pm

Re: Optimize dynamic range for security camera

Sun Jun 22, 2014 9:00 am

Hi

This is great news, unfortunately I don't have acces to my Pi this week but will try this out as soon as I can!

Is this something I will be able to access from the Python picam3 interface or is it for the time beeing only from raspistill?

Regards

Tim

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 16119
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Optimize dynamic range for security camera

Sun Jun 22, 2014 11:29 am

T-i-m wrote:Is this something I will be able to access from the Python picam3 interface or is it for the time beeing only from raspistill?
The python interface is down to waveform80. He's generally fairly reactive to adding extra parameters such as this though.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

RpiName
Posts: 745
Joined: Sat Jul 06, 2013 3:14 am

Re: Optimize dynamic range for security camera

Sun Jun 22, 2014 11:52 am

T-i-m wrote: Is this something I will be able to access from the Python picam3 interface or is it for the time beeing only from raspistill?
Is picam3 based on V4L2? If yes, you may want to try this driver, which supports DRC:
http://www.linux-projects.org/modules/n ... toryid=163

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 33342
Joined: Sat Jul 30, 2011 7:41 pm

Re: Optimize dynamic range for security camera

Sun Jun 22, 2014 2:50 pm

RpiName wrote:
T-i-m wrote: Is this something I will be able to access from the Python picam3 interface or is it for the time beeing only from raspistill?
Is picam3 based on V4L2? If yes, you may want to try this driver, which supports DRC:
http://www.linux-projects.org/modules/n ... toryid=163
It would be good for the Foundation to be credited with introducing the new features which are then used elsewhere...since they do all the hard work.....
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

Return to “Camera board”