6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13546
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

New feature implemented - forcing the sensor mode

Fri Aug 29, 2014 5:12 pm

Hi All.

As a final parting feature, once Dom pushes an updated firmware it should be possible to force the GPU to choose a particular sensor mode for everything.

Using MMAL_PARAMETER_CAMERA_CUSTOM_SENSOR_CONFIG these values should select following modes (done from memory as I forgot to check before I left):
0 - normal mode
1 - 1080P30 cropped 1-30fps mode
2 - 5MPix 1-15fps mode
3 - 5MPix 0.1666-1fps mode
4 - 2x2 binned 1296x972 1-42fps
5 - 2x2 binned 1296x730 1-49fps
6 - VGA 30-60fps
7+ - VGA 60-90fps
I'd advise setting the value before enabling the camera component or any ports. Changing it afterwards should work but I've not tried it and can't guarantee it will switch (no way to check now!)
The framerate ranges on the modes are fixed - forcing the 5MPix 1-15fps mode and then asking for 30fps will not achieve 30fps.

Hope it is of some use.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

SirNight
Posts: 36
Joined: Fri Jul 18, 2014 1:01 pm

Re: New feature implemented - forcing the sensor mode

Fri Aug 29, 2014 5:33 pm

Hi,
Does it make impact on raspistill relating to colour issue?
How can I use it with raspistill ?
Thx

User avatar
jbeale
Posts: 3928
Joined: Tue Nov 22, 2011 11:51 pm

Re: New feature implemented - forcing the sensor mode

Fri Aug 29, 2014 9:24 pm

Really appreciate this last feature, 6x9! Thank you for all you have done for us in the R-Pi camera world!

@SirNight: this feature is accessed at the MMAL level, so you would have to modify the C source code to raspistill and recompile before it would be available to the end-user. Maybe it will be incorporated into the Python picamera library soon.

SirNight
Posts: 36
Joined: Fri Jul 18, 2014 1:01 pm

Re: New feature implemented - forcing the sensor mode

Fri Aug 29, 2014 9:42 pm

jbeale wrote:Really appreciate this last feature, 6x9! Thank you for all you have done for us in the R-Pi camera world!

@SirNight: this feature is accessed at the MMAL level, so you would have to modify the C source code to raspistill and recompile before it would be available to the end-user. Maybe it will be incorporated into the Python picamera library soon.
@jbeale , so you confirm it makes positive impact on the colour issue. ?
So how can I use this, is there any way to learn? Thanks in advance

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13546
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New feature implemented - forcing the sensor mode

Mon Sep 01, 2014 7:11 am

SirNight wrote:
jbeale wrote:Really appreciate this last feature, 6x9! Thank you for all you have done for us in the R-Pi camera world!

@SirNight: this feature is accessed at the MMAL level, so you would have to modify the C source code to raspistill and recompile before it would be available to the end-user. Maybe it will be incorporated into the Python picamera library soon.
@jbeale , so you confirm it makes positive impact on the colour issue. ?
So how can I use this, is there any way to learn? Thanks in advance
I would expect no effect at all.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13546
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New feature implemented - forcing the sensor mode

Fri Sep 19, 2014 5:28 pm

Raspicam mods done to add this option to the command line. Currently on https://github.com/6by9/userland master or PR-20140919 branches, and PR done to get it onto the main raspberrypi/userland repo.

-md X or --mode X, where X is the mode to select as follows:
0 = auto. Default and same as in the past.
1 = 1080P cropped mode.
2 = 5MPix
3 = 5MPix/s 0.1666-1 fps
4 = 2x2 binned
5 = 2x2 binned 16:9
6 = 30-60 fps VGA
7 = 60-90 fps VGA
(I actually managed to get it correct in my earlier post. Miracles do happen :D )
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

gregeric
Posts: 1509
Joined: Mon Nov 28, 2011 10:08 am

Re: New feature implemented - forcing the sensor mode

Sat Sep 20, 2014 7:05 pm

Many thanks 6by9. I wasn't able to keep pace with your updates / features added / bugs fixed in your last weeks at Broadcom. Can someone point me to a thread where this was requested, or briefly describe the instances it may be best deployed?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13546
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New feature implemented - forcing the sensor mode

Sat Sep 20, 2014 7:46 pm

There were numerous people wanting to force binned mode or the like. I haven't the energy to go and find the exact threads as it was more that I thought of an easy way to do it when a comment was made on http://www.raspberrypi.org/forums/viewt ... 43&t=83353

The main one that it is likely to have benefit for is being able to force the buffering that is allocated for burst captures. Half the issue is having to drop a frame every time the sensor is restarted. If you've forced preview to use the full 5MPix mode then it will allocate 3 input frames at full res and that means that burst captures don't need to stop the sensor during the stills capture phase.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

semek
Posts: 3
Joined: Wed Jun 15, 2016 10:28 am

Re: New feature implemented - forcing the sensor mode

Wed Jun 15, 2016 10:48 am

Hi,

It seems that sensor mode 5 (1296x730) is not centered properly. I have captured two 1296x730 images using raspistill.

1. raspistill -w 1296 -h 730 -md 4 -o mode4.jpg
2. raspistill -w 1296 -h 730 -md 5 -o mode5.jpg

On first image small dot is in the center while on the second image it's shifted down. Could it be some firmware bug? Currently I workaround this problem by manually setting sensor mode and avoiding mode 5.
Attachments
mode5.jpg
Mode 5
mode5.jpg (19.83 KiB) Viewed 8123 times
mode4.jpg
Mode 4
mode4.jpg (21.53 KiB) Viewed 8123 times

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13546
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New feature implemented - forcing the sensor mode

Wed Jun 15, 2016 11:27 am

Potentially a firmware bug.
Can you take a 1296x972 image for me too using both modes 4 and mode 2? It'd help to work out roughly how many pixels we are off by? It also doesn't help that the images have been resized to stick on the forum (648px × 365px).
(How many complaints will I get if i fix this though? Any nicely framed images are going to shift!)
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

semek
Posts: 3
Joined: Wed Jun 15, 2016 10:28 am

Re: New feature implemented - forcing the sensor mode

Thu Jun 16, 2016 11:21 am

Here are links to full size files:

https://drive.google.com/file/d/0BzNjRb ... sp=sharing
https://drive.google.com/file/d/0BzNjRb ... sp=sharing
https://drive.google.com/file/d/0BzNjRb ... sp=sharing
https://drive.google.com/file/d/0BzNjRb ... sp=sharing
https://drive.google.com/file/d/0BzNjRb ... sp=sharing

1296x972 is centered in modes 2 and 4. There is only slight shift between mode 2 and mode 4. You don't have to fix this issue but it would be good to confirm it's firmware bug and document it. I can replace mode 5 with mode 4 and use cropping to get 1280x720.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13546
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New feature implemented - forcing the sensor mode

Thu Jun 16, 2016 12:06 pm

Many thanks for doing the legwork - it means I can just look at the code rather than having to try reproducing problems.

From the images I reckon that it's 120 pixels off.

Readout should be 2592 x (2592/16*9) = 2592x1458, and then 2x2 binned to 1296x729. Odd numbers don't work, so we make it 1460 before binning, or 730 after.
That's 1944-1460 = 484 lines to lose. Centre crop and I'd expect 242 lines off top and bottom. The driver claims 240 lines (before binning) off the top, and 244 lines off the bottom which is a little odd.
In the register settings it looks like Y_START has been set to 120, except that is meant to be before binning value, so should be 240 - ah, out by 120 lines. Smoking gun!
(I'll skip using git blame so I don't self-incriminate!)

I will just check with Pi Towers for their view, but I think this should be fixed in the firmware. It is blatantly wrong, and anyone doing an upgrade should test again afterwards.

How much of a hacker are you? I wouldn't recommend this normally, but it's the easiest way to test.
If you use a hex editor (I use wxHexEditor under x86 Linux) on start_x.elf, search for the hex sequence 02387800 and there should be one instance. Amend that to 0238F000 (just change the one byte) and try it in the Pi. I'd hope that should give the correct FOV, or possibly those 2 pixels off mentioned earlier.
It may be that two changes of 02387800 to 0238F200, and 42 bytes later change 0738b300 to 0738b500 should shift down by those 2 lines and more accurately match.
It goes without saying DON'T DO THIS ON A CRITICAL PI!
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

semek
Posts: 3
Joined: Wed Jun 15, 2016 10:28 am

Re: New feature implemented - forcing the sensor mode

Thu Jun 16, 2016 1:58 pm

I've changed start_x.elf as you suggested but it doesn't help. With first modification it works the same way. After second modification capture in mode 5 is not working.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13546
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New feature implemented - forcing the sensor mode

Thu Jun 16, 2016 2:09 pm

semek wrote:I've changed start_x.elf as you suggested but it doesn't help. With first modification it works the same way. After second modification capture in mode 5 is not working.
Thanks for trying. I'll have to try this myself and get the firmware amended properly.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

barney-ng
Posts: 4
Joined: Sun Feb 11, 2018 10:43 am
Location: Hamburg, Germany

Re: New feature implemented - forcing the sensor mode

Tue May 01, 2018 4:55 pm

Hello 6by9 may I kindly ask if there is a plan to fix this someday?

With the OV5647 and a fisheye lens you are not able to adjust the camera properly in a straight direction.
The 49 frames / second delivered a good result in object detection for us. Now we have to go with 17% less in mode 4.

https://drive.google.com/file/d/1cghN3N ... sp=sharing


Thank you in advance

Barney

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 31838
Joined: Sat Jul 30, 2011 7:41 pm

Re: New feature implemented - forcing the sensor mode

Tue May 01, 2018 6:48 pm

Is this in our issue tracker? Worth checking, and adding if not. That way its more noticeable to the dev team, and 6by9, who now works for RPF(T)
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13546
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New feature implemented - forcing the sensor mode

Tue May 01, 2018 8:43 pm

barney-ng wrote:
Tue May 01, 2018 4:55 pm
Hello 6by9 may I kindly ask if there is a plan to fix this someday?
If you're referring to mode 5 being slightly off-centre vertically, then no I won't be fixing it.
The issue is that fixing it means that anyone who has a camera set up will suddenly get a field of view change should they upgrade the firmware. I recognise it's annoying but it'd be more annoying to those who get their system broken by an update.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

barney-ng
Posts: 4
Joined: Sun Feb 11, 2018 10:43 am
Location: Hamburg, Germany

Re: New feature implemented - forcing the sensor mode

Tue May 01, 2018 9:32 pm

6by9 wrote:
Tue May 01, 2018 8:43 pm
If you're referring to mode 5 being slightly off-centre vertically, then no I won't be fixing it.
Ok -- got it. Even if it is annoying ;-)

Thank you for all the time you supported us in the past years!

Barney

mamor
Posts: 20
Joined: Thu Nov 09, 2017 1:45 am

Re: New feature implemented - forcing the sensor mode

Mon Oct 22, 2018 6:41 pm

Hello:
So, what exactly does an 'automatic selection' (mode = 0) do?
Will it ever do binning?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13546
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: New feature implemented - forcing the sensor mode

Mon Oct 22, 2018 7:48 pm

mamor wrote:
Mon Oct 22, 2018 6:41 pm
Hello:
So, what exactly does an 'automatic selection' (mode = 0) do?
Will it ever do binning?
Yes it will do binning as that is the only way to increase the frame rate.
Read https://picamera.readthedocs.io/en/late ... nsor-modes for more info on the modes and the selection algorithm.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”