Permanently turn AWB off

[poing]

how in the world do you get the raw?...

http://www.raspberrypi.org/phpBB3/viewt ... 25#p361647

[jamesh]

I'll do anything to get rid of this AWB even if it means dealing with the raw....
but....
how in the world do you get the raw?... i'm guessing the challenge of getting it to work in opencv is my own challenge

it's hard to believe we cant tell the chip to just NOT apply AWB, or just apply our own manual settings for, the camera should be able to handle that.... why cant our gpu?

There speaketh the person who hasn't seen the source code.

Camera code is very complicated. AWB has never been necessary in all the millions of phones that use the software, so until now, no-one has wanted it, so there was no idea it didn't work. So, I need to work out how to turn it off properly.

[poing]

AWB has never been necessary in all the millions of phones that use the software, so until now, no-one has wanted it

Raspberry Pi RULEZ!!!

[jamesh]

That should read "AWB OFF has never been....".

Of course, all the phones with the VC4 ISP in have AWB! (Including the Nokia 808, still the best camera in a phone)

[SnowLeopard]

Figured I'd let those that know better decide if this is useful information:
Android v4l2 code relating to the OV5647 sensor. https://bitbucket.org/ayysir/android_ke ... at=cm-10.1
Line 308 has a comment regarding the following lines: /* manual AWB,manual AE,close Lenc,open WBC*/

[jamesh]

The module itself does have an internal ISP which can do stuff like AWB etc, but all that is disabled and we use the ISP built in to the GPU. A lot of register setting you find on other product will be using the modules on board ISP.

The onboard ISP's are generally of lower quality than the GPU ones, but are cheaper to integrate in to product.

[jbeale]

I'm (trying to) use the R-Pi camera with fixed ISO and fixed shutter, in an application where the fixed lighting is very blue. That lighting condition is apparently not a "baysean match" to any known lighting model. The observed behavior is that the image gets lighter and darker and changes color seemingly at random, even setting AWB to a fixed setting and with the same incident light level. As far as I know RAW is my only option for now, but that is painfully slow. I need to take many photos at relatively low resolution, but I believe RAW comes only in full-resolution. Processing all those pixels on the ARM would be at least an order of magnitude too slow.

So (to pile on the bandwagon) it would be really great if we could turn off AWB and get a true "fixed white balance" mode.

[poing]

I think some color correction is applied even when shooting RAW, or there's something going on I don't understand well. Here's a RAW image with stock lens:

000098small.jpg (47.82 KiB) Viewed 7946 times

[jamesh]

No, raw is raw straight off the sensor. Although in the ISP we do do some processing on the bayer data (black level, defective pixel, WB), the raw data is extracted before any of these.

I think what you are seeing on that image is a lack of lens shading correction. Which is correct for a raw image - no correction has been done.

[jamesh]

I'm (trying to) use the R-Pi camera with fixed ISO and fixed shutter, in an application where the fixed lighting is very blue. That lighting condition is apparently not a "baysean match" to any known lighting model. The observed behavior is that the image gets lighter and darker and changes color seemingly at random, even setting AWB to a fixed setting and with the same incident light level. As far as I know RAW is my only option for now, but that is painfully slow. I need to take many photos at relatively low resolution, but I believe RAW comes only in full-resolution. Processing all those pixels on the ARM would be at least an order of magnitude too slow.

So (to pile on the bandwagon) it would be really great if we could turn off AWB and get a true "fixed white balance" mode.

Once I sort out the shutter speed issue (mostly there), if I have time I'll take a look at this. I have a feeling we have a fix for something similar on another source tree for another project, but whether it will merge I don't know.

[paranoidcoder]

We found a solution:
we are using robidouille (wrapper for opencv) to capture frames.

In robidouille/raspicam_cv.c
we added this method:
void set_awb_off() {
raspicamcontrol_set_awb_mode(camera, 0);
}
To take a snapshot, we wrapped it in a nice little method that sets awb off, then takes a new image
(include raspicamvc)
IplImage* snapshot(RaspiCamCvCapture *capture) {
set_awb_off();
return raspiCamCvQueryFrame(capture);
}

you need to call awb off for every frame. We still get 20fps with this implementation.

[jamesh]

That interesting. I knew that setting it to off only worked for one frame, but there is not reason why you cannot keep setting it to off.

Although that doesn't work for video as that is stream based rather than frame based, although you might be able to bodge something in the callback...hmmm...

[ethanol100]

The mentioned program uses the video port to capture yuv images to a buffer. It will not work for the still port. Have tried to set it in raspistill in the same way as the shuter speed and for i.e. only red backgrounds it gets gray again. So he awb can't be disabled like this for the still port.

[paranoidcoder]

For our uses, we got it to do blob tracking. Which is something we previously thought would be impossible! so if you're doing blob tracking with opencv you should be able to use this just fine

[lzars]

Any word on turning AWB off?

[elnexo]

I don't have a pi but work with Omnivision sensors and others.
Most of the sensors have awb intelligence built in. thus, you can set it up via i2c register settings.

- get datasheet of the camera chip.
- find auto white balance section.
- use i2c command to set register of Awb control to fix current White balance gains
- make sure you are pointing to some reference white with the camera

if awb is not built in and is done outside the chip it is still trivial to determine the correct white balance gains if you point the camera to a reference white and then after white balancing fix the gains. This of corse is only if you need to make sure that white appears white and the other colors just stay fixed. For a full color correction with chromatic adaptation you need to either use a LUT or matrix. I don't think this is the case here, though.

I'll be glad to look into this further if needed.

[elnexo]

Ok checked Datasheet on (http://dlnmh9ip6v2uc.cloudfront.net/dat ... 7_full.pdf)
I believe the Pi camera used the OV5647, right?

Check out register 0x5001 bit[1] awb_en
But, this is not what actually will fix your issue alone.
I believe the key register here is 0x5180 bit [5] freeze_gain_en. This should freeze the gains of the current white balance gains determined by AWB algorithm.

In register 0x5180 you can also force to use manual gains. That probably is where you can load in presets. Just check out the data sheet section 5.4. All registers are described there.

Registe 0x5002 bit[0] (awe_gain_en) is probably 1, but you need to take that one also into account.

All these things should be set by the camera driver.

What is needed is a way to store the previous white balance gains used and when the camera is started again those values need to be loaded. Otherwise, every time the camera is used you have to go through white balance or use a preset as mentioned in other posts. But sometimes presets are not good enough. Thus, a storage of previously used white balance gains would be necessary.

When I get my pi and camera board I'll need to fix this anyways. Will give update when ready.

[jamesh]

We don't use the on chip AWB, its all done in the GPU.

And you cannot access the camera registers from ARM space.

[elnexo]

We don't use the on chip AWB, its all done in the GPU.

And you cannot access the camera registers from ARM space.

The pi camera implementation of the CSI interface does not expose an i2C (SDA SCL) interface?
Are you using the default register settings of the sensor then? It would be really great if the registers could be set. Then Auto exposure settings, AWB and other features of the sensor could be used.
Where can I find out more about the implementation details of the camera, like schematics sensor setup?
Thanks.

[jbeale]

The camera schematics have not been released. The OmniVision camera register configurations do not affect output white balance in this case, because the R-Pi implementation bypasses the onchip processing and always takes pure raw frames from the sensor, then does all the processing onboard the GPU. JamesH and the sensor team have stated that this approach gives a better image, due to superior horsepower and algorithms on the GPU, and I believe that. However, the firmware running on the GPU, including the camera pipeline is not open and not publicly available.

If you're really keen on doing your own white balance you can get the raw data and manipulate it yourself, see for example http://bealecorner.org/best/RPi/
The problem with that is you are stuck with the relatively weak ARMv6 CPU, even the bandwidth to just transfer the raw data is painfully limited when considering a 5 MP frame, let alone the processing power to transform it into a usable image. If you are willing to wait 30 seconds or more for one image, then you are OK. Otherwise the GPU is the right tool for that job and since the camera pipeline is proprietary, you get what you get.

[jdb]

We don't use the on chip AWB, its all done in the GPU.

And you cannot access the camera registers from ARM space.

The pi camera implementation of the CSI interface does not expose an i2C (SDA SCL) interface?
Are you using the default register settings of the sensor then? It would be really great if the registers could be set. Then Auto exposure settings, AWB and other features of the sensor could be used.
Where can I find out more about the implementation details of the camera, like schematics sensor setup?
Thanks.

To clarify:

The OV5647 sensor is set up and operated using i2c. The CSI interface is the high-speed LVDS communication channel that pumps high-definition video data through 6 wires.

The GPU expects to have exclusive access to the i2c bus accessing the camera: in theory you could snoop on the registers being read/written either through spamming reads to the i2c peripheral from the ARM or through a i2c bus analyzer, but like Jamesh says all the real processing is done on the GPU.

[elnexo]

To clarify:

The OV5647 sensor is set up and operated using i2c. The CSI interface is the high-speed LVDS communication channel that pumps high-definition video data through 6 wires.
.

When you say GPU, you mean that kind of graphical processing unit that you can use openGL program it, right? So probably the processing engine is using GLSL to do the color correction or just white balance (once you determined the right gains with the auto detection in AWB)?

My understanding is that the code for it is proprietary? But, do I have all the information needed to to get started making my own engine. I have some experience in OpenGL and OpenCL creating image processing pipelines but more on PC hardware which is less restrained compared to embedded implementations of GPUs.

Can you point me to some resources how to get started on the Raspberry pi?

The GPU expects to have exclusive access to the i2c bus accessing the camera: in theory you could snoop on the registers being read/written either through spamming reads to the i2c peripheral from the ARM or through a i2c bus analyzer, but like Jamesh says all the real processing is done on the GPU.

But could I write my own i2c driver and get access to the i2c interface? The pins are still accessible through the CPU, or is it completely separated from the CPU?

[jamesh]

No, the GPU contains a full camera ISP, independent of any other GPU features. It's a pretty good one (as used in the Nokia 808, still regarded as one of the best camera phones), certainly better than the inbuilt on in the sensor. It's implemented in mostly HW blocks that are managed by the vector/scaler processing core on the GPU. All the code and documentation for this is closed source - it's Broadcom proprietary information. So there is very little a third party can do to change it.

[elnexo]

No, the GPU contains a full camera ISP, independent of any other GPU features. It's a pretty good one (as used in the Nokia 808, still regarded as one of the best camera phones), certainly better than the inbuilt on in the sensor. It's implemented in mostly HW blocks that are managed by the vector/scaler processing core on the GPU.

Ok, now it actually makes more sense. I was already wondering what a great performing "GPU" you had on that chip not taking into account that the Image processing would be done mostly in seperate ISP HW blocks.

As a workaround for "AWB off" you suggested to use presets or fixed modes. Is there a way to modify the settings in the fixed modes to constrain to very little changes and even use a specific color correction? If so, can you point me to an example?

[jamesh]

No, the GPU contains a full camera ISP, independent of any other GPU features. It's a pretty good one (as used in the Nokia 808, still regarded as one of the best camera phones), certainly better than the inbuilt on in the sensor. It's implemented in mostly HW blocks that are managed by the vector/scaler processing core on the GPU.

Ok, now it actually makes more sense. I was already wondering what a great performing "GPU" you had on that chip not taking into account that the Image processing would be done mostly in seperate ISP HW blocks.

As a workaround for "AWB off" you suggested to use presets or fixed modes. Is there a way to modify the settings in the fixed modes to constrain to very little changes and even use a specific color correction? If so, can you point me to an example?

No, I don't think you can do that. Woud be nice to be able to use a fixed CCM but fas with all these things its finding the time to implement it on the GPU and expose the controls to the ARM. Its quite long-winded.