Degradation or ageing of camara sensor?

[hkoenig]

Hi,

my PiNoir camera takes a photo every 5 seconds since April. That's around 17.000 a day or half a million photos per month.

Since some time I notice a decreased sensitivity in the upper quarter of the picture. There's a sharp vertical line above which the picture is noticeably darker. In the attached picture it is 74 pixels from the top.
At first it was only noticeable if a person with a unicoloured shirt crossed that line. Now it's on every daylight picture.
Pictures are taken at 1296x972 but size and raspistill options don't make a difference. At night with infrared light this is barely noticeable if at all.

The camera lives outside but I never noticed above 72°C CPU temperature.
It's ventilated so I think there's no problem of moisture.

Any idea what the problem might be?
Is it possible that the camera module can't take this load?

Cheers,

Herbert

[jamesh]

I'm not aware of any issues like this. I'll ask around, but the sensor is standard, so I would expect it to last a fair time. I wonder if there is an issue with the lens or something underneath it. Fungus? Have you given it a good clean (you can take the lens off and clean underneath - very VERY carefully)

[RaTTuS]

can you post the full size image to imgur.com
so we can actually see something

[hkoenig]

Hi,

everything close to the lens gets blurred and its a sharp perferct horizontal line at y=256 from the top as can be seen on the picture at http://imgur.com/DSBAhL2.

If the person wore a red or yellow shirt the contrast would be bigger, in the blue channel the line is least visible.

IMHO a CCD sensor should not be affected by taking a few million photos but I have no idea as to what could produce this.

As long as it doesn't get much darker I can live with it, I'm only interested in the difference between two consecutive pictures. The camera and Raspi are about 3m above ground so I'll try to avoid dismounting and readjusting it and there's a faic chance to destroy the camara in the process of me dismounting the lens. .
The whole setup is mounted inside a fake surveillance camera with a plastic window in front of the lens. It hasn't been touched in months.

BTW I use uncompressed bmp files imgur made a jpeg from the upload.

Cheers,

Herbert

[RaTTuS]

H....
The whole setup is mounted inside a fake surveillance camera with a plastic window in front of the lens. It hasn't been touched in months. ...

could it be that the plastic window is deteriorating -

[jbeale]

It took me a while to see what you are talking about, but I do see it. I don't think that's anything optical; it looks like the top portion of the image is about 1/2 stop underexposed relative to the rest of the image (or maybe it's the bottom part that's overexposed) and the "horizontal line" is the pixel-sharp boundary between the two regions. I have (rarely) seen this on my cameras too. It seems to be possible, under some circumstances to get a frame buffer from the RPi camera which contains a combination of two separate exposures. I'm not sure what those circumstances are, possibly more likely with the first buffer returned after the camera is powered up. What does your code look like? Is it raspistill in timelapse mode, or something else? Are the camera software & firmware versions current?

I wonder if it is related to this issue, which was also on images taken at fixed time intervals, where the problem only appeared after running a long time, but a reboot fixed it: viewtopic.php?f=43&t=118139

[hkoenig]

I have sometimes seen this on my cameras too. It seems to be possible, under some circumstances to get a frame buffer from the RPi camera which contains a combination of two separate exposures. I'm not sure what those circumstances are, possibly more likely with the first buffer returned after the camera is powered up. What does your code look like? Is it raspistill in timelapse mode, or something else? Are the camera software & firmware versions current?

Hi,
the images are taken via
raspistill -e bmp -n -tl 5000 -w 1296 -h 972 -ex auto -drc low -br 49 -co 30 -sh 40 -mm matrix -t 400000000 -o /dev/shm/%d.bmp.
When the sensor in an infrared floodlight switches the floodlight on, raspistill is killed (pkill raspistill) and restarted with changed exposure to -ex night -drc low -co 30 -sh 40 -mm matrix. And reverse when daylight comes up.
Another program picks the images out of the ranmdisk deletes them and does the analysis (for cats abusing the garden as a toilet. really).

With the lower resolution I get away with a video memeory of 64MB instead of 128. (RasPi A+)
I checked that one too, checked full resolution and no options at all. Same result, except different y coordinate. Also disconnected the power and restarted.
Cheers,

Herbert

[hkoenig]

Sorry forgot:
Latest kernel and software I assume:
Linux herpi4 4.1.6+ #810 PREEMPT Tue Aug 18 15:19:58 BST 2015 armv6l GNU/Linux
raspistill Camera App v1.3.8

[6by9]

Please try without DRC - that's the one postprocessing algorithm that I wouldn't guarantee the quality of, and could quite possibly give the sort of effects you're seeing.

[hkoenig]

Hi,

Please try without DRC - that's the one postprocessing algorithm that I wouldn't guarantee the quality of, and could quite possibly give the sort of effects you're seeing.

The cause is not drc but saving in bmp format. The other options only influence the vertical position of the "line" but saving as jpeg dindn't produce the error regardless of drc, all tests with -e bmp showed the line.

For speed reasons I need an uncompressed format because while the compression is hardware accelerated, the decomression in the analysing program is not and I'd have to lower the framerate.

Maybe someone else can verify if it's only my board? If your camera is accessible, a uniform reddish or greeneish background shows the line most clearly, in blue it's harder to see.

<rant> I spent some very healthy hours. It's raining, the power for the camera pi is in the cellar, the computer controlling it via VNC is on the second floor and for each photo I ran into the garden to give some uniform colour which makes the line more visible </rant>

Cheers,

Herbert

[jbeale]

Definitely, BMP output should not have such a bug. However it does seem that one image at 1296x972 resolution every five seconds is not particularly fast. How long does your Pi take to handle a JPEG of that size?

On my RPi-2, where t.jpg is a 1296 x 972 image from the RPi camera, 'time convert t.jpg t.bmp' yields 0.455 seconds. 'time convert t.bmp t2.bmp' takes 0.219 seconds. So the advantage of the uncompressed format is less than a quarter of a second, and that is only about 10% of your 5-second interval.

[jbeale]

By the way, I confirmed the "horizontal line" artifact with raspistill BMP output. Here is a white sheet of paper at an angle.
(EDIT: ignore the radial color gradient; that camera has an aftermarket lens and the radial color is a known consequence due to ISP baked-in lens shading corrections and/or sensor microlens array)

https://picasaweb.google.com/1099282360 ... 8666987554

[jamesh]

Hmm, defo. something awry there. I no longer have access to the source code, so not able to check on that. 6x9 will have to do it.

[hkoenig]

Hi,

Definitely, BMP output should not have such a bug. However it does seem that one image at 1296x972 resolution every five seconds is not particularly fast. How long does your Pi take to handle a JPEG of that size?

On my RPi-2, where t.jpg is a 1296 x 972 image from the RPi camera, 'time convert t.jpg t.bmp' yields 0.455 seconds. 'time convert t.bmp t2.bmp' takes 0.219 seconds. So the advantage of the uncompressed format is less than a quarter of a second, and that is only about 10% of your 5-second interval.

thanks for taking the time to check!
On my A+
Time millisecondsToRun: [Form fromFileNamed: '/dev/shm/1.jpg']
takes 1.47 seconds on average. The same with a bmp takes 0.22 seconds on average. Form is the class of the objects I use for the analysys. SCNR to drop some Smalltalk on you
The analysys eats about 50 percent CPU (top with an interval of 20 s) raspistill around 1.3 %, tightvnc around 1 %.
When I sftp the photos that raised an alarm out of the Pi (only a 12V power chord to the whole setup) I have to limit the transfer rate to 700 kilobytes per second to not loose images. So I have no CPU cycles to spare.

Cheers,

Herbert

[timrowledge]

Since nearly all the time for reading a jpeg is in the C code of a Smalltalk plugin I don't think we can do all that much to speed it up - other than improving the C code, obviously. It's built from a slightly out of date copy of the JPEG group's sample library that we have to keep because the latest (at least as of last time anyone had time to check) version provided in Debian is just plain broken in some manner that is/was considered important. If anyone knows a better/faster/more-chocolatey jpeg library, do let us know.

On my B+ loading a similar (1146x1110) size jpeg takes 1400 mS, on the Pi-2 it takes 730. Going right to the primitive level where the work is passed to the plugin saves ~30mS on the B+.

[6by9]

Ok, will add looking at bmp captures to my long list of things to look at.
I'm hoping it is the colourspace info for the yuv to rgb conversion changing after the first stripe as that is just a signalling issue. If it's actually in the codec then I really don't fancy digging.

It sounds like you'd be better taking raspistillyuv as a basis to avoid any form of conversion - depends where any bandwidth limit is.

[hkoenig]

Hi,

It sounds like you'd be better taking raspistillyuv as a basis to avoid any form of conversion - depends where any bandwidth limit is.

Seems to make a lot of sense but first I have to learn how to get a raw rgb file into Squeak. I'll report back. Right now I have no means to view these files.

BTW nowadays the program is called raspiyuv, no 'still' in the middle.

Cheers,

Herbert

[6by9]

BTW nowadays the program is called raspiyuv, no 'still' in the middle.

I was writing it on a phone and without a Pi or source easily to hand. You got the program I meant.

[hkoenig]

Hi,

It sounds like you'd be better taking raspistillyuv as a basis to avoid any form of conversion - depends where any bandwidth limit is.

Seems to make a lot of sense but first I have to learn how to get a raw rgb file into Squeak. I'll report back. Right now I have no means to view these files.

BTW nowadays the program is called raspiyuv, no 'still' in the middle.

Cheers,

Herbert

I can confirm raspiyuv's rgb mode does not have the problem. If someone's interested I can email a rgb and a bmp file.
I could only test 1280x972 image size as I didn't find documentation on how the padding of rows happens. 1296x972 which I use to take advantage of the camera's binning capability already does padding and my (actually Tim's, thanks) simple approach doesn't take padding into account.

As said earlier I can live with the error and bmp reading is 5 times faster than manual rgb importing.

Next I'll see where I can request to change the camara board documentation to point to raspiyuv instead of raspistillyuv.

[6by9]

I can confirm raspiyuv's rgb mode does not have the problem. If someone's interested I can email a rgb and a bmp file.
I could only test 1280x972 image size as I didn't find documentation on how the padding of rows happens. 1296x972 which I use to take advantage of the camera's binning capability already does padding and my (actually Tim's, thanks) simple approach doesn't take padding into account.

Width is always padded to a multiple of 32, and height to a multiple of 16. So 1296x972 will appear as a buffer of 1312x976.

As said earlier I can live with the error and bmp reading is 5 times faster than manual rgb importing.

Something very wrong in your RGB import then, as the compression available with BMP is unlikely to reduce size so significantly that the memory bandwidth change is less significant than the RLE compression.

Next I'll see where I can request to change the camara board documentation to point to raspiyuv instead of raspistillyuv.

Assuming it is the main docs, then git clone https://github.com/raspberrypi/documentation, amend, and create a pull request.

[jamesh]

I definitely should have called it rapistillyuv. But at the time there wasn't a video version...