User avatar
yv1hx
Posts: 384
Joined: Sat Jul 21, 2012 10:09 pm
Location: Now an expatriate, originally from Zulia, Venezuela
Contact: Website Skype

Long exposure imaging for capturing lighting flashes

Fri Oct 17, 2014 5:48 pm

Hello RPi fellas,

I have used in the past my RPi + Camera Board for taking stills to create time-lapse videos with great success, but now I would to like to capture some long exposure images at night (about 5 minutes per image), my primary goal is trying to capture lighting flashes during a thunderstorm using the same hardware, so I have three concrete questions:

a) The standard camera board (Not the Pi-Noir) will be suitable for this purpose ?

b) What should be the "recommended" settings for the camera? I'm planning to use the "raspistill" command.

c) There is any advantage in the image quality using either .jpg or .png formats? The disk space is not a concern by now.

Thanks in advance.
Marco-Luis
Telecom Specialist (Now Available for Hire!)

http://www.meteoven.org
http://twitter.com/yv1hx

ethanol100
Posts: 667
Joined: Wed Oct 02, 2013 12:28 pm

Re: Long exposure imaging for capturing lighting flashes

Fri Oct 17, 2014 9:17 pm

a) The longest possible exposure is 6sec. Additionally the "rolling shutter" can have strange effects, the bottom rows of the sensors are exposed later than the top rows. I have not tried capturing lighting flashes, but I think it should work. But it will be quite hard to get the right timing.

b) I think there are no "recommend" settings. If you do not need the preview you can start with
"raspistill -v -fp -bm -tl 6000 -ss 6000000 -ISO 800 -awb off -awbg 1,1 -t 600000 -o %04d.jpg".
This would create a six second long exposed images all six seconds for 10 min. If the images are overexposed you can reduce the ISO value to 100. The awb gains depend of the lighting of the scene and can be varied between 0 and 6.
You can take an automatic adjusted image using "raspistill -ss 6000000 -t 60000 -o test.jpg" giving the calibration algorithm enough time to find a good setting. Then you can use "strings test.jpg| grep gain" to find some starting values(take the values for gain_r and gain_b).

c) ".png" is a lossless compression, it is just like "zip" and can be decompressed without quality loss. While ".jpg" will compress the image by "simplifying" the image (which removes changes not so much visible to the human eye). The amount of simplification is set with the Quantization value("-q"), a smaller value simplifies more than a greater ones(value can be in the range from 1 to 100). I would argue that the compression used by default is reasonable. Additionally you can save the "raw" Bayer data from the sensor with "-r", which contain a higher dynamic range but it has not be processed/optimized by the "Image processor" and needs much post processing.

Jpg is the only "hardware" accelerated format which utilize the gpu. All other image formats are compressed using the arm cpu which is quite slow.

Good luck hunting lightnings...

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11815
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Long exposure imaging for capturing lighting flashes

Sat Oct 18, 2014 7:48 am

ethanol100 wrote:c) ".png" is a lossless compression, it is just like "zip" and can be decompressed without quality loss. While ".jpg" will compress the image by "simplifying" the image (which removes changes not so much visible to the human eye). The amount of simplification is set with the Quantization value("-q"), a smaller value simplifies more than a greater ones(value can be in the range from 1 to 100). I would argue that the compression used by default is reasonable. Additionally you can save the "raw" Bayer data from the sensor with "-r", which contain a higher dynamic range but it has not be processed/optimized by the "Image processor" and needs much post processing.
PNG is also an RGB format (not YUV), so it requires a format conversion. RGB doesn't have the "benefit" of chroma subsampling, so you're adding more information to compress with the conversion.
ethanol100 wrote:Jpg is the only "hardware" accelerated format which utilize the gpu. All other image formats are compressed using the arm cpu which is quite slow.
Nearly right. With raspistill / MMAL, all other formats are encoded on the GPU in software, not on the ARM. It is still quite a fair amount slower than the JPEG h/w block though.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

PiGraham
Posts: 4862
Joined: Fri Jun 07, 2013 12:37 pm
Location: Waterlooville

Re: Long exposure imaging for capturing lighting flashes

Sat Oct 18, 2014 8:14 am

FLashes and motion aren't captured well with the sort of rolling shutter sensor used in the Pi Camera. You may be lucky, using long exposures, but there is a significant chance that you will see banding on your images.

See these examples.

User avatar
jbeale
Posts: 3893
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Long exposure imaging for capturing lighting flashes

Sat Oct 18, 2014 6:05 pm

If you use a 6 second shutter, I think you have better than 90% chance of not having banding, because (AFIK) the sensor readout time even with the slower clock is measured in milliseconds, not seconds. I'm pretty sure the examples of banding linked to in the last post were using more "normal" camera settings with sub-second exposures, in which case the sensor readout time would be a much larger fraction of the total exposure time, producing the banding / split frames more frequently.

User avatar
yv1hx
Posts: 384
Joined: Sat Jul 21, 2012 10:09 pm
Location: Now an expatriate, originally from Zulia, Venezuela
Contact: Website Skype

Re: Long exposure imaging for capturing lighting flashes

Sun Oct 19, 2014 10:41 pm

Thanks you all for your thoughts and suggestions, now I'm pretty sure about the feasibility of this experiment.

My next step is find/build a suitable waterproof cabinet able to endure our harsh environmental conditions for housing the components.

I'll keep you posted..
Marco-Luis
Telecom Specialist (Now Available for Hire!)

http://www.meteoven.org
http://twitter.com/yv1hx

nic32seajunk
Posts: 16
Joined: Thu Oct 23, 2014 3:05 pm

Re: Long exposure imaging for capturing lighting flashes

Thu Oct 23, 2014 3:13 pm

Hi....I'm new to raspi but am having fun doing the timelapse photos using the pi....I too would like to do longer exposures. I tried the command provided

Code: Select all

[raspistill -v -fp -bm -tl 6000 -ss 6000000 -ISO 800 -awb off -awbg 1,1 -t 600000 -o %04d.jpg/code]
but get an error:
"invalid command line option (-tl)"
I've also tried changing the ISO but when looking at the output jpg, I don't see any differance between exposures.... I've tried 
-ex verylong and don't see any change....
Am I doing something wrong?
Thanks in advance...
Neil

ethanol100
Posts: 667
Joined: Wed Oct 02, 2013 12:28 pm

Re: Long exposure imaging for capturing lighting flashes

Fri Oct 24, 2014 3:31 pm

I guess you are not using the latest firmware. The "-bm" switch is not yet released with the normal package manager. You can just skip some parameter and start with a minimum of paramters:

Code: Select all

 raspistill -v -ISO 800 -ss 6000000 -tl 60000 -t 3600000 -o %04d.jpg
You will not be able to get one image each 6s but rather one image a minute without the "burst capture mode". The important part is the "--ss" option which sets the exposure length in µs.

Generally it does not make sense to set the shutter speed and iso value and to set an exposure mode. You can try to not set the shutter speed and ISO, but using the exposure mode -ex night. This allows up to 1s exposure with a higher ISO value(up to 1600).

Alternatively, to get the newest firmware you can use the program "rpi-update".

Finally, to check if raspistill uses the right values for shutter speed and ISO, you can use the program strings to extract them. You can test it with i.e.

Code: Select all

raspistill -ss 6000000 -t 60000 -o 6s.jpg
strings 6s.jpg | grep gain


This gives you a line like this:

Code: Select all

ev=-1 mlux=-1 exp=5969487 ag=256 focus=255 gain_r=1.152 gain_b=1.160 greenness=-59 ccm=6058,-1704,-254,-348,4598,-150,888,-1788,5002,0,0,0 md=0 tg=39 39 oth=0 0 b=0 f=39 39 fi=0 ISP Build Date: Oct 17 2014, 17:52:51 VC_BUILD_ID_VERSION: 845092531e360acd37f2f1964157f8079c77edfb (clean) VC_BUILD_ID_USER: dc4 VC_BUILD_ID_BRANCH: master 
Here the value exp=5969487 describes the shutter speed which was 5.968487s.
The ag=256 is the analog gain, for the long exposure values we can have values from 256 (which mean ISO 100) up to 2048 (which would mean ISO 800).

nic32seajunk
Posts: 16
Joined: Thu Oct 23, 2014 3:05 pm

Re: Long exposure imaging for capturing lighting flashes

Sat Oct 25, 2014 2:34 pm

Ethanol100...Thanks for taking the time to explain a few things! First thing I did was the update. After the reboot all was magic. Now I just need to continue testing. All the best to you and yours.
And hello to yv1hx.....de wa4chq/qrp (cw)
nic32seajunk

User avatar
yv1hx
Posts: 384
Joined: Sat Jul 21, 2012 10:09 pm
Location: Now an expatriate, originally from Zulia, Venezuela
Contact: Website Skype

Re: Long exposure imaging for capturing lighting flashes

Sat Oct 25, 2014 4:51 pm

nic32seajunk wrote:[snip...]
And hello to yv1hx.....de wa4chq/qrp (cw)
nic32seajunk
wa4chq 73's & DX to wa4chq
Marco-Luis
Telecom Specialist (Now Available for Hire!)

http://www.meteoven.org
http://twitter.com/yv1hx

007
Posts: 16
Joined: Thu Aug 01, 2013 9:29 am

Re: Long exposure imaging for capturing lighting flashes

Fri Oct 31, 2014 8:15 pm

Hi!

Maybe this is your solution? It has 10 second to 3600 second exposure :-D https://sites.google.com/site/meteotuxpi/

riklaunim
Posts: 269
Joined: Tue Apr 22, 2014 7:34 pm

Re: Long exposure imaging for capturing lighting flashes

Fri Oct 31, 2014 10:47 pm

You don't need long exposures, but just AVI recording and getting out the frames that caught the lightning. I used a webcam that way. Long exposures with noisy and uncooled sensor will just give you very bad resuts. Not to mention that it will easily saturate at X minute exposures when using high gain.

User avatar
yv1hx
Posts: 384
Joined: Sat Jul 21, 2012 10:09 pm
Location: Now an expatriate, originally from Zulia, Venezuela
Contact: Website Skype

Re: Long exposure imaging for capturing lighting flashes

Sun Nov 02, 2014 2:03 am

007 wrote:Hi!

Maybe this is your solution? It has 10 second to 3600 second exposure :-D https://sites.google.com/site/meteotuxpi/
Thanks for your suggestion, 007, but the author of that program recommends a pi noir camera http://www.raspberrypi.org/products/camera-module/, I don't have such device.

However, I'll give a try when I can solve the outdoors enclosure problem.
Marco-Luis
Telecom Specialist (Now Available for Hire!)

http://www.meteoven.org
http://twitter.com/yv1hx

User avatar
Amarand
Posts: 19
Joined: Tue Apr 28, 2015 7:32 pm
Location: Galloway, OH, USA
Contact: Website Yahoo Messenger Facebook YouTube

Re: Long exposure imaging for capturing lighting flashes

Tue Apr 28, 2015 7:56 pm

yv1hx wrote:
nic32seajunk wrote:[snip...]
And hello to yv1hx.....de wa4chq/qrp (cw)
nic32seajunk
wa4chq 73's & DX to wa4chq
Useful post, thanks everyone!

73/DX de WB1HIH!

User avatar
Amarand
Posts: 19
Joined: Tue Apr 28, 2015 7:32 pm
Location: Galloway, OH, USA
Contact: Website Yahoo Messenger Facebook YouTube

Re: Long exposure imaging for capturing lighting flashes

Tue Apr 28, 2015 7:59 pm

ethanol100 wrote:I guess you are not using the latest firmware. The "-bm" switch is not yet released with the normal package manager. You can just skip some parameter and start with a minimum of paramters:

Code: Select all

 raspistill -v -ISO 800 -ss 6000000 -tl 60000 -t 3600000 -o %04d.jpg
So here's a silly question (perhaps)....

I followed the directions for getting the IR camera set-up, and although I went through all of the set-up steps of making sure the OS and firmware were updated, I still don't have the -tl for raspistill. Is there a better distro or location I should be using to grab the latest version of this? Also, is raspistill a part of the firmware (rpi-update) or the OS (apt-get)?

ethanol100
Posts: 667
Joined: Wed Oct 02, 2013 12:28 pm

Re: Long exposure imaging for capturing lighting flashes

Tue Apr 28, 2015 8:44 pm

The only really supported distro is still "raspbian". raspistill is part of the package "libraspberrypi-bin" and part of the "userland" which is updated with rpi-update. If you have a standard raspbian installation(with noobs or from the Raspbian image) you can get the last "stable" version with

Code: Select all

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
Only if you need the "bleeding edge" version for new features you could run "rpi-update" which will overwrite the version from the repositories.
The version in the repository is "1.20150212-1" from the 12. Feb. 2015. You can check your version with "dpkg -l | grep libraspberrypi".
Which command did you use for raspistill? Can you invoke raspistill without any argument and see if you can find the -tl switch?

User avatar
Amarand
Posts: 19
Joined: Tue Apr 28, 2015 7:32 pm
Location: Galloway, OH, USA
Contact: Website Yahoo Messenger Facebook YouTube

Re: Long exposure imaging for capturing lighting flashes

Wed Apr 29, 2015 1:35 pm

ethanol100 wrote:The only really supported distro is still "raspbian". raspistill is part of the package "libraspberrypi-bin" and part of the "userland" which is updated with rpi-update. If you have a standard raspbian installation(with noobs or from the Raspbian image) you can get the last "stable" version with

Code: Select all

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
Only if you need the "bleeding edge" version for new features you could run "rpi-update" which will overwrite the version from the repositories.
The version in the repository is "1.20150212-1" from the 12. Feb. 2015. You can check your version with "dpkg -l | grep libraspberrypi".
Which command did you use for raspistill? Can you invoke raspistill without any argument and see if you can find the -tl switch?
It's so weird, because after doing all of that, and rebooting multiple times, I still had that error...until I brought it to the library, did a demo (off the Internet, no access to the outside world), brought it back to the house and now all of a sudden, it's working. :)

Yesterday, it would show the -tl but it would complain about it not being an option. So weird!

User avatar
DougieLawson
Posts: 41777
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Long exposure imaging for capturing lighting flashes

Wed Apr 29, 2015 1:37 pm

Amarand wrote: I followed the directions for getting the IR camera set-up, and although I went through all of the set-up steps of making sure the OS and firmware were updated, I still don't have the -tl for raspistill. Is there a better distro or location I should be using to grab the latest version of this? Also, is raspistill a part of the firmware (rpi-update) or the OS (apt-get)?
git clone http://github.com/raspberrypi/userland
cd userland
./buildme
Languages using left-hand whitespace for syntax are ridiculous

DMs sent on Twitter/LinkedIn will be answered next month.
Fake doctors - are all on my foes list.

The use of crystal balls and mind reading is prohibited.

User avatar
Amarand
Posts: 19
Joined: Tue Apr 28, 2015 7:32 pm
Location: Galloway, OH, USA
Contact: Website Yahoo Messenger Facebook YouTube

Re: Long exposure imaging for capturing lighting flashes

Wed Apr 29, 2015 2:26 pm

DougieLawson wrote:git clone http://github.com/raspberrypi/userland
cd userland
./buildme
Awesome, thank you! So much to learn....

Return to “Camera board”