User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

diy autofocus with /dev/video16 and stepper motor?

Wed Jun 24, 2020 10:25 pm

I added a 28BYJ-48 stepper motor to microscope stand I use for HQ camera microscope:
Image

With full-stepping it allows to move in steps of 8.8µm (with half-stepping 4.4µm per step, with microstepping even less).
This 1360x768 animation is built from screenshots after moving one step down in z direction repeatedly:
viewtopic.php?f=43&t=210605&start=100#p1679177
Image

You can see new raspistill "--focus" feature in action.
6by9 wrote:
Tue May 19, 2020 9:13 am
rpi-5.4.y kernel.
You'll have /dev/video13 to /dev/video16. 13 is the input, 14 is the high res output, 15 is the low res output, and 16 is the stats output.

The format of the stats is as documented in https://github.com/raspberrypi/linux/bl ... -stats.rst and the struct at https://github.com/raspberrypi/linux/bl ... isp.h#L310
I have one SD card where I have executed "rpi-update" and therefore have access to /dev/video16.
But trying to read from that file using fopen() and fread() always returns 0 bytes only.

What is the correct way to read data from /dev/video16?
Can it be done with fopen() and fread()?
Does "raspistill --focus" or other code need to be running while reading statistics?
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 16115
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: diy autofocus with /dev/video16 and stepper motor?

Thu Jun 25, 2020 6:43 am

HermannSW wrote:
Wed Jun 24, 2020 10:25 pm
I have one SD card where I have executed "rpi-update" and therefore have access to /dev/video16.
But trying to read from that file using fopen() and fread() always returns 0 bytes only.

What is the correct way to read data from /dev/video16?
Can it be done with fopen() and fread()?
Does "raspistill --focus" or other code need to be running while reading statistics?
It's a memory to memory device, not a capture device.
You need to capture the raw frames (probably using /dev/video0 from bcm2835-unicam, but could be a file) and feed them into /dev/video13 (the input device). You can then dequeue buffers from /dev/video14 (high res output) and /dev/video16 (stats).
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

naushir
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 453
Joined: Mon Apr 25, 2016 10:21 am

Re: diy autofocus with /dev/video16 and stepper motor?

Thu Jun 25, 2020 7:37 am

6by9 wrote:
Thu Jun 25, 2020 6:43 am
HermannSW wrote:
Wed Jun 24, 2020 10:25 pm
I have one SD card where I have executed "rpi-update" and therefore have access to /dev/video16.
But trying to read from that file using fopen() and fread() always returns 0 bytes only.

What is the correct way to read data from /dev/video16?
Can it be done with fopen() and fread()?
Does "raspistill --focus" or other code need to be running while reading statistics?
It's a memory to memory device, not a capture device.
You need to capture the raw frames (probably using /dev/video0 from bcm2835-unicam, but could be a file) and feed them into /dev/video13 (the input device). You can then dequeue buffers from /dev/video14 (high res output) and /dev/video16 (stats).
An alternative is to use the libcamera stack, where we are *just* about to merge a change that provides the focus FoM to the application. Using the libcamera qcam test application as a base would provide most (if not all) of what you would need.

User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

Re: diy autofocus with /dev/video16 and stepper motor?

Thu Jun 25, 2020 8:06 am

@6by9, thanks for the detailed explanation.
naushir wrote:
Thu Jun 25, 2020 7:37 am
An alternative is to use the libcamera stack, where we are *just* about to merge a change that provides the focus FoM to the application. Using the libcamera qcam test application as a base would provide most (if not all) of what you would need.
That sounds perfect.
Does "*just* about to merge" mean waiting will get it say end of next week?
What is needed to get it then, execute "rpi-update" again?
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

naushir
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 453
Joined: Mon Apr 25, 2016 10:21 am

Re: diy autofocus with /dev/video16 and stepper motor?

Thu Jun 25, 2020 8:42 am

HermannSW wrote:
Thu Jun 25, 2020 8:06 am
@6by9, thanks for the detailed explanation.
naushir wrote:
Thu Jun 25, 2020 7:37 am
An alternative is to use the libcamera stack, where we are *just* about to merge a change that provides the focus FoM to the application. Using the libcamera qcam test application as a base would provide most (if not all) of what you would need.
That sounds perfect.
Does "*just* about to merge" mean waiting will get it say end of next week?
What is needed to get it then, execute "rpi-update" again?
The patches are under review right now, you can follow the progress in the libcamera-dev mailing list. There is still one patch that is a work in progress to push the FoM result back to the app though metadata, but everything else should be merged by the end of the week hopefully.

In order to try these changes, you will have to manually build the latest version of libcamera (instructions here). Given that this library is still very much in a state of development, it is not part of our standard image just yet.

User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

Re: diy autofocus with /dev/video16 and stepper motor?

Sat Jul 04, 2020 8:53 am

naushir wrote:
Thu Jun 25, 2020 8:42 am
The patches are under review right now, you can follow the progress in the libcamera-dev mailing list. There is still one patch that is a work in progress to push the FoM result back to the app though metadata, but everything else should be merged by the end of the week hopefully.

In order to try these changes, you will have to manually build the latest version of libcamera (instructions here). Given that this library is still very much in a state of development, it is not part of our standard image just yet.
Hi Naushir,

interesting discussion:
https://lists.libcamera.org/pipermail/l ... 10385.html

I did try on 6/29 when rpi/focus.h was available in libcamera repo
And I tried today.
The diff seems to indicate that work is still needed:

Code: Select all

pi@raspberrypi4B2:~/libcamera/src/ipa/raspberrypi/controller/rpi $ find -type f -exec diff {} ~/libcamera.old2/src/ipa/raspberrypi/controller/rpi/{} \;       
24a25
> 	void Read(boost::property_tree::ptree const &params) override;
25a27,28
> private:
> 	bool print_;
9,10d8
< #include "libcamera/internal/log.h"
< 
11a10
> #include "../logging.hpp"
15,17d13
< using namespace libcamera;
< 
< LOG_DEFINE_CATEGORY(RPiFocus)
30a27,31
> void Focus::Read(boost::property_tree::ptree const &params)
> {
> 	print_ = params.get<int>("print", 0);
> }
> 
39,42c40,43
< 
< 	LOG(RPiFocus, Debug)
< 		<< "Focus contrast measure: "
< 		<< (status.focus_measures[5] + status.focus_measures[6]) / 10;
---
> 	if (print_) {
> 		uint32_t value = (status.focus_measures[5] + status.focus_measures[6]) / 10;
> 		RPI_LOG("Focus contrast measure: " << value);
> 	}
pi@raspberrypi4B2:~/libcamera/src/ipa/raspberrypi/controller/rpi $ 

How should "print" be used?
"build/src/qcam/qcam -s print=1" results in error, "build/src/qcam/qcam print=1" does not change output.
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1116
Joined: Tue Jan 07, 2020 9:15 am

Re: diy autofocus with /dev/video16 and stepper motor?

Sat Jul 04, 2020 10:12 am

Hi

There's been an update to this patch just now, if you check out the latest version and rebuild you should run qcam like so:
LIBCAMERA_LOG_LEVELS=RPiFocus:DEBUG build/src/qcam/qcam
(the output of the focus information is now controlled by that LIBCAMERA_LOG_LEVELS environment variable).

User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

Re: diy autofocus with /dev/video16 and stepper motor?

Sat Jul 04, 2020 1:55 pm

therealdavidp wrote:
Sat Jul 04, 2020 10:12 am
There's been an update to this patch just now, if you check out the latest version and rebuild you should run qcam like so:
LIBCAMERA_LOG_LEVELS=RPiFocus:DEBUG build/src/qcam/qcam
Thanks, I did "git pull" and already was up to date.
(the output of the focus information is now controlled by that LIBCAMERA_LOG_LEVELS environment variable).
For easy access I made your command a script:

Code: Select all

pi@raspberrypi4B2:~/libcamera $ cat qcan_focus 
#!/bin/bash
LIBCAMERA_LOG_LEVELS=RPiFocus:DEBUG build/src/qcam/qcam
pi@raspberrypi4B2:~/libcamera $ 
Nice -- I cannot tell now whether it is better to see focus value on separate terminal window, or with rectangle in preview window as before. It would definitely help to optionally at least display the rectangle(rectangles) for knowing what area is used for focus value computation.

1024x600 7" DPMI display screenshot taken with raspi2png, converted to jpeg with "pngtopnm snapshot.png | pnmtojpeg > snapshot.png.jpg" for attachment size limitation:
snapshot.png.jpg
snapshot.png.jpg
snapshot.png.jpg (128.59 KiB) Viewed 6538 times

P.S:
As mentioned in initial posting of this thread, autofocus should be possible now, with access to focus values. In the animation I did move microscope one full-step down between each screenshot, or 8.8µm. Using qcam instead of "raspistill --focus -t 0" should allow for autofocus for the microscope at least:
Image


For CS or C mount lenses, a mechanism to control focus via a stepper motor needs to be created first:
Image
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

Re: diy autofocus with /dev/video16 and stepper motor?

Sat Jul 04, 2020 3:58 pm

From libcamera/include/linux/bcm2835-isp.h:

Code: Select all

...
/*
 * ISP statistics structures.
 *
 * The bcm2835_isp_stats structure is generated at the output of the
 * statistics node.  Note that this does not directly map onto the statistics
 * output of the ISP HW.  Instead, the MMAL firmware code maps the HW statistics
 * to the bcm2835_isp_stats structure.
 */
 ...
.#define FOCUS_REGIONS 12
...
So the structure gets magically filled from "statistics node" of GPU?

There are 12 focus regions, and the focus value logged is built from regions 5 and 6 only.
Do those two regions combined build the rectangle we know from "raspistill --focus"?
What are the coordinates and sizes of the 12 regions?
Can the focus regions be changed, or will it be possible later?

From libcamera/src/ipa/raspberrypi/controller/rpi/focus.cpp:

Code: Select all

void Focus::Process(StatisticsPtr &stats, Metadata *image_metadata)
{       FocusStatus status;
        unsigned int i;
        for (i = 0; i < FOCUS_REGIONS; i++)
                status.focus_measures[i] = stats->focus_stats[i].contrast_val[1][1] / 1000;
        status.num = i;
        image_metadata->Set("focus.status", status);

        LOG(RPiFocus, Debug)
                << "Focus contrast measure: "
                << (status.focus_measures[5] + status.focus_measures[6]) / 10;
}
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1116
Joined: Tue Jan 07, 2020 9:15 am

Re: diy autofocus with /dev/video16 and stepper motor?

Sat Jul 04, 2020 7:12 pm

Yes, there are 12 regions. They actually form an even 4x3 grid across the image which, I think it's fair to say, was not particularly intentional, that's just how it turned out for historical reasons. So it's not the same as what you get from the little window in the GPU firmware stack, it's rather larger (though still central-ish).
Can the focus regions be changed, or will it be possible later?
Indeed, I think we know we need to go there. Hold on to that thought...

User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

Re: diy autofocus with /dev/video16 and stepper motor?

Sat Jul 04, 2020 8:51 pm

Thanks, so currently focus is computed from squares 5 and 6, but that can easily be adapted to any subset of the 12 squares:
focus_rectangles.jpg
They actually form an even 4x3 grid across the image
focus_rectangles.jpg (12.77 KiB) Viewed 6442 times

P.S:
For autofocus I need to find out how to add stepper motor control code to qcam application, that runs in parallel with the capture and display task. Or alternatively, how to query current focus value from running qcam application from separate stepper motor control program (I would prefer that, so that qcam keeps qcam and just provides focus value)..
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1116
Joined: Tue Jan 07, 2020 9:15 am

Re: diy autofocus with /dev/video16 and stepper motor?

Sat Jul 04, 2020 10:54 pm

For autofocus I need to find out how to add stepper motor control code to qcam application, that runs in parallel with the capture and display task. Or alternatively, how to query current focus value from running qcam application from separate stepper motor control program (I would prefer that, so that qcam keeps qcam and just provides focus value)..
So this is of course why we're working on the open source camera stack running on the ARM - so that people can do exactly this sort of thing.

One of your suggestions was to have an "autofocus" program external to qcam which can simply receive focus measures, and qcam can then be left largely alone. There will certainly be ways of accomplishing this, however, in the libcamera world the hope is that AF becomes one of the "control algorithms" (as we call them) or IPAs ("Image Processing Algorithms", as libcamera calls them) baked right into the camera system. In this way, any libcamera application would be able to "request an autofocus cycle" and I think you end up with a more useful end result.

We provide a complete framework for people to write their own control algorithms, and your starting point would be chapter 4 of https://github.com/raspberrypi/document ... ra_1p0.pdf . But in a nutshell, you more-or-less want to copy focus.cpp and "make it do your own thing".

We are in principle very keen to help people do this kind of thing, and if we get results that will be genuinely useful to the community then I'd certainly like to see us incorporate them in the distribution.

User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

Re: diy autofocus with /dev/video16 and stepper motor?

Sun Jul 05, 2020 9:23 am

@therealdavidp
Thanks for the detailed information, and pointing to chapter 4.
I totally agree that AF (AutoFocus) finally belongs into qcam itself.
But it would need its own button, like the start/stop capture or save buttons.
And it maybe would need parameters, eg. the range of stepper to search for optimal focus.
The range will avoid microscope crashing into the object, or hopping out of rail on top.

So for prototype implementation I will do separation of concerns.
Although all (stepper motor control, status value determination, AF algorithm) can be done in qcam, for now:
  • stepper control will be done by Arduino Uno, Pi only writes z coordinates to /dev/ttyUSB0, and Uno moves there
  • qcam will give viewfinder window and calculate focus values
  • AF will be standalone program, starting qcam via pipe() and so having access to qcam focus value output
Stepper motor steps typically are done with single digit microsecond delays in between.
Demo qcam application requests 800x600 frames, libcamera determines 2028x1520 pbCC as best fit.
Not sure why real qcam frame is 800x480 only.
qcam application runs at 30fps, so only ever 34ms a new focus value is available.
Therefore hierarchical stepping might be needed to find best in focus position.
Eg. "go 1000 step up, then down in steps of 100" (8.8µm full steps or 4.4µm half-steps).
Then knowing "best" position search best-100..best+100 in steps of 10, ...

I just used microscope with lanzet on slanted NEMA17 stepper with qcam for the first time:
20200705_105650.10%.jpg
20200705_105650.10%.jpg
20200705_105650.10%.jpg (27.07 KiB) Viewed 6366 times
This is 1360x768 raspi2png screenshot, only lanzet pin point is in focus.
Not sure how the 2028x1520 pixels map into 800x480 frame yet, pixels in 2028x1520 frame are 0.42µm apart in xy direction:
snapshot.png.50%.jpg
snapshot.png.50%.jpg
snapshot.png.50%.jpg (30.82 KiB) Viewed 6366 times
For further AF work I will use micrometer with 100 divisions of 1mm, each 10µm apart.
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

Re: diy autofocus with /dev/video16 and stepper motor?

Sun Jul 05, 2020 10:09 am

This screenshot of 802x544 qcam window clarifies on the resolution with qcam application as is.
I measured 569 pixels for 600µm distance horizontally, so 600µm/569px = 1.05µm per pixel:
qcam.802x544.jpg
qcam.802x544.jpg
qcam.802x544.jpg (93.03 KiB) Viewed 6333 times
And it proves that qcam displaying 800x ̶4̶8̶0̶472 frame for requested 800x600 is a qcam display scale bug.
I measured 281 pixels for 300µm horizontally, but only ̶2̶4̶0̶221 pixels for the (300µm) long divisions of micrometer vertically.
̶2̶8̶1̶/̶2̶4̶0̶*̶4̶8̶0̶=̶6̶1̶3̶

Code: Select all

281/221*472=600.1

The same scale issue exists on 1024x600 DPMI display, as well on 1360x768 HDMI monitor.


P.S:
The 2028x1520 frame gets scaled to 800 horizontally.
I said that pixels are (sligtly less than) 0.42µm apart horizontally for 2028x1520 frames.
The (slightly more than) 1.05µm stated above between pixels horizontally in qcam window match that scaling:

Code: Select all

(600/569)/0.42*800=2009

P.P.S:
I did report bug:
"[libcamera-devel] qcam display scale bug"
https://lists.libcamera.org/pipermail/l ... 10497.html
Last edited by HermannSW on Sun Jul 05, 2020 9:21 pm, edited 2 times in total.
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

Re: diy autofocus with /dev/video16 and stepper motor?

Sun Jul 05, 2020 9:04 pm

As said above, for separation of concerns, this is minimal C program that forks qcam and reads its output in parent process. That way parent process can determine focus values reported by qcam and do the autofocus stepper motor stuff by writing to stdout, which is redirected to /dev/ttyUSB0 where Arduino Uno is connected to control the microscope z axis stepper motor.

I ran into problems when trying to use execvpe() passing the environment as well (X11 display issue).
Since running ./pip with environment variable setting in front works, code uses execvp() as "good enough".
Compiling by simply "gcc pip.c -o pip", run by:

Code: Select all

$ LIBCAMERA_LOG_LEVELS=RPiFocus:DEBUG ./pip
libEGL warning: DRI3: failed to query the version
libEGL warning: DRI2: failed to authenticate
...
seq: 000092 bytesused: 1440000 timestamp: 45770839316000 fps: 30.02
[12:42:50.915073194] [10417] DEBUG RPiFocus focus.cpp:40 Focus contrast measure: 85
seq: 000093 bytesused: 1440000 timestamp: 45770872637000 fps: 30.01
[12:42:50.948489537] [10417] DEBUG RPiFocus focus.cpp:40 Focus contrast measure: 85
...

minimal pip.c:

Code: Select all

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>

int main(int argc, char** argv)
{
  int pipefd[2];
  int childpid, n;
  char *args[3]={"build/src/qcam/qcam",NULL,NULL};
  char buf[1000];
  pipe(pipefd);
  if(childpid=fork()){
    //parent
    close(pipefd[1]);
    dup2(pipefd[0],STDIN_FILENO);
    while ( (n=read(STDIN_FILENO, buf, 1000)) != 0 )
    {
      write(STDOUT_FILENO, buf, n); 
    }
  }else{  
    //child
    //write
    close(pipefd[0]);
    dup2(pipefd[1],STDOUT_FILENO);
    execvp("build/src/qcam/qcam", args);
  }
  return 0;
}
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

User avatar
Gavinmc42
Posts: 8192
Joined: Wed Aug 28, 2013 3:31 am

Re: diy autofocus with /dev/video16 and stepper motor?

Mon Jul 06, 2020 2:25 am

If you keep the lighting controlled and autowhite balance etc off then the maximum jpeg file size is when the lens is at the focus point.
Made a jig to do this for a security camera production line more than a decade ago.
Very sensitive to stray light changes so a sealed box helps.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

DarkElvenAngel
Posts: 2952
Joined: Tue Mar 20, 2018 9:53 pm

Re: diy autofocus with /dev/video16 and stepper motor?

Mon Jul 06, 2020 3:20 am

This is very interesting,

I would love to adapt this to my telescope build.

Following.

User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

Re: diy autofocus with /dev/video16 and stepper motor?

Mon Jul 06, 2020 10:04 am

Gavinmc42 wrote:
Mon Jul 06, 2020 2:25 am
If you keep the lighting controlled and autowhite balance etc off then the maximum jpeg file size is when the lens is at the focus point.
Nice idea, but saving as jpeg would be needed. Also it would be a global "focus" value, and not like the FoM values of raspistill (small center rectangle) or qcam (center two squares from 4x3 squares covering the frame) as shown in previous posting.

DarkElvenAngel wrote:
Mon Jul 06, 2020 3:20 am
I would love to adapt this to my telescope build.
I found a stepper with belt solution for telescope, so that could be the way to got for you:
https://www.cloudynights.com/topic/6065 ... try8366267
Image

HermannSW wrote:
Sat Jul 04, 2020 1:55 pm
For CS or C mount lenses, a mechanism to control focus via a stepper motor needs to be created first:
A belt around NEAR/FAR ring of 35mm lens and stepper motor coupler could work for 35mm lens as well:
20200706_114857.10%.jpg
20200706_114857.10%.jpg
20200706_114857.10%.jpg (59.19 KiB) Viewed 6157 times
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

tombsar
Posts: 46
Joined: Wed May 13, 2020 12:40 am

Re: diy autofocus with /dev/video16 and stepper motor?

Mon Jul 06, 2020 10:58 am

Edit: This relates to the idea of attaching a motor to the focus ring of the lens, not the mechanism that moves the whole camera up and down.

Do you plan to use limit switches? I'm worried about the motor pushing the mechanical limits of the focus ring and breaking something (they are surprisingly strong). Perhaps a slip-clutch to limit the maximum torque it can apply.

User avatar
Gavinmc42
Posts: 8192
Joined: Wed Aug 28, 2013 3:31 am

Re: diy autofocus with /dev/video16 and stepper motor?

Mon Jul 06, 2020 11:36 am

I have used toothed belts inside out on flat pulleys.
Spring loaded tension pulley.
But these were on ball bearing shafts.

Perhaps some sort of torque control on the stepper.
Good stepper motor drivers have adjustable current limits.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

tombsar
Posts: 46
Joined: Wed May 13, 2020 12:40 am

Re: diy autofocus with /dev/video16 and stepper motor?

Mon Jul 06, 2020 1:46 pm

I don't think current control is a viable option with these particular steppers, unfortunately, as they have an internal 64:1 gear ratio. It might still be possible, but would require you to replace the drivers that come with them.

I'm toying with the idea of a 3D printed torque-limiting ring to attach to the focus ring. It's fiddly though...

User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

Re: diy autofocus with /dev/video16 and stepper motor?

Mon Jul 06, 2020 1:53 pm

I was wrong last night, I will need fork() as well, but the basic task of extracting focus values is done by qcam started via popen():

Code: Select all

pi@raspberrypi4B2:~/libcamera $ LIBCAMERA_LOG_LEVELS=RPiFocus:DEBUG ./pop
7
7
7
7
8
23
23
23
...

This is the minimal code, since all output of qcam goes to stderr, redirection has to be specified in popen():

Code: Select all

rypi4B2:~/libcamera $ cat pop.c 
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char** argv)
{
  char buf[1000], *q;
  FILE *qcam = popen("build/src/qcam/qcam 2>&1", "r");
  fgets(buf, 1000, qcam);
  while (!feof(qcam))
  {
    if ( (q=strstr(buf, "Focus contrast measure:")) != 0)
    {
      printf("%d\n", atoi(q+23)); 
    }
    fgets(buf, 1000, qcam);
  }

  return 0;
}
pi@raspberrypi4B2:~/libcamera $ 

Autofucus code needs to know where stepper is, so stepper control will not be done by Arduino as stated before.
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

DarkElvenAngel
Posts: 2952
Joined: Tue Mar 20, 2018 9:53 pm

Re: diy autofocus with /dev/video16 and stepper motor?

Mon Jul 06, 2020 2:27 pm

HermannSW wrote:
Mon Jul 06, 2020 10:04 am
DarkElvenAngel wrote:
Mon Jul 06, 2020 3:20 am
I would love to adapt this to my telescope build.
I found a stepper with belt solution for telescope, so that could be the way to got for you:
https://www.cloudynights.com/topic/6065 ... try8366267
Image
Looks promising thanks!

naushir
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 453
Joined: Mon Apr 25, 2016 10:21 am

Re: diy autofocus with /dev/video16 and stepper motor?

Mon Jul 06, 2020 4:55 pm

The libcamera commit to export FoM to the application via metadata has been merged today, see here. You should now be able to extract the metadata for the FoM value from within qcam (or any application).

Currently the FoM is generated from the average of the middle two regions. It would be useful to get community feedback if this is useful or not.

User avatar
HermannSW
Posts: 6327
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany

Re: diy autofocus with /dev/video16 and stepper motor?

Mon Jul 06, 2020 5:12 pm

@naushir
Thanks for that, will finally make use of that after AF prototype work.

AF work is not negatively affected, but qcam cannot be used for any application that requires correct scaling on display (like my microscope app) without the display scale bug being fixed:
https://lists.libcamera.org/pipermail/l ... 10497.html


Now I have good basis to start AF work with, calling to qcam via popen(), and using pthread library.
One thread calls line_reader(), which starts qcam and then reads line by line.
If a line contains focus data, then tupl.focus and tupl.last get updated (last is time since AF start in milliseconds).
The other thread calls AF() and waits 8 seconds for qcam to start, and then outputs 10 times focus value and time difference in milliseconds to when that focus value was determined by qcam, with 1 second sleep in between.
After qcam has started, with 30fps framerate, maximal difference is 33ms:

Code: Select all

pi@raspberrypi4B2:~/libcamera $ LIBCAMERA_LOG_LEVELS=RPiFocus:DEBUG ./AF0
qcam: no process found
73(1)
76(903)
76(1911)
76(2911)
76(24)
75(24)
75(25)
76(26)
76(27)
76(27)
pi@raspberrypi4B2:~/libcamera $ 

The initial message comes from system() executing "killall qcam" to make sure no running qcam blocks AF:

Code: Select all

pi@raspberrypi4B2:~/libcamera $ cat AF0.c 
#include<stdio.h>
#include<string.h>
#include<pthread.h>
#include<stdlib.h>
#include<unistd.h>
#include<assert.h>

pthread_t tid[2];
struct tupl_t { int focus; unsigned long last; } tupl;
int stop = 0;

static struct timespec time0 = { .tv_sec=0, .tv_nsec=0 };

unsigned long millis() 
{
  struct timespec time1;
  clock_gettime(CLOCK_REALTIME, &time1);

  if (time0.tv_sec==0 && time0.tv_nsec==0)
  {
    time0.tv_sec =time1.tv_sec;
    time0.tv_nsec=time1.tv_nsec;
  }

  return 1000*(time1.tv_sec-(long long)time0.tv_sec) + (time1.tv_nsec-(long)time0.tv_nsec)/1000000;  
}

void line_reader()
{
  char buf[1000], *q;
  FILE *qcam = popen("build/src/qcam/qcam 2>&1", "r");
  assert(qcam);
  fgets(buf, 1000, qcam);
  while (!feof(qcam) && !stop)
  {
    if ( (q=strstr(buf, "Focus contrast measure:")) != 0)
    {
      tupl.focus = atoi(q+23); 
      tupl.last = millis();
    }
    fgets(buf, 1000, qcam);
  }
  system("killall qcam");
  pclose(qcam);
}

void AF()
{
  sleep(8);

  for(int i=0; i<10; ++i)
  {
    printf("%d(%lu)\n", tupl.focus, millis()-tupl.last);
    sleep(1);
  }

  stop = 1;
}

void* doSomeThing(void *arg)
{
  pthread_t id = pthread_self();

       if (pthread_equal(id,tid[0]))  line_reader();
  else if (pthread_equal(id,tid[1]))  AF();

  return NULL;
}

int main(void)
{
  int i = 0;  
  int err;
  int *ptr[2];

  millis();
  system("killall qcam");

  assert(0 == pthread_create(tid+0, NULL, &doSomeThing, NULL));
  assert(0 == pthread_create(tid+1, NULL, &doSomeThing, NULL));

  pthread_join(tid[1], NULL);
  pthread_join(tid[0], NULL);

  return 0;
}
pi@raspberrypi4B2:~/libcamera $

I have no idea why "killall qcam" is required at end of line_reader() for pclose(qcam) to finish. That is not nice, but works.
Compile with:

Code: Select all

gcc AF0.c -o AF0 -lpthread

Now I will integrate simple stepper motor control code for the Pi, since there is no Arduino anymore used for that.
https://github.com/Hermann-SW/RSA_numbers_factored
https://stamm-wilbrandt.de/GS [304+402+536fps]
https://hermann-sw.github.io/planar_graph_playground
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de

Return to “Camera board”