virtual line following

[Tonyapi]

Hello,

I've seen line following robot using OpenCV.
Like this: https://www.instructables.com/OpenCV-Ba ... ing-Robot/

But here, its following an actual line on the floor. Is there a way to have it follow a virtual line? I was planning on drawing lines using OpenCV.

[PiGraham]

https://www.bing.com/search?q=ROS+virtu ... cc=0&ghpl=

https://www.bing.com/search?q=robot+vir ... cc=0&ghpl=


https://www.robotc.net/getting-started-lego-rvw.html

https://github.com/Unity-Technologies/U ... botics-Hub

[PiGraham]

Hello,

I've seen line following robot using OpenCV.
Like this: https://www.instructables.com/OpenCV-Ba ... ing-Robot/

But here, its following an actual line on the floor. Is there a way to have it follow a virtual line? I was planning on drawing lines using OpenCV.

Are you thinking of superimposing a virtual line on real video? i.e. augmented reality rathe then virtual environment?

That's probably harder than doing it in VR with a virtual camera since you would have to accurately map the real environment into a virtual one to 'draw the line' then map the line onto the live video.

What are you trying to achieve? Testing line following algorithms? Controlling a real robot by drawing a virtual line?

[Tonyapi]

I like to put a camera on the robot, and superimpose lines to it to follow.
I'm trying to have my robot follow a virtual line. Eventually I like to have this may be rove around my backyard.

[PiGraham]

I like to put a camera on the robot, and superimpose lines to it to follow.
I'm trying to have my robot follow a virtual line. Eventually I like to have this may be rove around my backyard.

Superimpose lines on what?
To be meaningful it will have to be registered in a virtual model of what the camera sees.

[Tonyapi]

I'll have a camera mounted on the robot, so its a live camera feed. I like to superimpose lines on the live feed so the robot can follow the lines.
I've used OpenCV before but not for anything like this.

The tutorials shows that the robot can follow an actual line drawn on the floor with a live camera, so my theory is why can't it follow a virtual line?

I know most people use a GPS receiver for this, but I'm trying to see if its possible to do with just opencv.

[MiscBits]

Only way I can think is to add an overlay of the line to the camera feed before it gets to the recognition stage...

Possibly something like the rectangle (OpenCV Docs) function but how you would change it in real time is beyond my brain today!

[Tonyapi]

Thanks for all the replies and suggestions, more are welcome.
If I come up with something, I'll definitely post it here.

[PiGraham]

I'll have a camera mounted on the robot, so its a live camera feed. I like to superimpose lines on the live feed so the robot can follow the lines.
I've used OpenCV before but not for anything like this.

The tutorials shows that the robot can follow an actual line drawn on the floor with a live camera, so my theory is why can't it follow a virtual line?

I know most people use a GPS receiver for this, but I'm trying to see if its possible to do with just opencv.

If you want the robot to follow a a line in a camera feed the line must relate to the scene the camera sees. Otherwise you can draw lines on blank backgrounds unrelated to robot motion, but there's not much point in that.

So the key is to know where to paint the line in the real world image so that it relates to the robot position in the real world.

That is Augmented Reality.
Analyse the camera secene to generate a virtual 3D representation of the world around the robot.
Draw the virtual lines ON THE FLOOR of the virtual scene is the position they would appear if they were actually on the real floor.

You must work in the virtual world space, not the camera image space.

ROS seems like a good starting point for something like this.

I think it can generate virtual maps from sensor data (SLAM)

Once you have the 3D model and a virtual camera to match your real camera you can place lines on the virtual floor, map those to the virtual camera and superimpose the virtual camera image of the lines onto the real world camera image of the environment.

I think this goes way beyond this forum (Python) or Open CV..

I think I would start with ROS and get a virtual robot moving in a virtual world. You may even find examples of line following robots.
Develop your line slowing algorithms in the virtual world.

That should work should map into the real world without AR.

If you really want a real robot following virtual lines you then have a starting point.

[Tonyapi]

Thank you, this is unchartered territory for me, and sounds like ROS is what I need to look into.

I'm not sure if this will work, but the whole point is for the robot to rover my backyard. So may be I can get a satellite image of my backyard from Google Maps, then manually add lines using MS Paint, and have the robot follow that. Then of course I need to think about making sure the rover speed will match the scale of the image etc. I can still add a camera to the rover for things like obstacle avoidance.

Just thinking aloud.

[memjr]

I'm probably missing something there.

If you overlay the line on the video, then you know where the line is supposed to be. If you know where the line is supposed to be, why not just tell the robot to move there to begin with? There is no need for a line.

[Tonyapi]

That is true, I could also tell the robot to go forward for x seconds, then turn left etc, but I'm trying to avoid that.

[PiGraham]

Thank you, this is unchartered territory for me, and sounds like ROS is what I need to look into.

I'm not sure if this will work, but the whole point is for the robot to rover my backyard. So may be I can get a satellite image of my backyard from Google Maps, then manually add lines using MS Paint, and have the robot follow that. Then of course I need to think about making sure the rover speed will match the scale of the image etc. I can still add a camera to the rover for things like obstacle avoidance.

Just thinking aloud.

I think it would be easier to either do this is the real world and put down tape to mark the path in your yard, or do it all virtually, in ROS, and have a virtual robot following a virtual line.

Line following is relatively simple and you can start on your desk. AR or VR adds a lot of complexity to start with, although it can benefit you in the long run.

Why don't you want to do this with lines in your yard?

[memjr]

Why don't you want to do this with lines in your yard?

And if you can do the actual line on the ground, then no Pi is needed, only a pair of transistors will get your bot following a line on the ground.

[Tonyapi]

I'll look into ROS, since I don't want to draw lines in my backyard or put any electrical cables. Thanks

[PiGraham]

This could be a good starting point.

https://www.virtualroboticstoolkit.com/documentation

https://www.youtube.com/watch?v=b-a_kle ... W_&index=4