Low-level (Linux framebuffer) graphics programming tutorial

[-rst-]

Not sure how much there is need for this kind of stuff, but just in case...

The Raspberry Pi (RPi) comes built with hardware support - and supporting software programming libraries - for all the current state of the art standardised graphics goodies: OpenGL ES, OpenVG, EGL etc. and considering the performance gains of using the VideoCore GPU over the ARM CPU, it definitely makes sense to utilise these libraries to their full extent.

However, one of the main ideologies of the Raspberry Pi Foundation - the people who conceived the crafty little appliance we now know as RPi - was to introduce new generations to 'what goes behind the scenes' of fancy applications and user-interfaces. In my opinion, this goes as well for the 'fancy' graphics libraries and technologies. Therefore I would like to think it makes sense to introduce also the lower level interfaces for programming graphics on the RPi (most principles and some of the code I will introduce apply to other systems as well).

http://raspberrycompote.blogspot.com/20 ... _9509.html

Basic command-line and file editing skills expected - some understanding of C programming would not hurt...

Constructive comments and critique welcome.

[r00x]

Thanks for this - I've been looking for days for concise info on the framebuffer and only found disjointed discussion by people who already know what they're on about.

What other information on the framebuffer can we siphon from you? Do you know how one might make a second framebuffer (fb1) for a second display?

I'd like to get a few different colour TFTs of my own choosing running via SPI/GPIO (one at a time, not all at once, haha) and it would be nice to pipe videos, images etc to them, and optionally have it run as the 'primary' display (so, pointing it to fb0 rather than a second fb1 I suppose). There is precious little concise information on how to go about this and I am getting lost.

[-rst-]

Thanks for this - I've been looking for days for concise info on the framebuffer and only found disjointed discussion by people who already know what they're on about.

What other information on the framebuffer can we siphon from you? Do you know how one might make a second framebuffer (fb1) for a second display?

...

Good to hear at least someone reading it. Unfortunately I have no experience using multiple framebuffers. I can take a rummage through my links and references, just in case...

[DexOS]

Good to hear at least someone reading it.

I sure there are many coders that find it interesting, i been working on something simular, but coded it in assembly.
I hope to run it on a stripped down linux, so it will boot fast.
Its very interesting to see how the desktop is drawn.
Example: when something is written to the terminal after filling the full screen, it only up dates a small block, also how you can redraw the desktop with the mouse cursor.
Anyway, great works.

[pinbot]

Truly -rst-, *THANK YOU* for this! - ESPECIALLY SOME DEMO CODE!

Right off the bat, what are your thoughts on SDL 1.2 for direct interactions w/ fbdev on Rpi? There are many features that stand out to me for trying to approach Rpi w/ SDL, starting out with the obligatory display:

• fbdev
• sound
• network

The Wikipedia Entry for SDL goes on to list out more, 'including file access, event handling, timing, theading, and more'

From what I understand w/ SDL 1.2, it doesn't have the hardware support for Rpi's GPU because it's bindings are to OpenGL, not OpenGL ES? Where as, OpenGL ES is what is behind Rapsberry Pi along w/ OpenMax and 1-2 others?

Meanwhile, SDL 2.0 looks promising because it's slated to have more support (OpenGL ES?) based upon my understanding?

I've been trying to spend what free time I can reading the following for trying to understand what I need, in order to accomplish varying goals centered around Rpi:

• Programming Linux Games
• Linux from Scratch
• GNU/Linux Application Programming

and after finding this forum thread tonight, finally a light bulb or two is clicking on THANK YOU! THANK YOU! THANK YOU!

Where as Qt5 is handing my ass to me on Rpi, and I have't been this frustrated in a long time (I've lost track of how many days I've spent just trying to get fundamental basics accomplished for this,) in part because I can barely even get it to compile, let alone something so simple as displaying "Hello World" on the screen (deep breaths into a paper sack)

I've truly considered posting up a bounty on the forum for verbose, articulate, step-by-step instructions so that mere mortals with <= 140 IQ can jump into the razzle dazzle side of Rpi instead of just NASA engineers and anyone else hovering at 180-200 IQ, if not higher

With that said, what are your thoughts on the ability of merging the two together? E.g. if SDL is the /appropriate/ choice for covering many bases (fbdev, peripheral i/o, sound, network, etc) is there a way to marry the two together where a nice menu system can be built on top of Qt5 (including perhaps picture in picture for example,) while as the meat & potatoes are just raw fbdev?

I'm *absolutely clueless* when it comes to this, and coming from a decade+ in LAMP stack, where as my first taste of EE was with Arduino boards, then multiple processing capability w/ Propeller, and Rpi feels like the missing piece of the puzzle for trying to chase down what I dream about on a nightly basis (including some fbdev work in nursing homes, and as time progresses, hopefully children's hospitals as well)

Best regards, and thank you once again

[-rst-]

Glad to be of at least some help.

Hmm... at the brink of the depths to very elementary life choices I am not exactly an expert in all of the libraries mentioned here...

Yes, SDL at the moment is not the best choice for performance on RPi - it lacks (afaik) all hw acceleration (due to the OpenGL vs OpenGL ES gap, as you mention) and even such basic stuff as hw page flipping (with vsync). This is partly due to lack of standard support in Linux (and an oversight on 'nearly standard' stuff in SDL) - partly due to RPi fb driver not implementing all features. However, SDL is so widely used and somewhat portable, that I'd say it would still be a good contender and the new version of SDL looks promising... There is a slightly modified port of SDL that provides vsync by utilising the RPi hw exposed through the dispmanx library: http://www.raspberrypi.org/phpBB3/viewt ... 78&t=25146

Since you have got Qt5 compiled yourself, you might not want to see these ready compiled packages: http://twolife.be/raspbian/pool/main/qt5/ Qt5 is still work in progress, so might not provide a stable enough platform at the moment - it looks very promising, and if it will fulfill all the promises...

One option - unfortunately not so easily portable (some advanced 'configure' scripting though...) - would be to use SDL for input+sound+etc and the RPi provided libraries EGL/dispmanx for display/window management and then straight on OpenGL ES etc.

Unfortunately - for anything advanced - the 'pure framebuffer' option is not going to give acceptable performance due to the somewhat weak CPU on RPi and the lacking functionality in the fb driver (have written about some of this on this forum http://www.raspberrypi.org/phpBB3/viewt ... 67&t=19073 , http://www.raspberrypi.org/phpBB3/viewt ... 72&t=11682 and http://www.raspberrypi.org/phpBB3/viewt ... 67&t=23869 - was going to get into these in the blog as well...).

Not much of a help is it?

[-rst-]

Just in case anyone is interested...

Some new posts including an explanation of the most typical display modes (bit depths) http://raspberrycompote.blogspot.ie/201 ... art_8.html (applicable to dispmanx etc, not just framebuffer...) - also full source code examples available in github now (see blog sidebar 'Code Repository').

[10robinho]

-rst- that is really great. Thanks a lot for your effort! This tutorial helps a lot to fill gaps in knowledge.

[-rst-]

Some more playing around with color depths http://raspberrycompote.blogspot.ie/201 ... -part.html and resolutions http://raspberrycompote.blogspot.ie/201 ... art_3.html


...shameless self-promotion yes - but hey, some have found the stuff useful ...and I am not even signed-up for 'Earnings' with this blog (no ads)

[DexOS]

Some more playing around with color depths http://raspberrycompote.blogspot.ie/201 ... -part.html and resolutions http://raspberrycompote.blogspot.ie/201 ... art_3.html


...shameless self-promotion yes - but hey, some have found the stuff useful ...and I am not even signed-up for 'Earnings' with this blog (no ads)

What FPS are you getting in different modes ?, i am getting 100fps at 800x600 16bpp, using double buffering in bare metal, and i think about the same under linux using asm (not tested fps under linux, just seems about the same).

[-rst-]

What FPS are you getting in different modes ?, i am getting 100fps at 800x600 16bpp, using double buffering in bare metal, and i think about the same under linux using asm (not tested fps under linux, just seems about the same).

Obviously depends a lot of how complicated 'animation' going on: simple 'bouncing sprites' in 960x540 8bpp seem to run around 100fps - 'the fire' runs only around 20fps in 448x256 8bpp (for every frame: 4 reads per every pixel with subsampling, 2 writes per every pixel + 1 write for each pixel on the bottom row)... All (so far) programmed in plain C - mostly not optimised.

[DexOS]

You could use DMA

[-rst-]

Sure, and could code shaders and... For now I just like to stick to the Linux frame buffer to keep things simple

[DexOS]

Sure, and could code shaders and... For now I just like to stick to the Linux frame buffer to keep things simple

I am the same, all my stuff is KISS.
Here's a fb menu sys i working on for the pi http://www.dex-os.com/MiniDos/fb.zip

[HappyPiUser]

This thread is excellent. Thank you to the OP for all of your hard work!

And to those who were asking about SDL on the Pi, you might want to take a peek at

https://github.com/vanfanel/SDL12-kms-dispmanx#readme

That's is a Raspberry Pi-optimized backend for SDL 1.2, using the "dispmanx" approach.

It's been useful for some folks who are porting SDL games and emulators.

[GrandAdmiral]

I'll add my thanks for the tutorial! I was wondering if you (or somebody else) could help with a question. Is it possible to disable updates to the framebuffer from outside sources (e.g. turn off the command prompt if you're just displaying a terminal or turn off the GUI/mouse pointer if you've stated LXDE) prior to manually drawing into the framebuffer? When you are done, is it then possible to cause the framebuffer to get redrawn from the previous sources?

I think I can simulate this behavior when the terminal is displayed by disabling the prompt, drawing my test pattern(s), then blank the screen and re-enabling the prompt. I'm not sure how to handle the case when LXDE is running. Maybe this is a strange request - I'm just looking for a way to display a test pattern and then revert to the previous display. Finally, let me know if I should post this question on it's own. I figured the tutorial got me most of the way to where I wanted to go, I would start here.

[DexOS]

You could copy the screen to a buffer and restore it after you have finished.

[GrandAdmiral]

You could copy the screen to a buffer and restore it after you have finished.

Yeah, I thought of that. While I don't see a big problem with this when the command prompt is displayed, it doesn't stop the mouse cursor from being moved and causing parts of the Frame Buffer to be redrawn when LXDE is running. I'd also like to avoid the case where something changes on the GUI and coming out of the test pattern the user sees the old view and there is an inconsistency between the two.

[DexOS]

You could copy the screen to a buffer and restore it after you have finished.

Yeah, I thought of that. While I don't see a big problem with this when the command prompt is displayed, it doesn't stop the mouse cursor from being moved and causing parts of the Frame Buffer to be redrawn when LXDE is running. I'd also like to avoid the case where something changes on the GUI and coming out of the test pattern the user sees the old view and there is an inconsistency between the two.

Try the fb.zip i posted above and see if that stops the redraw, until you exit it.

[GrandAdmiral]

No, the program in the fb.zip program does not work. As soon as you move the mouse, it corrupts the image. To the point I wasn't able to cleanly exit the program and had to reboot the system.

[mrhobbeys]

I really enjoyed reading this and will be giving it a try tonight.

Thank you very much!

[JamesLehman]

This is perhaps a little OT, but I can tell you from personal experience that SDL kind-of works on RPi!

Take a look at the LaserBoy code!
http://www.raspberrypi.org/phpBB3/viewt ... 38&t=40587

It uses SDL to open a window in X and allows me to copy a bitmap into that area of the screen. It also reads the keyboard.

LaserBoy also uses a very limited set of functionality from Boost C++ for some file system information.

On a regular PC, LaserBoy works in the console. SDL knows how to hook up to the frame buffer and it works great. I cannot get these results on the RPi. I get a black screen.

James.

[-rst-]

Restart of the blog...

And a new post in 'Low-level Graphics on Raspberry Pi'

[AndyD]

Welcome back!

[cleverca22]

Glad to be of at least some help.

Hmm... at the brink of the depths to very elementary life choices I am not exactly an expert in all of the libraries mentioned here...

Yes, SDL at the moment is not the best choice for performance on RPi - it lacks (afaik) all hw acceleration (due to the OpenGL vs OpenGL ES gap, as you mention) and even such basic stuff as hw page flipping (with vsync). This is partly due to lack of standard support in Linux (and an oversight on 'nearly standard' stuff in SDL) - partly due to RPi fb driver not implementing all features. However, SDL is so widely used and somewhat portable, that I'd say it would still be a good contender and the new version of SDL looks promising... There is a slightly modified port of SDL that provides vsync by utilising the RPi hw exposed through the dispmanx library: http://www.raspberrypi.org/phpBB3/viewt ... 78&t=25146

i happen to be working on a port of SDL with proper hardware 3d support, so far all it can do is build a vertex list for a frame, but once i get a texture shader working, i can start to render a frame