Simon's accelerated X development thread

[teh_orph]

When I say I new to this I don't think I could be any more shiny. I've used linux for a few weeks in 15 years but I now have my debian VM running (should have gone with ubuntu) and tk is compiling ready for debugging. Even if I don't succeed I'm learning.

Good work... Can you now debug through a Tk application's function calls?

Hi, have the open-sourced Broadcom drivers proved useful for this project?

No, that code is not useful for this driver project. It's useful for allowing EGL code to run in an X window without any extra code, though.
Email with various Broadcom people and that datasheet released many months ago have been invaluable.

Any progress on this project?

Lols, all my update posts are in this thread! Scroll back! (to be fair though the last page of conversation has been about application inefficiency)

I'll likely be releasing something some and with THIS KERNEL
http://www.raspberrypi.org/phpBB3/viewt ... 65#p217665

it uses a new memory scheme which I'm sure Dom would like as many people as possible to test. So if you have time try using it for a while (and I'll require you to)

[jamesh]

Hi, have the open-sourced Broadcom drivers proved useful for this project?

I would love to see a hardware-accelerated Raspbian LXDE environment one day with faster graphics rendering, it would make the Raspberry Pi much more usable overall. Any progress on this project?

Er, that's what this thread is all about. LXDE runs over X, so accelerate X and you accelerate LXDE.

[charliedurrant]

When I say I new to this I don't think I could be any more shiny. I've used linux for a few weeks in 15 years but I now have my debian VM running (should have gone with ubuntu) and tk is compiling ready for debugging. Even if I don't succeed I'm learning.

Good work... Can you now debug through a Tk application's function calls?

We'll ummmm,......no

Everything looks lovely:

- I've built the libtk8.5.so file and copied it to /usr/lib/
- There was a libtk8.5.so.0 file in /usr/lib
- Ran ldd /usr/lib/python2.6/lib-dynload/_tkinter.so to show it dependencies - depends on /usr/lib/libtk8.5.so.0
- Renamed libtk8.5.so.0 to libtk8.5.so.0.old
- Renamed libtk8.5.so to libtk8.5.so.0
- Ran ldconfig -v to update the shared cache config
- Started idle


and....

Python crashes out (window never appears). There's clearly a (so/dll) interface issue and at this point I'm not sure how to trace the problem. It's good to be learning something though!

Charlie

[teh_orph]

Do the two copies of libtk8.5.so have different lib dependencies themselves? eg maybe the new library has an unresolved linkage.
Perhaps if you run with gdb attached you'll see what the trouble is. (either way, keep going!)

[charliedurrant]

One tiny difference, I'd missed out the threading configure switch to include threading.

After recompiling and re-testing to see if idle loads (it didn't) the ldd reports are the same:

ldd output for new libtk8.5.so.0
--------------------------------
linux-gate.so.1 => (0xb776e000)
libpthread.so.0 => /lib/i686/cmov/libpthread.so.0 (0xb7605000)
libXft.so.2 => /usr/lib/libXft.so.2 (0xb75f2000)
libX11.so.6 => /usr/lib/libX11.so.6 (0xb74d4000)
libXss.so.1 => /usr/lib/libXss.so.1 (0xb7421000)
libXext.so.6 => /usr/lib/libXext.so.6 (0xb7411000)
libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0xb745c000)
libz.so.1 => /usr/lib/libz.so.1 (0xb7273000)
libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0xb742d000)
libXrender.so.1 => /usr/lib/libXrender.so.1 (0xb7424000)
libdl.so.2 => /lib/i686/cmov/libdl.so.2 (0xb740d000)
libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb73e7000)
libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb72a0000)
/lib/ld-linux.so.2 (0xb776f000)
libxcb.so.1 => /usr/lib/libxcb.so.1 (0xb7287000)
libexpat.so.1 => /usr/lib/libexpat.so.1 (0xb724c000)
libXau.so.6 => /usr/lib/libXau.so.6 (0xb7249000)
libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0xb7244000)

ldd output for old libtk8.5.so.0
----------------------------------
linux-gate.so.1 => (0xb77d4000)
libpthread.so.0 => /lib/i686/cmov/libpthread.so.0 (0xb7691000)
libXft.so.2 => /usr/lib/libXft.so.2 (0xb754e000)
libX11.so.6 => /usr/lib/libX11.so.6 (0xb7574000)
libXss.so.1 => /usr/lib/libXss.so.1 (0xb7570000)
libXext.so.6 => /usr/lib/libXext.so.6 (0xb7561000)
libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0xb74d6000)
libz.so.1 => /usr/lib/libz.so.1 (0xb74c2000)
libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0xb7492000)
libXrender.so.1 => /usr/lib/libXrender.so.1 (0xb7489000)
libdl.so.2 => /lib/i686/cmov/libdl.so.2 (0xb7485000)
libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb745f000)
libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7318000)
/lib/ld-linux.so.2 (0xb77d5000)
libxcb.so.1 => /usr/lib/libxcb.so.1 (0xb72ff000)
libexpat.so.1 => /usr/lib/libexpat.so.1 (0xb72d8000)
libXau.so.6 => /usr/lib/libXau.so.6 (0xb72d5000)
libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0xb72d0000)


Bar that the order of the entries reported by ldd was different and all the base addresses of the dependencies are different.

Clocking off now - trust the driver is going well. Writing something that is between X and the display hardware must be an absolute pig to write - my problems pale into insignificance!

Charlie

[charliedurrant]

Oh, and I'll try gdb tomorrow - thanks for the hint.

[teh_orph]

Bar that the order of the entries reported by ldd was different and all the base addresses of the dependencies are different.

Sorry I've been racking my brains as to what could be wrong and have come up blank. I'm gonna go with compiler flag differences.
Although isn't there scope to just download the debug package of this library in which ever distro you're using?

[charliedurrant]

No worries, I'll get to the bottom of it - it's the best way to learn.

[charliedurrant]

Hooray! For any one who wishes to compile libtk you need to:

1) Get the source from the debian repositry - I mistakenly got the source from the tcl/tk site

Code: Select all

apt-get source tk8.5-dev

2) Shared objects have identifiers built into the in the elf header called SONAME. This is is very important! There are many articles on how linux determines which shared objects to load as you can have many versions, but the important thing is that if you create another version you must have the the same SONAME as the version being loaded by your application. On my debian distro idle loads

Code: Select all

/usr/lib/libtk8.5.so.0  

this means version 8.5 sub major version 0 (the 0 on the end). If you run

Code: Select all

readelf -d /usr/lib/libtk8.5.so.0 | grep SONAME

to get the SONAME it outputs 'libtk8.5.so.0'

When you build libtk8.5 from the debian source it doesn't have an SONAME. So you have to pass a compile option to configure:

Code: Select all

CFLAGS="-Wl,-soname,libtk8.5.so.0" ./configure ...... etc etc

'make install' installs the so to /usr/local/lib, in my debian distro this is included in the search path for shared objects (it isn't on all distros as far as I know). This new so is ultimately the same as the one in /usr/lib/ so we have to tell the system it is a later version:

Code: Select all

mv /usr/local/lib/libtk8.5.so /usr/local/lib/libtk8.5.so.0.1

The 0.1 is the important part as it tells the sytem that this so is compatible with libtk8.5.so.0 (major sub version is still 0) but it now has a minor version of 1.

Then run

Code: Select all

[code]ldconfig -v | grep libtk

[/code]

To update the library cache list and it will show:

Code: Select all

libtk8.5.so.0 -> libtk8.5.so.0.1  (this is /usr/local/lib)
libtk8.5.so.0 -> libtk8.5.so.0 (this is /usr/lib)
libtk8.4.so.0 -> libtk8.4.so.0

And that is it - idle will now load the new library! Now to actually make some code changes (if required). This post was not too relevant to accelerated X but it might be useful to someone.

Charlie

[teh_orph]

So knowing this SONAME stuff, what was wrong with the code downloaded from the tk site?

[charliedurrant]

So knowing this SONAME stuff, what was wrong with the code downloaded from the tk site?

I'm not entirely sure without doing a diff on all files (another thing to learn).

However, with further investigation, when you download the source from debian it comes with a debian/patch folder. In there is a patch to add the SONAME via configure and with a trailing '.0'.

However, the application of the patches fails, well I can't get it to work with:

Code: Select all

quilt push -a 

failes at the first one 'tkPort' or by directly applying the patches (diff files) using

Code: Select all

patch < file.diff

I think it's okay to proceed as is, but it is annoying as I can't get the patches to apply and that would mean the 'so' will be exactly the same but with debug symbols. All in the fact that any of this is works and is well documented is amazing.

Charlie

[charliedurrant]

Simon,

All debugging and there is nothing obvious in IDLE. TK is compiled for double buffering (X and windows but not for the Mac though). If there are any 1 pixel draws then it must be in XFillRectangles - the ultimate function that fills rectangles.

The only obvious enhancement is if you have to draw 10 lines then only erase the background once for all the lines instead of each line. Every line draws a rect in white and then the background line colour and then the text. To enhance it would mean another private TK function to take an additional param to not erase the background. Erase the background for all the lines, then draw the lines with the new param set to true.

In a plain black and white text window this will save a lot of draws, if there is a lot of syntax highlighting then the relative cost would decrease. I could be barking up the wrong tree here and I'd be happy to send over my Debian VM.

Charlie

p.s. I'm no doubt in the minority as a primary windows developer / user but I'm using the linux build of codelite c++ ide - it has all the same shortcuts as visual studio and it took minutes to get full debugging working.

[tuxit]

Hi,

I have a (hopefully) quick question.

Earlier in this thread, Simon mentioned that a lightweight window manager makes things a bit snappier.

I was wondering if a heavier window manager might be of benefit, in particular kwin.

My thoughts are that kwin can use a GL ES backend, and Jon Severinsson is currently building KDE packages for use with Raspbian: viewtopic.php?f=66&t=22637

From reading this thread, there might be a possibility that kwin will offload to the GPU when there would be less work to do on the CPU. The outcome might be a slower response with more eye candy, and there is also the memory overhead associated with libraries such as qt and kde-runtime.

Is there any straightforward way of profiling a window manager other than playing with the windows and keeping an eye on top? I intend trying kwin with lxde.

[teh_orph]

It's unclear what differences the window manager make to the performance. They will certainly change the performance of the composition of the windows (eg dragging windows over each other), but the contents of the windows are normally rendered independent of the the manager. In which case, that part of the performance will not change. So this will mean if you scroll within a window it'll scroll at the same speed as it'll use the same code path.

I see kwin is a compositing window manager, and it targets XRender in addition to GL/GLES: luckily this is what I have already coded
I'm about to connect large composition ops to GLES in which case you'll probably get similar performance to their GLES backend (I could imagine it could be faster or slower), but it will "just work".

I will have a go with it though.
And in direct answer to your question: profiling is tricky! x11perf is not representative, cairo-perf-trace only really tests cairo and gtkperf has similar issues. Gtkperf might be best though. I'd do it by eye!

[charliedurrant]

Simon,

Took at look at TK again. Isn't the answer to turn off double buffering in TK? I say that as question to myself as I know you don't have the code, here is what is happening....

Double buffering on:
(#define TK_NO_DOUBLE_BUFFERING=1)

1) Create a buffer the size of the area to draw called pixmap (sized for all the lines to draw)
for all lines ()
{
a) Draw line at the top of pixmap (it really it does!) - remember that the pixmap is sized for all the lines
b) Copy the line to screen
}

It's clear that it should be

1) Create a buffer the size of the area to draw = pixmap (sized for all the lines to draw)
2) Draw line(s) at an offset to the top of pixmap
3) Copy the buffer to the screen

Double buffering off
1) Draw each line to the screen buffer

Now, if I'm not mistaken (I'm easily mistaken) but if we write directly to the screen is that not better as the driver will handle all the off screen buffering?

Charlie

[teh_orph]

Yeah go with double-buffering off. The generic fbdev driver has shadowing on, so is effectively double-buffered.
I don't bother myself (mine is single-buffered) but it appears to have no adverse effects.

[charliedurrant]

...some work to do as it doesn't compile without double buffering, I'll upload the so and install guide so you can test it.

Charlie

[charliedurrant]

Simoin,

Are you using a cross compiler? I'm getting C compiler can not create executables when compiling libktk on the Pi itself (gcc v 4.7.2)?

Charlie

[charliedurrant]

Finally a libtk with double buffering off for testing:

http://www.wilverley.com/other/libtk8.5.so.0.1.zip

1) download, extract and copy the file libtk8.5.so.0.1 to /usr/local/lib
2) ldconfig -v | grep libtk
you should see libtk8.5.so.0.1 in the list
3) nano /usr/share/tcltk/tk8.5/tk.tcl
comment the line that reads
package require -exact Tk 8.5.11
and add below it
package require Tk 8.5.0

4) start idle and you will see it doesn't draw its window correctly - that is becaue clipping is not correct since double buffering is off - also file open shows nothing (not sure why at the moment) so you'll have to pass a file to idle on startup or type something in.

To remove:
1) delete the new so file
2) Edit the tk.tcl file back to as before - (not a problem to leave though)
3) run ldconfig -v

Charlie

[teh_orph]

Interesting. It's a definite improvement in the workload that it's sending in.
Now I see no extraneous memory allocations. It seems to do everything with one buffer that I control and one that I don't (let's call them A and B). Both buffers appear to be the width of the screen and high enough for one line of text.

It properly clears the area taken by one line of text in A.
It iterates through each character in a row, and draws them to A.
It then does a full synchronise.
It then downloads that row of text from A into B, copying it into the top left corner of B (it is not 'assembling' the output a row at a time or anything - looks like just a copy).
It then does potentially anything with B (I think it internally blits it into place in the full image).
It then repeats for the next line of text.

Much better! However you can properly see the display paint itself (most noticeably if you select a block of text and keep your mouse button down, but don't move the cursor). I've been thinking about this double/single buffering issue and have realised, that since I do all my work in offscreen memory (A) you shouldn't see any trouble from *my* code if you select single buffering. So perhaps any painting effect (like selecting text) is the rest of X at work...

Yeah and the menus don't work
What now?

[charliedurrant]

Turning off the double buffering was an easy win - there is a lot more (well, some) that can be done in terms of optimising it . In reply:

1) If the code does double buffering would you own that buffer and would the calls be accelerated? I'm asking this as I don't know about the clipping region and with double buffering off which is causing those initial artifacts and the menu / file open dialog issues.

2) I need to mate the calls to exactly what you are receiving so I can optimise it properly, I could go and change things by looking at the code but I must first make sure I can trace the graphics calls to your callstack. Is there a way to have real time output of the calls you are receiving? That's a big ask but in general for optimising code it would proove invaluable.

3) "However you can properly see the display paint itself (most noticeably if you select a block of text and keep your mouse button down, but don't move the cursor). I've been thinking about this double/single buffering issue and have realised, that since I do all my work in offscreen memory (A) you shouldn't see any trouble from *my* code if you select single buffering. So perhaps any painting effect (like selecting text) is the rest of X at work..."

I'll take a look at this. It's all hand drawn so I doubt it's X - that's a guess though.

4) "It then does a full synchronise" - apologies but in laymans terms, what does this mean.
5) This A & B thing, I understand A, but I don't see B being used in TK - I'll investigate on the train in to work tomorrow.
6) What is your tool chain, are you solely working on the PI?
7) Numpty question, but the PI expects libtk8.5.11 and apt-get source libtk8.5-dev gives 8.5.8. I believe the PI uses it's own repositories, apt-get source libtk8.5-dev on the PI gives me nought. Where do you get your PI source code from?

Charlie

[teh_orph]

In answer to your questions,
1) it depends on how it implements it of course. If proper X pixmaps are used (not something internal to Tk or whatever) then I own that memory - but I can't necessarily do anything with it (see #5). If operations such as copy, fill or composite are performed on them then an addition copy is made that I can touch. Think of this copy living in "video memory". The XRender extension must be used by the application, else I don't get a go at all. Many functions may be turned into calls that I can accelerate (copy, fill, composite are the only thing I'm allowed to do), so it's always a bit of guess work where the work will go! If you like I can give you a copy of the driver that prints out everything that is happening?

2) yep. You can't turn the feature on or off (AFAIK) in the stock driver or X but as mentioned I could give you a driver which does this.

3) this may be an artefact of something in X that I'm guessing exists: if the drawing is taking too long, it says "stop as soon as you can, once you've done that then let's drawn to the screen what you've got". So I have a feeling that the workload may depend on the run-time.

4) this is when the subsystem that I implement (XRender) requires you to have finished all asynchronous work that has been previously submitted. eg it may ask me for a fill operation on a pixmap but I'm free to do it whenever I want, as long as it's in order with respect to the other work items (think of a dependency graph of all the drawing operations). However when a synchronise message comes in it's a blocking call from X that requires me to ensure all previous work has been completed and nothing is outstanding. This allows the rest of the system to free up memory etc or allow the application access to pixmap memory, which would require the pixmap in question to be in a coherent state. Also alluded to by #3 I think it sends more synchronise calls when the update speed gets low, so the display at least updates even if partially complete.

NB synchronise call frequency is a blocker on parallelism. If they come too often I can't hide hardware latency etc. This is why I've implemented a CPU fall-back in case it's faster to do the work on the CPU. However, I never know how long until a synchronise call is going to come in...so it's a guessing game whether its worth batching stuff up for the hardware to handle or just get going on the CPU.

5) B may be a buffer that has operations done on it that I'm not allowed to handle. Again XRender only accelerates copy, fill, composite. Non-XRender code can't touch my A buffer, as it thinks it's in "video memory", so it maintains an equivalent buffer in CPU memory where it can do its CPU work. In future I'm hoping to wiggle EXA such that I can just do everything in one buffer.

6) I do everything on the Pi. Damn it's slow! When building Xorg however I use distcc just to take the edge off. My PC idles during this process but it does greatly improve compile times. It's a lot of hassle though.

7) Whilst on the Pi I do the apt-get source PACKAGENAME thing too. Then apt-get build-dep PACKAGENAME, then cd into that package directory then dpkg-buildpackage -rfakeroot -uc -us to build the package file. I'm not very savvy to this Debian stuff though.

[charliedurrant]

Simon,

I haven't dissapeared and thanks for the detailed reply. I suddenly decided to build a wheezy vm so I can match the pi install as close as possible. I've beeen making a right meal of it. I'll start on optiising it as sson as I can and digest what you have written. I need to also to talk to the tk / tcl maintainers.

What you said about a 'full synchronise' is interesting as esentially it means that the drawing / text code needs to be as quick as possible!

I'm going to get cross compiling working on my VM, I've seen posts about remote debugging which may help you as you must be having a hard time working directly on the Pi.

I'll report back soon..

Charlie

p.s. I'm currently attempting to teach my children some python and I genuinely believe this is an important area, i.e. making the general UI as responsive as possible with respect to schools etc. Sounds a bit naff....but I do think what you are doing will make a big difference....!

[teh_orph]

What you said about a 'full synchronise' is interesting as esentially it means that the drawing / text code needs to be as quick as possible!

Haha yes! Or the least amount of work to get the same visual effect as possible
Ideally if work is to be submitted it's also nice to get it in big blocks. eg a fill that's 100x100, rather than 10000 1x1 fills. The copy and fill operations when powered by DMA need quite some time to get into their stride. I've geared composite for tiny operations though (think lots of characters of text) and doesn't scale well to large blocks...but that may change soon

I'm going to get cross compiling working on my VM, I've seen posts about remote debugging which may help you as you must be having a hard time working directly on the Pi.

Debugging is easy enough, I could even hook it up to Eclipse on my PC if required. I do it all via gdb and SSH, and have about a dozen putty windows open which delights my girlfriend. Also a USB serial port connected to the GPIO in case I need to debug the kernel!
It's the code build time that is annoying IMO

p.s. I'm currently attempting to teach my children some python and I genuinely believe this is an important area, i.e. making the general UI as responsive as possible with respect to schools etc. Sounds a bit naff....but I do think what you are doing will make a big difference....!

Definitely a worthy thing to do. But you need a good example of good versus bad. My mum always said no to hardware upgrades when I was a growing up but when she saw me playing Doom on xmas day in the tiniest of windows at 2-3 fps that's when she finally got what I was talking about! Waiting for a paint pot to ray trace is quite different to an interactive application...

---------------

Anyway, driver update: work has been done on the non-interesting driver stuff for about a week now to,
a) make it easy for people to just do rpi-update and get all the tricky things installed for you (ie the kernel module)
b) make it easy for people to set up the driver - it now supports the experimental floating CPU/GPU memory split so you haven't got to "pick a split" if you want to use my code.
c) provide a framework to accelerate large composition operations...this could pay off big time if I'm lucky.

If I can get some easy wins with (c) then I'd be much happier to release that version than what I had a month ago.

Big thanks to Dom for his work.

[charliedurrant]

Re - libTK

Ummm, things can certainly be done better but too much code needs to be refctored which with a well used library would be big risk. New so / dll at

http://www.wilverley.com/other/libtk.zip
1) extract the contents to /usr/local/lib
2) ldconfig -v to update the shared object cache
3) start idle and watch how slow it is (well it is on my pi)

I don't think you'll see much difference and A & B will still be there. Lib tk creates a graphics context (gc) for foreground work and background work, not sure why as it seems very inneficient - there will be a good reason though. I'm so new to this (and everything else linux) and I can't pass comment on why or what is going on underneath yet.

There are scrolling artifacts (on the right side with long lines, due to my lack of understanding of clipping (libtk only supports clipping via a bitmap and I want to do it via a rectangle) and clipping needs to be done on the foregound gc and the background gc?!?

Menus now work, and the version, 8.5.11, matches what is on the PI, but I think the the graphics call stack will be the same as the previous version.

Could I install your driver to test? My email is charliedurrant atttt g mail .com, if that is okay?

Charlie