Java OpenGL ES 2 bindings using JogAmp JOGL 2

[xranby]

Hi friends, I have worked with the JogAmp community to create a low overhead Java binding to OpenGL ES 2 on the Raspberry Pi. The first JogAmp JOGL auto-build with included Broadcom VC IV NEWT driver got published on the 20 Aug 2012. You can use this build to run some impressive Java demos on your Pi, it will work on both armhf and armel systems.

Code: Select all

sudo apt-get install p7zip-full

# Download and unpack the latest JogAmp release
wget http://jogamp.org/deployment/jogamp-current/archive/jogamp-all-platforms.7z
7z x jogamp-all-platforms.7z
cd jogamp-all-platforms

# Download, compile and run an OpenGL ES 2 vertex and fragment shader introduction
mkdir -p demos/es2
cd demos/es2
wget https://raw.github.com/xranby/jogl-demos/master/src/demos/es2/RawGL2ES2demo.java
cd ../..
javac -cp jar/jogl-all.jar:jar/gluegen-rt.jar demos/es2/RawGL2ES2demo.java
java -cp jar/jogl-all.jar:jar/gluegen-rt.jar:. demos.es2.RawGL2ES2demo

# Run the classic Gears demo, found in the JogAmp JOGL unit-tests.
# requires junit4
sudo apt-get install junit4
java -cp jar/jogl-all.jar:jar/gluegen-rt.jar:/usr/share/java/junit4.jar:jar/jogl-test.jar:jar/gluegen-test-util.jar \
com/jogamp/opengl/test/junit/jogl/demos/es2/newt/TestGearsES2NEWT \
 -time 400000

# Render streamed movies to textures
# Require the ffmpeg libavcodec libavutil and libavformat to be installed
# or else you will only see a test screen.
sudo apt-get install libav-tools
java -cp jar/jogl-all.jar:jar/gluegen-rt.jar:/usr/share/java/junit4.jar:jar/jogl-test.jar \
com/jogamp/opengl/test/junit/jogl/demos/es2/av/MovieCube \
-time 400000 -width 1980 -height 1080

A more technical introduction to this work have been posted here: http://labb.zafena.se/?p=547

[xranby]

You can always find the latest JogAmp signed release with RaspberryPi support at the following location:
http://jogamp.org/deployment/jogamp-current/

Unsigned "Nightly" test builds are also available here:
http://jogamp.org/deployment/archive/master/?C=M;O=D

[xranby]

http://www.raspberrypi.org/phpBB3/viewt ... 32#p186832 -
JOGL next steps this thread goes into how to load Textures and storing data inside the GPU using OpenGL Vertex Buffer Object (VBO).

[Howard]

Have just tried the demo that xranby posted:

Code: Select all

pi@despatcher:~$ cd jogamp-all-platforms/
pi@despatcher:~/jogamp-all-platforms$ javac -cp jar/jogl-all.jar:jar/gluegen-rt.jar RawGL2ES2demo.java 
pi@despatcher:~/jogamp-all-platforms$ java -Dnativewindows.ws.name=jogamp.newt.driver.bcm.vc.iv -cp jar/jogl-all.jar:jar/gluegen-rt.jar:. RawGL2ES2demo
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: could not query Visual by given VisualID 0x9428, bail out!
	at jogamp.newt.DefaultEDTUtil.invokeImpl(DefaultEDTUtil.java:205)
	at jogamp.newt.DefaultEDTUtil.invoke(DefaultEDTUtil.java:135)
	at jogamp.newt.DisplayImpl.runOnEDTIfAvail(DisplayImpl.java:204)
	at jogamp.newt.WindowImpl.runOnEDTIfAvail(WindowImpl.java:1576)
	at jogamp.newt.WindowImpl.setVisible(WindowImpl.java:776)
	at jogamp.newt.WindowImpl.setVisible(WindowImpl.java:780)
	at com.jogamp.newt.opengl.GLWindow.setVisible(GLWindow.java:394)
	at RawGL2ES2demo.main(RawGL2ES2demo.java:279)
Caused by: java.lang.RuntimeException: could not query Visual by given VisualID 0x9428, bail out!
	at jogamp.newt.driver.x11.WindowDriver.CreateWindow0(Native Method)
	at jogamp.newt.driver.x11.WindowDriver.createNativeImpl(WindowDriver.java:97)
	at jogamp.newt.WindowImpl.createNative(WindowImpl.java:284)
	at jogamp.newt.WindowImpl.setVisibleActionImpl(WindowImpl.java:723)
	at jogamp.newt.WindowImpl$VisibleAction.run(WindowImpl.java:768)
	at com.jogamp.common.util.RunnableTask.run(RunnableTask.java:131)
	at jogamp.newt.DefaultEDTUtil$EventDispatchThread.run(DefaultEDTUtil.java:313)
X11Util.Display: Shutdown (JVM shutdown: true, open (no close attempt): 3/3, reusable (open, marked uncloseable): 0, pending (open in creation order): 3)
X11Util: Open X11 Display Connections: 3
X11Util: Open[0]: NamedX11Display[:0.0, 0x14197a0, refCount 1, unCloseable false]
X11Util: Open[1]: NamedX11Display[:0.0, 0x1432fa8, refCount 1, unCloseable false]
X11Util: Open[2]: NamedX11Display[:0.0, 0x14441b0, refCount 1, unCloseable false]
X11Util: Reusable X11 Display Connections: 0
X11Util: Pending X11 Display Connections (creation order): 3
X11Util: Pending[0]: NamedX11Display[:0.0, 0x14197a0, refCount 1, unCloseable false]
X11Util: Pending[1]: NamedX11Display[:0.0, 0x1432fa8, refCount 1, unCloseable false]
X11Util: Pending[2]: NamedX11Display[:0.0, 0x14441b0, refCount 1, unCloseable false]
pi@despatcher:~/jogamp-all-platforms$ 
pi@despatcher:~/jogamp-all-platforms$ 

Doesn't sound like any of the files are missing. Same error root user and pi (apart from the big hex numbers in X11Util which I assume are memory pointers).

Any ideas?

[xranby]

Have just tried the demo that xranby posted:

Code: Select all

pi@despatcher:~$ cd jogamp-all-platforms/
pi@despatcher:~/jogamp-all-platforms$ javac -cp jar/jogl-all.jar:jar/gluegen-rt.jar RawGL2ES2demo.java 
pi@despatcher:~/jogamp-all-platforms$ java -Dnativewindows.ws.name=jogamp.newt.driver.bcm.vc.iv -cp jar/jogl-all.jar:jar/gluegen-rt.jar:. RawGL2ES2demo
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: could not query Visual by given VisualID 0x9428, bail out!
pi@despatcher:~/jogamp-all-platforms$ 

Doesn't sound like any of the files are missing. Same error root user and pi (apart from the big hex numbers in X11Util which I assume are memory pointers).

Any ideas?

You are using the correct file-names.
The message indicate that your jogl still defaults back to the X11 back-end.

Which version of JogAmp JOGL have you downloaded?
I think you are still using RC10 from august 3 2012 and that build do not contain the .jogamp.newt.driver.bcm.vc.iv driver.

Please check that you are using a recent "daily" build of JogAmp that include the jogamp.newt.driver.bcm.vc.iv driver

Code: Select all

cd jogamp-all-platforms
cat jogl.artifact.properties

Only JogAmp jogl builds from 20 aug 2012 and later is Pi compatible.

The current source tree from 1 oct 2012 and later include Pi auto-detection
https://github.com/sgothel/jogl/commit/ ... 77d2d7db6b
so that you do not need to pass -Dnativewindows.ws.name=jogamp.newt.driver.bcm.vc.iv any more!
The Pi NEWT driver and auto-detection code will both be part in the next JogAmp RC11 release candidate.

[xranby]

Please help test new JogAmp JOGL aggregated build (pre RC11)

The new pre RC11 build can be downloaded here:
http://jogamp.org/deployment/archive/ma ... atforms.7z
http://jogamp.org/deployment/archive/ma ... 86-signed/

If you encounter any issues please reply to the JogAmp forum thread:
http://forum.jogamp.org/Pls-test-new-JO ... 26402.html

[Howard]

Which version of JogAmp JOGL have you downloaded?
I think you are still using RC10 from august 3 2012 and that build do not contain the .jogamp.newt.driver.bcm.vc.iv driver.

That fix it thanks. I had seen your update and followed its link, but then in finding the file I needed I must have managed to find my way down the wrong path.

OK. So I have the wobbling triangle.
I was actually hoping to run the mandelbrot software from http://blog.nobel-joergensen.com/2010/0 ... rt-2-jogl/
but that seems to need more openGL than we have (after I made the code changes required to get it to compile). I thought a mandelbrot 'fly-past' would make a nice background when nothing else was happening.

[xranby]

JogAmp JOGL RC11 is now released with included Raspberry Pi drivers!
Rasbian armhf and Debian armel are both supported out of the box.

RC11 Release notes: http://forum.jogamp.org/Release-v2-0-rc ... 26703.html
Deployment instructions: http://jogamp.org/jogl/doc/deployment/J ... YMENT.html

JogAmp RC11 is published to the Maven Central for easy integration into Apache Maven built projects!
http://search.maven.org/#search|ga|1|jogamp

[xranby]

http://www.youtube.com/watch?v=7mBi7psP7VM
Teaser / Demo clip of our JogAmp BOF @ Siggraph 2012 in LA, 7 aug 2012
Where we demonstrated, for the first time, JogAmp JOGL 2 running on the Raspberry Pi!

[xranby]

http://www.youtube.com/watch?v=gYoj2s18GxU JogAmp BOF @ Siggraph 2012 LA - Full Session 1h 11min #Java #JogAmp #JOGL #OpenGL 2D/3D & multimedia across devices.

Raspberry Pi is demonstration starts at 11:30 min into the talk
http://youtu.be/gYoj2s18GxU?t=11m30s

Raspberry Pi is also demonstrated in the QA session:
http://youtu.be/gYoj2s18GxU?t=1h8m30s

[xranby]

Quick: Raspberry Pi JogAmp JOGL (OpenGL ES) setup demo (~7Mb) : https://gist.github.com/55de4fd65bfadae26203
Quick: Raspberry Pi JogAmp JOAL (3D Sound) setup demo (~2Mb) : https://gist.github.com/4160432

[xranby]

Slides, Video and Teaser from the JogAmp FOSDEM 2013 freejava love talk are now online:
http://jogamp.org/doc/fosdem2013/

Includes live demonstrations running on the Raspberry Pi.

[-rst-]

Would this thread be better off in the Java sub-forum (...or Graphics/OpenGL)?

[xranby]

Would this thread be better off in the Java sub-forum (...or Graphics/OpenGL)?

This thread was created some months before the java sub-forum got created.
It would now make sense to move this post to the Java sub-forum so that new Java developers find the thread IF such an move would leave all links pointing to this thread intact.

JogAmp is a community project targeting Java OpenGL/ES OpenCL OpenAL OpenMAX usb input bindings and additional similar engineering.

[xranby]

marwatk have published a "Photo collage screensaver app" written in Java using JogAmp JOGL.
http://www.raspberrypi.org/phpBB3/viewt ... 38&t=32932 to replace his old pc running picasa setup.

this uses MUCH less power

Cool!

[krishnak]

I am trying to use LWJGL on PI with out using X. Can you give me a pointer, as to where to start so that I can have apps starting up like the jogamp demo but using LWJGL.

[xranby]

I am trying to use LWJGL on PI with out using X. Can you give me a pointer, as to where to start so that I can have apps starting up like the jogamp demo but using LWJGL.

LWJGL is currently architectured to rely on AWT and hence X11 to open a window and to provide mouse and keyboard input. If you want to improve LWJGL you need to re-architecture LWJGL to have a platform independent interface to open a window and a platform independent way to get input. This would allow LWJGL to scale to operate on platforms where AWT / X11 may not be available such as the Raspberry Pi, iOS and Android.

JogAmp solves the above by using its NEWT architecture a platform independent windowing and event handling API that JOGL have been deploying and polishing since 2008. We have implemented keyboard and mouse /dev/input/* parsers that NEWT use on the Raspberry Pi in order to get input from console without having access to X11.

The LWJGL team know about the above limitations and is re-architecture their API to add functionality similar to NEWT into the future LWJGL 3.0 that is currently under initial testing. I would recommend them to simply use NEWT directly it would solve the Raspberry Pi console input.
http://lwjgl.org/forum/index.php/topic,4800.0.html

The OpenGL ES 2 API itself is the same for JogAmp JOGL and LWJGL so try use JogAmp if you need to run OpenGL ES 2 applications outside X11 using NEWT.

In general I recommend you to write games/apps using a high level engine such as libgdx or jMonkeyEngine3, The JogAmp community got a forum thread where we improve RaspberryPi support for said game engines.
http://forum.jogamp.org/JOGL-2-0-OpenGL ... 27689.html

We ran the opensource libgdx pax-britannica game on the RaspberryPi from console using JogAmp JOGL 2 during the FOSDEM 2013 JogAmp love talk. http://labb.zafena.se/?p=681

[krishnak]

Thank you for your detailed and quick response. I got confused by few of the postings I read either in Jogamp thread or over here, that Jmonkey depends on LWJGL and hence can't run in headless mode.

I will give both libgdx and JMonkey a try to develop a java headless application.

[xranby]

Thank you for your detailed and quick response. I got confused by few of the postings I read either in Jogamp thread or over here, that Jmonkey depends on LWJGL and hence can't run in headless mode.

We have users reporting running jMonkeyEngine3 60fps on the raspberry pi from console using the unified desktop/mobile JOGL 2 backend so these confusing/outdated rumour/posts you have read are simply not/nolonger true jME3 work great in headless mode.
http://forum.jogamp.org/JOGL-2-0-OpenGL ... 28115.html

I will give both libgdx and JMonkey a try to develop a java headless application.

Cool!

Both libgdx and jme3 got backends for both LWJGL and JOGL, of course only one backend is selected by default you simply need to pick the JOGL backend and pick the right window size to get it running in fullscreen from console. For example all changes i had to do to the libgdx pax-britannica game to switch to the JOGL backend was inside its main method like this: https://github.com/xranby/libgdx/blob/f ... sktop.java

If you run into any issues read and then post to the JogAmp backed support thread:
http://forum.jogamp.org/JOGL-2-0-OpenGL ... 27689.html

You may also post to the jME3 JOGL 2 thread:
http://jmonkeyengine.org/forum/topic/jo ... is/page/3/

[krishnak]

Many thanks, will keep updated

[krishnak]

I thought I will post an update:

To get JMonkey working with JOGL - Jmonkey needs to be built from source. With several helpful hints from JOGL forum members http://forum.jogamp.org/JOGL-2-0-OpenGL ... 89i60.html I managed it

I was developing it on a Ubuntu PC. With the appropriate native jars - I am able to run the JMonkey sample code on Ubuntu using JOGL.


When I move this application to RPi and replace the glugen /Jogl native jars with RPi specific armv6-hf.jars

I get this stack trace - I get this with and with out X - any thoughts on what is causing this

pi@raspberrypi ~/jogamp $ java -jar test1.jar
Feb 19, 2013 3:00:18 AM com.jme3.system.JmeDesktopSystem initialize
INFO: Running on jMonkeyEngine 3.0.0 RC2
BCM.Display initIDs ok
BCM.Screen initIDs ok
BCM.Window initIDs ok
BCM.Screen initNative ok 1920x1080
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: GLSL and OpenGL2 is required for the JOGL renderer!
at jogamp.newt.DefaultEDTUtil.invokeImpl(DefaultEDTUtil.java:205)
at jogamp.newt.DefaultEDTUtil.invoke(DefaultEDTUtil.java:135)
at jogamp.newt.DisplayImpl.runOnEDTIfAvail(DisplayImpl.java:204)
at jogamp.newt.WindowImpl.runOnEDTIfAvail(WindowImpl.java:1577)
at jogamp.newt.WindowImpl.setVisible(WindowImpl.java:777)
at jogamp.newt.WindowImpl.setVisible(WindowImpl.java:781)
at com.jogamp.newt.opengl.GLWindow.setVisible(GLWindow.java:394)
at com.jme3.system.jogl.JoglNewtDisplay.createGLFrame(JoglNewtDisplay.java:67)
at com.jme3.system.jogl.JoglNewtDisplay.privateInit(JoglNewtDisplay.java:147)
at com.jme3.system.jogl.JoglNewtDisplay.create(JoglNewtDisplay.java:168)
at com.jme3.app.Application.start(Application.java:387)
at com.jme3.app.Application.start(Application.java:366)
at com.jme3.app.SimpleApplication.start(SimpleApplication.java:131)
at mygame.Main.start(Main.java:104)
at mygame.Main.main(Main.java:29)
Caused by: java.lang.UnsupportedOperationException: GLSL and OpenGL2 is required for the JOGL renderer!
at com.jme3.renderer.jogl.JoglRenderer.initialize(JoglRenderer.java:168)
at com.jme3.system.jogl.JoglNewtDisplay.init(JoglNewtDisplay.java:162)
at jogamp.opengl.GLDrawableHelper.init(GLDrawableHelper.java:332)
at jogamp.opengl.GLDrawableHelper.init(GLDrawableHelper.java:352)
at jogamp.opengl.GLAutoDrawableBase$1.run(GLAutoDrawableBase.java:268)
at jogamp.opengl.GLDrawableHelper.invokeGLImpl(GLDrawableHelper.java:653)
at jogamp.opengl.GLDrawableHelper.invokeGL(GLDrawableHelper.java:594)
at com.jogamp.newt.opengl.GLWindow.display(GLWindow.java:543)
at jogamp.opengl.GLAutoDrawableBase.defaultWindowResizedOp(GLAutoDrawableBase.java:128)
at com.jogamp.newt.opengl.GLWindow.access$100(GLWindow.java:94)
at com.jogamp.newt.opengl.GLWindow$1.windowResized(GLWindow.java:112)
at jogamp.newt.WindowImpl.consumeWindowEvent(WindowImpl.java:2374)
at jogamp.newt.WindowImpl.sendWindowEvent(WindowImpl.java:2317)
at jogamp.newt.WindowImpl.setVisibleActionImpl(WindowImpl.java:758)
at jogamp.newt.WindowImpl$VisibleAction.run(WindowImpl.java:769)
at com.jogamp.common.util.RunnableTask.run(RunnableTask.java:131)
at jogamp.newt.DefaultEDTUtil$EventDispatchThread.run(DefaultEDTUtil.java:313)
EGLDisplayUtil.EGLDisplays: Shutdown (open: 1)
EGLDisplayUtil: Open EGL Display Connections: 1
EGLDisplayUtil: Open[0]: 0x1: refCnt 1

[xranby]

I thought I will post an update:

To get JMonkey working with JOGL - Jmonkey needs to be built from source. With several helpful hints from JOGL forum members http://forum.jogamp.org/JOGL-2-0-OpenGL ... 89i60.html I managed it
...
I get this stack trace - I get this with and with out X - any thoughts on what is causing this

pi@raspberrypi ~/jogamp $ java -jar test1.jar
Feb 19, 2013 3:00:18 AM com.jme3.system.JmeDesktopSystem initialize
INFO: Running on jMonkeyEngine 3.0.0 RC2
BCM.Display initIDs ok
BCM.Screen initIDs ok
BCM.Window initIDs ok
BCM.Screen initNative ok 1920x1080
Exception in thread "main" java.lang.RuntimeException: java.lang.UnsupportedOperationException: GLSL and OpenGL2 is required for the JOGL renderer!
at jogamp.newt.DefaultEDTUtil.invokeImpl(DefaultEDTUtil.java:205)
...
at com.jme3.app.SimpleApplication.start(SimpleApplication.java:131)
at mygame.Main.start(Main.java:104)
at mygame.Main.main(Main.java:29)
Caused by: java.lang.UnsupportedOperationException: GLSL and OpenGL2 is required for the JOGL renderer!
at com.jme3.renderer.jogl.JoglRenderer.initialize(JoglRenderer.java:168)
...
EGLDisplayUtil: Open[0]: 0x1: refCnt 1

The lines
BCM.Display initIDs ok
BCM.Screen initIDs ok
BCM.Window initIDs ok
BCM.Screen initNative ok 1920x1080
Indicate that the Raspberry Pi native screen have opened OK...

When you run into an exception good debugging practise is to locate the Caused by: exception line in this case it looks like the jMonkeyEngine3 source tree have requested an fixed function profile and tries to use programmable shaders, this work on desktop because OpenGL 2 support both fixed function and shaders, on mobile you have to choose of using OpenGL ES 1 (fixed function) or OpenGL ES 2 (programmable shaders) unlike desktop on mobile the two OpenGL modes cant mix, and if you try to mix them you get these java.lang.UnsupportedOperationException. Make sure that the jME3 JOGL initialization line request an GL2ES2 profile so that JOGL gives you an OpenGL profile that is compatible with both desktop OpenGL 2 and OpenGL ES 2 calls.
Analyse the jME3 sourcecode at com.jme3.renderer.jogl.JoglRenderer.initialize(JoglRenderer.java:168)

Most likely you are using the upstream jME3 sourcetree that is still missing some OpenGL ES 2 specific patches to make jME3 run on the Pi.

Use the jME3 SourceTree maintained by noxo since it is known to work on the Raspberry Pi.
git clone the following git source tree maintained by noxo and then git checkout the unified_renderer git branch
noxo's RaspberryPi jME3 git branch https://github.com/noxo/jmonkey-engine- ... d_renderer
This tree shows the commits needed to update the jME3 svn to become GL2ES2 (both desktop and mobile) compatible. And with these changes in place jME3 work on the Raspberry Pi.

noxo have fixed the exception you ran into using this commit:
https://github.com/noxo/jmonkey-engine- ... f043158195
The Default profile gives the higest profile available on the device and on the Raspberry Pi it will return an OpenGL ES 2 compatible profile. It would have in my humble opinion been better to explicitly request an GL2ES2 profile.

[xranby]

The best way to fix the exception you ran into is to change JME3 to use code similar to the following libgdx JOGL initialization. By initializing JOGL to use OpenGL ES 1 or OpenGL ES 2 depending on the features requested by the jME3 application configuration would also fix the TODO in the jME3 sourcecode.
https://github.com/xranby/libgdx/commit ... 86e9c1f506

[krishnak]

Many thanks for clearly explaining a novice about the finer aspects of JOGL - I will follow your advise and update the progress

[xranby]

...
OK. So I have the wobbling triangle.
I was actually hoping to run the mandelbrot software from http://blog.nobel-joergensen.com/2010/0 ... rt-2-jogl/
but that seems to need more openGL than we have (after I made the code changes required to get it to compile). I thought a mandelbrot 'fly-past' would make a nice background when nothing else was happening.

Remember that OpenGL ES SL (the shader language) gets sent to the OpenGL ES 2 driver in clear text and then gets compiled by the Raspberry Pi Broadcom driver, the compilation is performed using code running on the GPU part of the Video Core IV firmware. If you check the mandelbrot shader demo inside /opt/vc/src hello triangle 2 i think notice that the fragment shader only allows loops about 16 iterations deep. This is a limitation inside the Broadcom firmware itself and can only be fixed if we reverse engineer and create an opensource BCM IV firmware, or if the Broadcom engineers release a new firmware with less restrictions in the OpenGL ES SL compiler.