Processing + Java 8 + JOGL

[guidow]

I'm having a shot at running Processing 2.07 with Java 8 EA (Just the JIT). Basic sketchs that don't require OPENGL work fine. Anything using P2D or P3D now uses OPENGL as I understand it.

So I headed for JOGL:
Followed the instructions by the ever helpful xranby here:
http://www.raspberrypi.org/phpBB3/viewt ... 35#p238135

Got the -Djava.library.path=/usr/lib/arm-linux-gnueabihf by editing the processing startup shell script

Got this setting which I saw in xranby's examples into the execution of a Processing sketch by editing the Processing preferences.txt
run.options=-Dnativewindows.ws.name=jogamp.newt.driver.bcm.vc.iv
Used System.getProperties in a println to make sure they were set inside the executing sketch

Downloaded the JOGL implementation, unzipped and copied the files into processing/core/library:
gluegen-rt.jar, gluegen-rt-natives-linux-armv6hf.jar, jogl-all.jar, jogl-all-natives-linux-armv6hf.jar

Fired up the MoveEye sample and got the following error:

Code: Select all

Exception in thread "Animation Thread" java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Native windowing type jogamp.newt.driver.bcm.vc.iv (custom) not yet supported, platform reported native windowing type: .bcm.vc.iv
at jogamp.nativewindow.NativeWindowFactoryImpl.getAWTNativeWindow(NativeWindowFactoryImpl.java:99)
at jogamp.nativewindow.NativeWindowFactoryImpl.getNativeWindowImpl(NativeWindowFactoryImpl.java:65)
at javax.media.nativewindow.NativeWindowFactory.getNativeWindow(NativeWindowFactory.java:486)
at jogamp.newt.awt.NewtFactoryAWT.getNativeWindow(NewtFactoryAWT.java:70)
at com.jogamp.newt.awt.NewtCanvasAWT.reparentWindow(NewtCanvasAWT.java:428)
at com.jogamp.newt.awt.NewtCanvasAWT.addNotify(NewtCanvasAWT.java:391)
at java.awt.Container.addImpl(Container.java:1114)
at java.awt.Container.add(Container.java:966)
at processing.opengl.PGL.initSurface(PGL.java:606)
at processing.opengl.PGraphicsOpenGL.initPrimary(PGraphicsOpenGL.java:5735)
at processing.opengl.PGraphicsOpenGL.requestDraw(PGraphicsOpenGL.java:1568)
at processing.core.PApplet.run(PApplet.java:2020)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.IllegalArgumentException: Native windowing type jogamp.newt.driver.bcm.vc.iv (custom) not yet supported, platform reported native windowing type: .bcm.vc.iv
at jogamp.nativewindow.NativeWindowFactoryImpl.getAWTNativeWindow(NativeWindowFactoryImpl.java:92)
... 12 more
BCM.Display initIDs ok
BCM.Screen initIDs ok
BCM.Window initIDs ok

I did not edit out the " ... 12 more", that's how the error dumped.

Any other suggestions?
Maybe I have to put some other libraries where Processing will find them?
If so which ones?

[xranby]

The Processing + Java 8 + JOGL setup you have is correct and would work perfectly on all ARM Linux systems where the system installed libEGL used a regular X11 "Window" data-structure. Unfortunately the libEGL on the Raspberry Pi is not one of those systems :/
The libEGL.so (the library that you use to initialize OpenGL ES) on the RaspberryPi is different compared to libEGL you find on most Android and GNU/Linux machines.

EGL API background:
It is common that libEGL EGLNativeWindowType differ for OpenGL ES implementations across operating systems, for example:
On Windows EGLNativeWindowType is a "HWND" datastructure.
On Linux/Android EGLNativeWindowType is a "ANativeWindow*" datastructure pointer.
On Linux/X11 EGLNativeWindowType is a "Window" datastructure.
and so on...
On The Raspbeery Pi using a Linux/Broadcom Videocore IV EGLNativeWindowType is a "EGL_DISPMANX_WINDOW_T" datastructure.
http://www.khronos.org/egl
http://elinux.org/RPi_VideoCore_APIs

JOGL can render OpenGL ES on a EGL_DISPMANX_WINDOW_T datastructure using the NEWT bcm.vc.iv driver on the Pi. Code using JOGL and the NEWT windowing tool-kit work hardware accelerated on the Pi. http://jogamp.org/jogl/doc/NEWT-Overview.html
Example of JOGL NEWT code that work on the Pi: https://gist.github.com/55de4fd65bfadae26203

Exception in thread "Animation Thread" java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Native windowing type jogamp.newt.driver.bcm.vc.iv (custom) not yet supported, platform reported native windowing type: .bcm.vc.iv
at jogamp.nativewindow.NativeWindowFactoryImpl.getAWTNativeWindow(NativeWindowFactoryImpl.java:99)

The exception error message you see here is a known limitation inside JOGL, the missing part is support for JOGL to render OpenGL ES directly on top of existing AWT X11 windows on the Raspberry Pi where JOGL know that the RaspberryPI native window type is a .bcm.vc.iv "EGL_DISPMANX_WINDOW_T".
https://jogamp.org/bugzilla/show_bug.cgi?id=626

Processing implements P2D and P3D by design using a AWT Window, and this is implemented inside OpenJDK using a X11 "Window" datastructure.
You receive the runtime exception when JOGL is unable to connect OpenGL ES (that the Broadcom libEGL require to be a "EGL_DISPMANX_WINDOW_T" on the Raspberry Pi) with the existing AWT X11 "Window" native surface datastructure.

Possible ways to solve this situation:
A) make JOGL render to an "offscreen" EGL_DISPMANX_WINDOW_T and then blit to a X11 window using DMA. This solution would add some blitting overhead for each frame (lower fps) but would make the hardware accelerated OpenGL ES work across remote X11 sessions and vnc viewers. New code have to be added to JOGL to make this work.
Example of OpenGL ES blitting code that blit between Dispmanx and X11: https://github.com/shirro/pi-eglonx

B) Use the EGL_DISPMANX_WINDOW_T like an "overlay" that track the X11 AWT Window. This would be faster but not compatible with remote X11 sessions and vnc viewers. (This is similar how hardware accelerated Video playback is done on some platforms, the video output is displayed on top of the operating-system GUI) New code have to be added to JOGL to make this work.
This is the solution chosen by the Mojang Minecraft port.

C) Change the Processing source-code to use NEWT instead of AWT for P2D and P3D sketches. This would improve portability of Processing. It would completely remove the need to have a JRE with AWT and also remove the need for a separate Processing Android back-end because JOGL NEWT already work on mobile systems like Android, MeeGo and desktop GNU/Linux MacOS and Windows.

[guidow]

xranby,
Thank you again for your thorough explanation of the problem. Seems like my experiment will need to come to a halt for now. If I had any say, C) would look to be the best bet since Oracle is not showing much support for AWT in its JavaFX future.
It also seems like Processing would unhitch itself from Oracle. I will await to see if the maintainers of Processing show interest in your recommendation.

[henrik]

xranby,
Thank you again for your thorough explanation of the problem. Seems like my experiment will need to come to a halt for now. If I had any say, C) would look to be the best bet since Oracle is not showing much support for AWT in its JavaFX future.
It also seems like Processing would unhitch itself from Oracle. I will await to see if the maintainers of Processing show interest in your recommendation.

It's correct that we are not enhancing Swing/AWT, but they will be maintained for many years to come so plenty of time to make a transition.

JavaFX might be an option, though. It will soon be fully open source, and we would be positive to native OpenGL integration (JOGL or some other solution).

Henrik

[Monranna]

xranby,
Thank you again for your thorough explanation of the problem. Seems like my experiment will need to come to a halt for now. If I had any say, C) would look to be the best bet since Oracle is not showing much support for AWT in its JavaFX future.
It also seems like Processing would unhitch itself from Oracle. I will await to see if the maintainers of Processing show interest in your recommendation.

It's correct that we are not enhancing Swing/AWT, but they will be maintained for many years to come so plenty of time to make a transition. You should be able to find any of the well accomplished Malvern Dentists

JavaFX might be an option, though. It will soon be fully open source, and we would be positive to native OpenGL integration (JOGL or some other solution).

Henrik

Is JavaFX fully open source yet? Unable to find any information on Github...

[henrik]

Yes, fully open source.

http://openjdk.java.net/projects/openjfx/

Instructions on how to build OpenJFX on RPi:
https://wiki.openjdk.java.net/display/O ... Mhardfloat

Henrik (Oracle Java team)

[dms]

Hi,

Some time ago, I wrote a Java interface to the PI OpenGL and put the code on
SourceForge. It includes examples. I don't know if it would be any help to you.

http://www.spanglefish.com/dmsconsulting/

Regards
Don