How to setup Dolphin on a Pi 4 with Raspbian Buster

[jdonald]

Decided to start a fresh topic. The "I managed to get Dolphin running on a Pi 2?!? (VERY SLOWLY)" thread covers a few different configs so I figured it would be clearest to focus on the Raspbian installation and control the first post on a new topic. Having so many Minecraft Java Edition threads hasn't seemed to cause a problem, after all.

Dolphin requires moving to a 64-bit kernel. This means your system will be prone to camera and video decode have problems, and improperly written wrapper scripts that depend on uname -m (e.g. Wolfram, Kodi) will run into problems. For this reason it's highly recommended to use a new SD card separate from your primary system.

Start with a standard Raspbian Buster desktop installation for your Pi 4B. Do not attempt this on a Pi 3B+ or older because those devices do not support GLES 3.0 in hardware.

(Updated) The Raspbian maintainers now publish a 64-bit kernel. Download it with:

Code: Select all

sudo rpi-update

Edit /boot/config.txt (requires sudo) and add the following line to the [pi4] section.

Code: Select all

arm_64bit=1

Reboot i.e.:

Code: Select all

reboot

Once your system starts up again, check that you are actually running a 64-bit kernel by running:

Code: Select all

uname -m

It should report back aarch64

Create a 64-bit chroot with all the Dolphin runtime dependencies:

Code: Select all

sudo apt-get install -y debootstrap schroot
cat << EOF | sudo tee /etc/schroot/chroot.d/pi64 >/dev/null
[pi64]
description=Debian Buster arm64
type=directory
directory=/srv/chroot/pi64
users=pi
root-groups=root
profile=desktop
personality=linux
preserve-environment=true
EOF
sudo debootstrap --arch=arm64 buster /srv/chroot/pi64
sudo schroot -c pi64 -- apt-get install -y sudo curl mesa-utils xz-utils libqt5widgets5 libevdev2 libudev1 libasound2 libbluetooth3 libopengl0 libglu1-mesa

You've now got a 64-bit chroot called "pi64" to play around with. Run the command:

Code: Select all

schroot -c pi64

and you should end up at a prompt like so:

Code: Select all

(pi64)pi@raspberrypi:~ $

This is your 64-bit chroot and you should take some time to play around with it. You can install aarch64 compilers or other 64-bit applications and run from here. You can exit just like any other shell, and create new instances anytime by running the above schroot command.

Our next roadblock is that because this is a Debian (not Raspbian) chroot, it's using Mesa 19.0 which won't be sufficient. Download and unpack this custom build of 64-bit Mesa 19.3 like so:

Code: Select all

(pi64)pi@raspberrypi:~ $ sudo echo

(pi64)pi@raspberrypi:~ $ curl -Lo- https://www.dropbox.com/s/yjpez224cotk3tj/mesa-v3d_19.3-EXT_buffer_storage_arm64.tar.xz?dl=0 | sudo tar -xJf- -C /

(When copypasting, from hereon please don't include the dollar sign or anything before it. That's there to emphasize that you must run this command inside the chroot.)

Next, check if this fixed GLES 3.0 hardware acceleration by running the command:

Code: Select all

(pi64)pi@raspberrypi:~ $ glxinfo | egrep 'Vendor|Device|GLES\[23\]'

You should get:

Code: Select all

    Vendor: Broadcom (0x14e4)
    Device: V3D 4.2 (0xffffffff)
    Max GLES[23] profile version: 3.0

If you instead see softpipe or a GLES profile version less than 3.0, something is wrong.

Let's get Dolphin. Normally it takes more than 30 minutes to build natively, so I've provided a precompiled binary:

Code: Select all

(pi64)pi@raspberrypi:~ $ curl -L -O https://www.dropbox.com/s/evw12h6q4j0dw7w/dolphin-emu_5.0-10886-230ff47c0a_arm64.tar.xz
...
(pi64)pi@raspberrypi:~ $ sudo tar xvf dolphin-emu_5.0-10886-230ff47c0a_arm64.tar.xz -C /

Now time to tell Dolphin to default to GLES. Open it up!

Code: Select all

(pi64)pi@raspberrypi:~ $ dolphin-emu

Go to Options -> Graphics Settings and select a Backend by setting it to OpenGL, then close the dialog and exit out of Dolphin.

This should create the appropriate configuration files. Find ~/.config/dolphin-emu/GFX.ini and edit it. You'll need to add the following line:

Code: Select all

PreferGLES = True

under the [Settings] group.

Dolphin is now almost ready to run, but if you try opening a ROM it'll immediately hit an error saying "MemoryMap_Setup: Failed finding a memory base." This is due to a shortcoming in debootstrap or schroot that by default shared memory is not available. You can hack around this like so:

Code: Select all

(pi64)pi@raspberrypi:~ $ sudo chmod a+w /dev/shm

It should be ready now. Run Dolphin again i.e.:

Code: Select all

(pi64)pi@raspberrypi:~ $ dolphin-emu

and open the GameCube ROM of your choice.

There, you made it!

Note: If this works the first time but after a reboot it runs into a memory error, that is typically because the /dev/shm and overwriting of Mesa /usr/lib files is not sticky. You'll have to rerun those particular commands to get the chroot back into its ready mode, so if you plan on going back into the emulator often consider putting together a helper script.

Note this whole thing is at the proof-of-concept stage. Turning on antialiasing immediately causes a crash and Wii games are unplayable. I'm sharing this mainly so that all Pi 4 owners can jump in figure out ways to move this forward.

To minimize dependencies, the pre-built binary I provided had minimal add-ons such as ALSA and Bluetooth controller support. I opted not to include AVI recording or PulseAudio. If you'd like to reconfigure your build or hack on the code, here's a rundown of what to do inside your 64-bit chroot:

Code: Select all

uname -m # i.e. double-check that you're inside the chroot. This should output aarch64
sudo apt install -y build-essential git cmake ninja-build ccache qtbase5-dev qtbase5-private-dev libxi-dev libxrandr-dev libudev-dev libevdev-dev libbluetooth-dev libasound2-dev
sudo update-ccache-symlinks
git clone https://github.com/dolphin-emu/dolphin.git
cd dolphin
mkdir build; cd build
PATH=/usr/lib/ccache:$PATH \
    cmake -G Ninja -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_C_FLAGS"-mtune=cortex-a72" -DCMAKE_CXX_FLAGS="-mtune=cortex-a72" ..
ninja -j3 all
sudo ninja install

Ninja and ccache are optional but are there to make the rebuild experience more manageable. C++ code is slow to compile natively, and as the codebase is very configurable you could find yourself rebuilding things largely unchanged multiple times. Another way to speed up the iteration time is to install clang and prefix the cmake command with CC=clang CXX=clang++, and at some point you may consider cross-compiling.

The -j3 option is there to avoid running out of memory or thrashing to swap on a 2 GB Pi 4. Ninja knows the number of cores but doesn't deal with RAM limitations. If you have a 4 GB model then you should be fine letting Ninja take all the cores.

[zRevengee]

this new mesa build will help on gentoo? it now support EXT_buffer_storage?

[jdonald]

The Mesa tarball above works on Gentoo too, but so far I haven't noticed any big differences between 19.3 and 19.1.

I made a separate build to add EXT_buffer_storage. Dolphin crashes when I enable that extension but it's not clear why. See these threads:

• Possible to add GLES EXT_buffer_storage driver support to Videocore VI on the Pi4?
• Dolphin on the pi 4? page 12

In the first thread there are links to Mesa 19.3 builds with and without EXT_buffer_storage, so you can test side-by-side. I posted a backtrace on the Dolphin forums but the devs haven't commented on it yet.

It's easy to confirm that the modified build exposes EXT_buffer_storage because you can run it through glxinfo. It's harder to verify that the extension is implemented correctly so I'm hoping someone knows which tests to run. You might also get further testing EXT_buffer_storage if you know some options to tweak which could avoid the segmentation fault.

[pica200]

Compiled Dolphin to try and see for myself. Compilation took 35 minutes building from a RAM disk. With the GC game i tried it reached only 60% average with ~70% max. htop reveals why. One core is sitting almost always at 99%. So if the AArch64 JIT is not getting miraculously much more efficient this will never reach fullspeed for most games. The missing GLES feature is not going to fix that.

[jdonald]

Thanks for trying this out.

With the GC game i tried it reached only 60% average with ~70% max.
...
So if the AArch64 JIT is not getting miraculously much more efficient this will never reach fullspeed for most games.

Not every game is the same though. If I run around the first level of Super Mario Sunshine with the null renderer it's consistently 100%. I'm not even overclocking, although currently that is the norm for Dolphin Pi4 demos on YouTube.

Note to test CPU emulation + sound without rendering overhead--outside of a title screen--the steps are: save a gameplay state, stop the game, switch to Null renderer under Graphics Options, restart game, load state.

Which GameCube game did you test?

One core is sitting almost always at 99%.

I also see this happening with the video plugin turned on, so video is effectively slowing down everything else. Engines such as Unity prevent graphics driver work from blocking the engine via multithreaded rendering. Dolphin has a flag called BackendMultithreading which is default on, but based on testimony it appears its implementation has shortcomings. That will also be a barrier to getting the full benefits of Vulkan when that becomes available.

On the Dolphin forums thread I asked the devs for clarification on EXT_buffer_storage's rationale and whether it makes as much sense for an embedded GPU, and have yet to hear back. Regardless, agreed that it won't fix everything and it's just one step along the way.

[pica200]

Not every game is the same though. If I run around the first level of Super Mario Sunshine with the null renderer it's consistently 100%. I'm not even overclocking, although currently that is the norm for Dolphin Pi4 demos on YouTube.

Note to test CPU emulation + sound without rendering overhead--outside of a title screen--the steps are: save a gameplay state, stop the game, switch to Null renderer under Graphics Options, restart game, load state.

Which GameCube game did you test?

I'm running this at stock 1.5 GHz clock. Changing the renderer to null and loading the game in the most demanding area (saved there) it gives 16 fps and with GLES 10-11. So clearly the CPU is bottlenecking.

It's Pokémon XD. This game seems have rather low GPU demands but it's absolutely hogging the CPU. I also noticed this has quite some graphics glitches with textures having a "stripe" effect or even parts missing. I'm not seeing these glitches on my Notebook with OpenGL or Vulkan (Intel HD4600).

I also see this happening with the video plugin turned on, so video is effectively slowing down everything else. Engines such as Unity prevent graphics driver work from blocking the engine via multithreaded rendering. Dolphin has a flag called BackendMultithreading which is default on, but based on testimony it appears its implementation has shortcomings. That will also be a barrier to getting the full benefits of Vulkan when that becomes available.

On the Dolphin forums thread I asked the devs for clarification on EXT_buffer_storage's rationale and whether it makes as much sense for an embedded GPU, and have yet to hear back. Regardless, agreed that it won't fix everything and it's just one step along the way.

This is not the case for this game. As mentioned above the null renderer barely does anything and i have another observation confirming this. Doing a ridiculous underclock of the GC CPU to 20% (!) makes this game reach 100% with 30 fps and 60 vfps in quite some areas but it's frameskipping hard in problem areas.

All of this was on Manjaro ARM with GPU driver on and compositing off. Mesa 19.1.6.
Dolphin was compiled as release build with gcc 9.1.0 and "-march=armv8-a+crc -mtune=cortex-a72 -Ofast -flto -ffunction-sections". Linker flags same but "-Wl,--gc-sections" instead of "-ffunction-sections".
Additionally "-CMAKE_POSITION_INDEPENDENT_CODE=off". And no, -Ofast is not the cause of the graphics glitches since i'm getting the same without.

Side note: Without lto this builds in 27 minutes and with 35 minutes. All from the /tmp RAM disk. The linking step takes absolutely ages with lto.


edit:
With arm_freq=2000 + gpu_freq=600 i can underclock the GC CPU to 35% and i'm getting 80-100% in many areas. The problematic area is 15 fps max now.
Having to underclock the CPU that much doesn't bode well for other games. I need to test other games. The CPU bottleneck is probably also a result of the ridiculous decision to go with a 32 bit bus for the RAM. The 3200 MHz is not helping much if you effectively halve the bandwidth and then on top a L2 cache which is a joke. Even the New Nintendo 3DS has 2 MB L2 cache.

[jdonald]

Pokemon XD is an interesting case. I can dial it down to 10% to make the sound smooth and the game remains somewhat playable, in part because it's mostly turn-based. One side effect is that at 10% I'm able to walk through walls in places like behind the elevator.

I'm not familiar with this game so I can't identify the corrupt texture issue, but you can test if the situation improves at all with Mesa 19.3. To use the Mesa libraries I compiled, on Manjaro you can unpack them and run:

Code: Select all

LD_LIBRARY_PATH=path/to/mesa/usr/lib/aarch64-linux-gnu \
LIBGL_DRIVERS_PATH=path/to/mesa/usr/lib/aarch64-linux-gnu/dri \
  dolphin-emu

[pica200]

Good news: Your mesa build works.
Bad news: It doesn't fix anything.

You can see this same stripe effect here. I bet this is a driver bug because Dolphin is motorious for making poor driver implementations visible.


I tried Super Mario Sunshine now and with overclock i can get it to 70-74% on the title screen. As soon as the cutscene ends and we are getting ingame this game absolutely destroys the poor Pi 4 going down to 2 fps and 99% CPU usage on one core. GC CPU underclock doesn't do anything but slows this game down even more. With 35% CPU it even crashed Dolphin once.

[Raspberry Pi Rules!]

Handy guide but I'm not having any luck with actually running games.

OpenGL renderer:

Code: Select all

Segmentation fault

Vulkan renderer:

Code: Select all

Failed to load Vulkan library.
Failed to load Vulkan library.
Failed to initialize video backend!

Software renderer:

Code: Select all

Failed to create OpenGL window
Failed to initialize video backend!

Null renderer:
The game window just contains the bottom-left part of the screen from when the game was opened.

Any ideas? glxinfo gives the same output as in your guide and everything seems like it ought to be dandy, but evidently it isn't.

[jdonald]

The Pi 4 doesn't yet have OpenGL [3.0] or Vulkan, and even if software rendering ran it would be very slow. The only working configuration is OpenGL ES.

Did you edit GFX.ini and add the PreferGLES = True line under [Settings]? Also check if the contents got overwritten?

If nothing obvious, I can provide a debug build with address sanitizer so we can at least identify the line triggering the segmentation fault.

[pica200]

Just recompiled Dolphin from the latest commits. With overclock and lto it took 29 minutes now. In Super Mario Sunshine the title screen reaches ~80 fps now but ingame it's still 2-3 fps. I was running around in the first level a bit watching the slide show ( ) only to be surprised by a nice crash. The entire X server caught on fire and crashed against the wall. Did not notice this previously maybe because i was not playing long enough or i won in the crash lotery The texture glitches are still there.

[jdonald]

You've seen zRevengee's gameplay footage, right? https://www.youtube.com/watch?v=1M-TetK1MtQ at 8:00. It sounds like your experience is much worse and the difference wouldn't be explained by his overclocking alone, so it might be one of the other speed hacks.

For any serious Dolphin development it makes sense to cross-compile to take those 30 minutes down to one minute. On an x86_64 laptop:

Code: Select all

docker run -it debian:buster

Code: Select all

sudo dpkg --add-architecture arm64 && sudo apt update
sudo apt-get install -y build-essential git cmake ninja-build ccache g++-aarch64-linux-gnu qtbase5-dev:arm64 qtbase5-private-dev:arm64 libxrandr-dev:arm64 libudev-dev:arm64 libevdev-dev:arm64 libbluetooth-dev:arm64 libasound2-dev:arm64 libxi-dev:arm64 libxi6:arm64 libegl1-mesa-dev:arm64
cat << EOF > toolchain.cmake
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(CMAKE_CROSSCOMPILING 1)
set(CMAKE_C_COMPILER aarch64-linux-gnu-gcc)
set(CMAKE_CXX_COMPILER aarch64-linux-gnu-g++)
set(ENV{PKG_CONFIG_PATH} /usr/lib/aarch64-linux-gnu/pkgconfig)
EOF

then add -DCMAKE_TOOLCHAIN_FILE=path/to/your/toolchain.cmake when configuring with CMake.

Running sudo update-ccache-symlinks and configuring with PATH=/usr/lib/ccache:$PATH ensures compiler caching too. Note Debian Buster binaries tend to be run-compatible with more bleeding edge distros like Gentoo and Arch.

[pica200]

Yeah, it's near fullspeed for him because he deletes the per game settings (Sys folder) and enables pretty much every speed hack Dolphin offers As for the texture issues maybe it's one of the options he changed fixing them. And the sound is ok here. He probably has an older build of Gentoo without the Pulseaudio fix.

[bomblord]

Yeah, it's near fullspeed for him because he deletes the per game settings (Sys folder) and enables pretty much every speed hack Dolphin offers As for the texture issues maybe it's one of the options he changed fixing them. And the sound is ok here. He probably has an older build of Gentoo without the Pulseaudio fix.

Also of note removing those per-game settings to get a speed up makes the game unbeatable.

[pica200]

I just tried this temporarily renaming the Sys folder and indeed, it suddenly runs around 80% ingame. Could not figure out what fixes the textures though (either it's a driver/mesa bug or it's a Dolphin master branch issue. Did not try 5.0.). It's noteworthy that he still had texture glitches in other games. Just not SMS.

[Raspberry Pi Rules!]

Did you edit GFX.ini and add the PreferGLES = True line under [Settings]? Also check if the contents got overwritten?

Indeed I did and the value is still True. :/

[graphicw]

I was reviewing recent commits on the V3D driver and have noticed a little work that may improve Dolphin performance on RPI 4. Not sure what the Pi 4 will be able to ultimately run once the drivers are mature though. Dolphin is notorious for needing systems with some beefy specs to pull off it's work.

[pica200]

@Raspberry Pi Rules!
Did you follow the guide by the letter? I can't speak for Raspian but Manjaro (using the official testing kernel now) works with Dolphin after compiling it myself. Maybe you are missing some libs the pre-compiled build needs?

[jdonald]

@Raspberry Pi Rules!, I am having a bit of trouble making a working -fsanitize=address build for debugging, so will supply that later. I did run through the binary install guide on a fresh Raspbian Buster installation and it ran Super Mario Sunshine with the latest 4.19.75-v8+ kernel.

I did come across issues such as using wget and curl before installing them, so I have fixed that bug in the first post and the steps now only depend on curl. There was also the issue that if you ran wget | sudo it might initially get stalled asking for a password the first time, so I added a sudo echo step for good measure. If you managed to verify GLES 3.0 support in glxinfo it sounds like you already dealt with those hiccups anyhow.

The build-from-source instructions were only missing the libxi-dev dependency and I have fixed that as well. (You would have noticed if it had encountered this.)

@graphicw I posted binaries for running Dolphin with EXT_buffer_storage / GLES 3.1 in the EXT_buffer_storage topic. I didn't highlight that here initially due to the complication of requiring a custom kernel, but now that the patch has landed I'll put it on the to-do list to get everything on that config. Unfortunately it does not seem to improve rendering performance in any noticeable way.

[Raspberry Pi Rules!]

Yes, I followed through the guide completely. I may try using a fresh installation of Raspbian to try and rule out issues.

I did get Dolphin working in Gentoo, but only when forcing OpenGL software rendering, which wasn't good for performance. The support seems better here, and I don't really know the ins and outs of Gentoo, so this is where I'll focus my efforts.

[jdonald]

Strange, Gentoo should be even easier to run with GLES because it already comes with the correct build of Mesa. My initial guess would be a spacing or other formatting issue in your GFX.ini, but if PreferGLES = True weren't getting picked up you would have seen some obvious dialog errors complaining about a lack of OpenGL 3.0 before the segfault.

I've posted a -DCMAKE_BUILD_TYPE=RelWithDebugInfo (i.e. includes debug symbols) build: https://www.dropbox.com/s/bbt90w0zaychh ... m64.tar.xz

If you run:

Code: Select all

ulimit -c unlimited

then get this dolphin-emu binary to segfault, it'll make a core.* file. You can then analyze the core dump in gdb to get a backtrace, or upload to Dropbox or other fileshare so I can take a look.

Edit: Now that rpi-update provides kernel version 4.19.75+, I have edited the instructions in the first post to use the EXT_buffer_storage build of Mesa.

[Raspberry Pi Rules!]

Sorry for the late reply. I've done your new

Code: Select all

curl -Lo- https://www.dropbox.com/s/yjpez224cotk3tj/mesa-v3d_19.3-EXT_buffer_storage_arm64.tar.xz?dl=0 | sudo tar -xJf- -C /

line and now have a Max GLES[23] profile version of 3.1. I'm also using the build of Dolphin you linked, plus there's been system and EEPROM updates between then and now. I don't know exactly what fixed the OpenGL ES renderer, but things seem to be working! Performance is poor but that's not too surprising considering the hardware. I have the Pi's CPU clocked at 2147MHz and the GPU at 750MHz.

I've only tested it with The Simpsons Road Rage, and what I've quickly noticed are these lines on polygon boundaries. I've not seen this issue when using Dolphin on other hardware. See the screenshot below - in this example, the issue is most visible on the NPC car's windows, those houses and along the tops of the trees. You wouldn't know what might be causing this, would you? No worries if this is just one of those issues you have to deal with due to compatibility though - I'm only setting this up as an experiment rather than to actually play games.

l.png (253.46 KiB) Viewed 40584 times

[pica200]

You are probably seeing the same graphic glitches as in my screenshot above. It's a GPU driver/mesa issue.

On a side note: 750 MHz for the GPU doesn't work. In all tests i have done everything higher than 600 was actually slower. This is probably another limitation like the 2 GHz limit before it got lifted.

[jdonald]

I don't know exactly what fixed the OpenGL ES renderer, but things seem to be working!

Congrats on getting Dolphin to run!

I haven't seen polygon edge lines like that in other games, but there are artifacts like the Super Mario Sunshine rendering issues that pica200 pointed out. Overall there do seem to be plenty of graphical issues specific to V3D but I've considered those moot until we can get many games at playable speeds.

I just read over the blog post from Igalia from a few weeks ago: https://www.raspberrypi.org/blog/vc4-an ... an-update/ It sounds like they're making great progress so at this point I'm just planning to check back every few months and try new builds if users report promising results for OpenArena and other engines.

[Raspberry Pi Rules!]

750 MHz for the GPU doesn't work. In all tests i have done everything higher than 600 was actually slower. This is probably another limitation like the 2 GHz limit before it got lifted.

Thanks for letting me know. I saw the 600MHz restriction mentioned at https://www.raspberrypi.org/documentati ... locking.md but enough articles were using 750MHz that I started to doubt it. Anyway, with the 600MHz setting now... I think performance is a little better? It's hard to tell.

While I'm here, I may as well tell the world that for my specific Pi with the official power supply, 2147MHz is quite stable, but I did have a reboot while web browsing. This was after a good few hours, but still annoying. I might be best stepping it back down. For anything like overnight uploads I put it back to stock clocks anyway.

I haven't seen polygon edge lines like that in other games, but there are artifacts like the Super Mario Sunshine rendering issues that pica200 pointed out. Overall there do seem to be plenty of graphical issues specific to V3D but I've considered those moot until we can get many games at playable speeds.

I just read over the blog post from Igalia from a few weeks ago: https://www.raspberrypi.org/blog/vc4-an ... an-update/ It sounds like they're making great progress so at this point I'm just planning to check back every few months and try new builds if users report promising results for OpenArena and other engines.

Yeah, there's not too much point in fixing graphical issues before performance is usable. I was hoping 50/60fps titles could be run at 25/30fps but it seems the frameskip option has been removed.

That news about graphics drivers progress looks good! I guess with all these things it's a matter of time.