Ubuntu 19.10 for the Raspberry Pi 4 with a KVM-enabled kernel & Windows 10 guest

[gilius2k15]

I managed to build an Ubuntu 19.10 with KVM enabled thanks to satmandu's docker system:
https://github.com/satmandu/docker-rpi4-imagebuilder/

However, the above is a thin deployment system that remains quite *hit and miss* since there's a lot of dependencies involved that must work in tandem for the final image to be created successfully - so expect many failures - and potentially even corrupt LZ4 files (best to choose IMG.XZ output). Nevertheless, it's still a new project and satmandu has been very helpful in trying to rectify any issues based on different environments - all the while updating the guide on the homepage as the project matures.

[Here] I've provided a successful prebuilt IMG.XZ image via the above that you can immediately write to Micro-SD card by first extracting the .IMG file from the IMG.XZ and then writing that using DD:

Code: Select all

sudo dd if=eoan-preinstalled-server-arm64+raspi4-08_09_2019_0152UTC.img of=/dev/mmcblk0 bs=1M

NOTE: the Micro-SD card must first be erased (I use gparted or windows disk management)

Once written remove the SD Card and reinsert it. You can then mount partition 1:

Code: Select all

sudo mkdir /mnt/piboot
sudo mount /dev/mmcblk0p1 /mnt/piboot

You then need to edit the config.txt to enable VC4 graphics acceleration (could also do this using notepad on a windows system):

Code: Select all

sudo nano /mnt/piboot/config.txt

Append this line to the bottom of the file then CTRL+X to save and exit:

Code: Select all

dtoverlay=vc4-fkms-v3d

Boot up in the Pi4 (mini-HDMI plugged in nearest to USB-C) and after a time hit enter to bring up the login screen - but NOT before the cloud-init process has completed, i.e. you must give it a good 5 minutes. Initial login is ubuntu/ubuntu; you will then be prompted to change password.

Once the Terminal prompt appears I found it's better to install both a lightweight and heavyweight desktop environment alongside each other to avoid any bugs with VC4 - particularly when using KVM. If you do get any problems with the login window not appearing you can press Alt+F4 and then sudo startx to prod it into action so to speak. Before logging in ensure you have switched to the lightweight distro via the button at the top-right of the screen; I chose to install the following:

Code: Select all

sudo apt install lxde
sudo apt install ubuntukylin-desktop
sudo reboot

NOTE: LXQT/XFCE will probably work too perhaps alongside ubuntu-desktop even?

Once inside the desktop GUI download these two DEB files [here] and [here] to upgrade to a KVM-enabled kernel with the necessary virtalization software via the following install commands:

Code: Select all

sudo dpkg -i linux-image-5.3.0-rc3-v8-gea837612b_5.3.0-rc3-v8-gea837612b-1_arm64.deb
sudo dpkg -i linux-headers-5.3.0-rc3-v8-gea837612b_5.3.0-rc3-v8-gea837612b-1_arm64.deb
sudo apt install qemu-kvm libvirt-daemon-system
sudo reboot

If there's any interest then in the next part I can describe the steps needed to get a Windows guest running through Qemu/KVM?

[JMK19]

Hello.
I'm just wondering, are you aware of the ongoing project to get Windows 10 ARM running on the Raspberry Pi? If not visit discourse.pi64.win
This would be an awesome other method of getting it running on the thing (although this version would require activation via a key rather than being free, I'm guessing)

Thank you very much for posting this awesome tutorial. I would be very glad if you followed it up with how you got Win10 on the thing.
(P.S. if you follow up, please reply with driver information)

And one more thing: Is it just me or is the download slow for the prebuilt image? If you ever need a file host you could host it up on G Drive or Mega or WeTransfer in case there's too much traffic.

[geev03]

Code: Select all

ubuntu@ubuntu:~$ uname -a
Linux ubuntu 4.19.64-v8-g76b2727ef #1 SMP PREEMPT Wed Aug 7 14:28:51 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@ubuntu:~$ free -h
              total        used        free      shared  buff/cache   available
Mem:          2.8Gi       1.1Gi       1.0Gi        36Mi       668Mi       1.6Gi
Swap:            0B          0B          0B
ubuntu@ubuntu:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            1.4G     0  1.4G   0% /dev
tmpfs           287M  3.3M  284M   2% /run
/dev/mmcblk0p2   30G  6.0G   22G  22% /
tmpfs           1.4G     0  1.4G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           1.4G     0  1.4G   0% /sys/fs/cgroup
/dev/loop1       49M   49M     0 100% /snap/lxd/11444
/dev/loop0       49M   49M     0 100% /snap/lxd/11598
/dev/loop2       79M   79M     0 100% /snap/core/7275
/dev/loop3       79M   79M     0 100% /snap/core/7399
/dev/mmcblk0p1  253M  106M  147M  42% /boot/firmware
tmpfs           287M   20K  287M   1% /run/user/120
tmpfs           287M   52K  287M   1% /run/user/1000
ubuntu@ubuntu:~$ 

[Gavinmc42]

Sakaki has now a 4GB version that has the 3GB issue fixed.
Another few months and most major distributions will be running on the Pi4?

Win10 is interesting, did not expect that

[sparticle]

I have to say I was very excited to see this post. My two favourite things, raspberry pi and Ubuntu. However I cannot get the image to complete a first boot. All looks good until it gets to installing new bcm2711-rpi-4-b.dtb then it hangs and does nothing. I left it for a good half an hour JIC it needed to download something.

Also is it possible to get a 18.04 LTS based image?

Any ideas?

Cheers
Spart

[ubertuber]

Thanks for posting this. However, I am having issues with getting any kvm instance to run. it doesn't seem like kvm is actually built into the kernel that you provided? Can you please post your sources the kernel so I can debug it?

[macromorgan]

I for one an interested in your writeup on how to create the Windows 10 ARM guest. I have my Raspberry Pi 4 (4GB) as a compute node of a 2 machine array for WebVirtCloud. I have Debian ARM running great on it, and want to add Windows ARM to the ARM node.

[gilius2k15]

Running Windows 10 on ARM with KVM acceleration enabled is quite tricky and requires a 64-bit OS with a KVM-enabled kernel (/dev/kvm must exist) - so not sure if the current Pi distros of Debian satisfy those requirements? Best bet is Ubuntu or Gentoo right now.

I'll take you through some of the prerequisites first...

You can install one of the latest versions of Qemu-kvm using a package manager, but it doesn't actually matter if it's version 3 or 4 because the only virtual machine version that works is 2.12 (and below) because of a bug that still hasn't been fixed in maybe 1.5 years.

Code: Select all

sudo apt install qemu-kvm libvirt-daemon-system samba

(The samba is needed if you like to create a shared drive to be accessible from within the guest VM - depends on network access).

You need to pick a folder - say the desktop or downloads folder - and download the following to the same folder:
http://snapshots.linaro.org/components/ ... EMU_EFI.fd
This is the Linaro UEFI bios that exists as a single file - but can be converted to two flash0 and flash1 IMG files if you wanted to save any bios settings - not really necessary though.

https://fedorapeople.org/groups/virt/vi ... noarch.rpm
Above is the latest ARM64 guest drivers for Qemu - comparable to VMware Tools - or VirtualBox guest add-ons, etc. You need to extract the ISO from the RPM file using an archive manager, such as 7-Zip.

Besides the drivers ISO you need to download and build up a Windows 10 on ARM ISO from one of the two main UUP dump sites (best done from an actual windows command line instead of linux terminal):
https://uup.rg-adguard.net/index.php or https://uupdump.ml/

Only newer versions work with the official Qemu:
Insider 18348.190226-1407 (1st March 2019) or newer i.e. feature upgrade 1903 (April/May 2019) or the latest Insider is recommended.
https://blogs.windows.com/windowsexperi ... ild-18348/
https://www.theregister.co.uk/2019/03/1 ... 10_on_kvm/

If you wish to run an earlier build of Windows 10 then you should use driver1998's Qemu fork - but the CPU speed may be reduced based on how many cores you include in your VM script:

Code: Select all

git clone https://github.com/driver1998/qemu
sudo apt install glib2.0 libpixman-1-dev gcc libsdl2-dev bison flex qemu-kvm libvirt-daemon-system
cd qemu
git checkout woa-kvm
mkdir build
cd build
sudo ../configure --target-list=aarch64-softmmu --disable-werror
sudo make
sudo make install

Lastly, you need to create a VHD or VHDX for your VM - about 13 GB minimum:

Code: Select all

qemu-img create -f vhdx -o subformat=fixed system.vhdx 13G

Ensure the above are all contained in the same folder then you just need to create an SH script to run the VM:
virtio-win-0.1.171.ISO
18362.1.190318-1202.19H1_RELEASE_CLIENTPRO_OEMRET_ARM64FRE_EN-US.ISO
QEMU_EFI.fd
system.vhdx
script.sh - blank for now - see bottom

BTW, if you want to run this on Windows 10 Intel host, without KVM ,in software simulation mode only then you need to install the following and then put the same files listed above under c:\program files\qemu (or move the qemu folder to c:\qemu after installing it):
https://qemu.weilnetz.de/w64/
(and Instead of script.sh it would be named script.bat)

So you see there are two main modes for running a Qemu guest VM:
Software simulation mode
near-native hardware acceleration mode (KVM) - Intel guest on Intel host - or in this case ARM64 guest on ARM64 host.

Software Simulation mode is generally more stable, but runs extremely SLOW - recommended to use an Intel host to try to build up/test/update your VHDX-based VM before copying across to the Pi as it would be quite painful to install the Windows.ISO to the system.vhdx using the Pi in sim mode and take about 3 hours. I can do it in about 30 minutes using an i7-8700K

KVM mode currently has problems with Ethernet drivers because the VirtIO NetKVM driver hasn't been fixed yet - but there are some workarounds.

The script requires about 15 lines to run the VM. Let's start with the first few lines - some subtle differences exit between sim vs. KVM modes:

Software Simulation Mode - Linux

Code: Select all

sudo qemu-system-aarch64 \
-cpu cortex-a57 \
-accel tcg,thread=multi

Software Simulation Mode - Windows

Code: Select all

qemu-system-aarch64.exe ^
-cpu cortex-a57 ^
-accel tcg,thread=multi

KVM mode

Code: Select all

sudo qemu-system-aarch64 \
-cpu host \
-enable-kvm

For sim mode you could choose cortex-a53 instead of a57 if you wish. The rest of the script is the same for both modes:

Code: Select all

-M virt-2.12 \
-smp 2 \
-m 1.5G \
-bios QEMU_EFI.fd \
-device ramfb \
-device usb-kbd \
-device usb-mouse \
-device usb-tablet \

You must use virt-2.12 or maybe 2.11 or 2.10, etc. You cannot use just -virt (latest version) or even version 3 - due to a bug.
The smp value is the number of CPU cores - can affect CPU speed with driver1998's qemu fork for some reason. I think with the Pi3-4 you should be safe with 2-3. -smp 4 seems to work too - but 1 core is meant to be reserved for the host.
The -m value represents RAM. With the Pi4 you could increase to 2 GB - but 1.5 is a more safe bet.
The bios we spoke about already...
Ramfb allows you to display the guest in a GUI window. In the past, -device VGA was used in Intel contexts - but with ARM KVM there's a bug with the framebuffer so ramfb is used instead. Should you manage to get network/RDP working later (with auto sign in) then you could use the -nographics option and just RDP to the guest.
USB devices must be added, such as keyboard and mouse. I think the tablet helps with cursor problems.

You must choose a USB controller - there are about 5 available - but the most widely used one is ich9:

Code: Select all

-device ich9-usb-ehci1 \

The CD-ROM is quite straightforward to add:

Code: Select all

-device usb-storage,drive=install \
-drive if=none,id=install,format=raw,media=cdrom,file=18362.1.190318-1202.19H1_RELEASE_CLIENTPRO_OEMRET_ARM64FRE_EN-US.ISO \

NOTE: If you find the CD-ROM doesn't boot for whatever reason (old Windows Server 2016 on ARM or perhaps really new Windows 10 Insiders) then you can extract the contents to a FAT32 VHDX and use that as a source instead (see below for adding VHDX to the script)

To add the VirtIO ISO:

Code: Select all

-device usb-storage,drive=drivers \
-drive if=none,id=drivers,media=cdrom,file=virtio-win-0.1.171.iso \

There's two ways of adding the main hard drive - as a standard USB or VirtIO block device - both require the VioStor drivers generally:

Code: Select all

-hda system.vhdx \

OR

Code: Select all

-device usb-storage,drive=system \
-drive if=none,id=system,format=raw,file=system.vhdx \

OR

Code: Select all

-device virtio-blk,drive=system \
-drive if=none,id=system,format=raw,file=system.vhdx \

This line is only for running old betas of Windows Server 2016 on ARM:

Code: Select all

-rtc base="2016-04-20",clock=vm

Enabling network access will cause crashes and unidentified network with KVM enabled - otherwise can be used in software simulation mode with RDP connections enabled for 127.0.0.1:5555 and a shared samba drive:

Code: Select all

-device virtio-net,netdev=net0 \
-netdev user,id=net0,hostfwd=tcp::5555-:3389,smb=/home/ubuntu/Desktop

You have to allow blank passwords over RDP via secpol.msc:
Open Local Policies -> Security Options
Change Accounts: Limit local account use of blank passwords to console from Enabled to Disabled
This will allow auto sign in, etc. Or see this page:
https://support.microsoft.com/en-gb/hel ... in-windows

Note: Qemu guest graphics acceleration does not yet exist for ARM64 Windows by the Fedora Project (driver is named QXL) - but if you RDP you should get a better graphical experience than ramfb.

The workarounds that exist for network access inside KVM (I haven't tried them):
https://github.com/virtio-win/kvm-guest ... -522130741
https://github.com/virtio-win/kvm-guest ... -468149012

BTW, Server 2016 betas are a real nightmare to get working since the VioStor drivers only work on Win 10 and not Win Server, so simply extracting the installer ISO to a FAT32 VHD is not enough - you need to use DISM to apply the actual WIM image from the ISO sources folder to a VHDX:

Code: Select all

diskpart
list disk
select disk x (after creating/attaching VHDX in disk management)
clean
convert gpt
create partition efi size=124
format fs=fat32
assign letter s
create partition primary size=13000
format fs=ntfs quick
assign letter w
exit
dism /apply-image /imagefile:c:\wim\install.wim /index:1 /applydir:W:
bcdboot W:\Windows /s s: /f UEFI
bcdedit /store s:\EFI\Microsoft\Boot\bcd /set {default} testsigning on
bcdedit /store s:\EFI\Microsoft\Boot\bcd /set {default} nointegritychecks on

Note: the last 2 lines may not be needed when it comes to the latest drivers.

Anyway, so here's the final script for Windows 10:

Software Simulation Mode

Code: Select all

sudo qemu-system-aarch64 \
-accel tcg,thread=multi \
-M virt-2.12 \
-smp 2 \
-m 1.5G \
-cpu cortex-a53 \
-bios QEMU_EFI.fd \
-device ramfb \
-device ich9-usb-ehci1 \
-device usb-kbd \
-device usb-mouse \
-device usb-tablet \
-device usb-storage,drive=install \
-drive if=none,id=install,format=raw,media=cdrom,file=18362.1.190318-1202.19H1_RELEASE_CLIENTPRO_OEMRET_ARM64FRE_EN-US.ISO \
-device usb-storage,drive=drivers \
-drive if=none,id=drivers,media=cdrom,file=virtio-win-0.1.171.iso \
-device virtio-net,netdev=net0 \
-netdev user,id=net0,hostfwd=tcp::5555-:3389,smb=/home/ubuntu/Desktop \
-hda system.vhdx

KVM

Code: Select all

sudo qemu-system-aarch64 \
-M virt-2.12 \
-smp 2 \
-m 1.5G \
-cpu host \ 
-enable-kvm \
-bios QEMU_EFI.fd \
-device ramfb \
-device ich9-usb-ehci1 \
-device usb-kbd \
-device usb-mouse \
-device usb-tablet \
-hda system.vhdx

Again, you add a NIC and RDP if you get that far by appending the following:

Code: Select all

-device virtio-net,netdev=net0 \
-netdev user,id=net0,hostfwd=tcp::5555-:3389,smb=/home/ubuntu/Desktop

Consider a -nographic option instead of ramfb is fully working with network, RDP and auto sign-in enabled.

Consider a block device too for the main hard drive - original source here using an NEC USB bus instead - but it's up to you:
https://withinrafael.com/2018/02/11/boo ... 0-in-qemu/

When you first start the script it will load the bios then attempt to boot from CD-ROM:
press any key for cd...
Leave default keyboard
install now
skip product key
accept terms
custom: install windows only (advanced)
load driver > browse > CD virtio > viostor > w10 > ARM64
load driver > browse > CD virtio > NetKVM > w10 > ARM64
Select system hard drive > Next to install

[macromorgan]

Windows 10 doesn't seem to be working for me, but that's perhaps because I am on 18.04 instead of 19.10 (I get a failure as soon as the ISO starts loading files).

I'll try updating my QEMU and libvirt later and see what that does.

[gilius2k15]

Windows 10 doesn't seem to be working for me, but that's perhaps because I am on 18.04 instead of 19.10 (I get a failure as soon as the ISO starts loading files).

I'll try updating my QEMU and libvirt later and see what that does.

Is /dev/kvm present? Did you follow the above guide? What's the error message?

[macromorgan]

It is present, and I have a KVM instance of Debian running.

Honestly aside from the steps to pull the ISO I don't think I followed any of your guide; I'm trying to get it installed through libvirt is the issue since I'm running headless and managing through WebVirtCloud. Additionally I am running on Ubuntu 18.04 rather than 19.10, so it's likely that some of my packages aren't up to the task (QEMU and Libvirt might be too old).

I have my Raspberry Pi 4 set up as a compute node in WebVirtCloud along with an AMD64 system so I can virtualize an OS on either aarch64 or amd64 (as well as the 32 bit versions of arm and x86). So far I only have Debian Buster running on the aarch64 node.

[emilfr]

I have to say I was very excited to see this post. My two favourite things, raspberry pi and Ubuntu. However I cannot get the image to complete a first boot. All looks good until it gets to installing new bcm2711-rpi-4-b.dtb then it hangs and does nothing. I left it for a good half an hour JIC it needed to download something.

Also is it possible to get a 18.04 LTS based image?

Any ideas?

Cheers
Spart

If you're still looking for 18.04 LTS image for RPI4

https://jamesachambers.com/raspberry-pi ... ion-guide/

[macromorgan]

Is /dev/kvm present? Did you follow the above guide? What's the error message?

I'm getting a Windows error 0xc0000225 when I attempt to boot. I've tried 2 different ISOs at this point. Is there a very specific one you think I should try?

I've tested with QEMU 2.11 and 4.1. I can't follow your guide to the T though as I'm attempting to do it through libvirt (thinks like ramfb don't work through libvirt).

[gilius2k15]

It is present, and I have a KVM instance of Debian running.

Honestly aside from the steps to pull the ISO I don't think I followed any of your guide; I'm trying to get it installed through libvirt is the issue since I'm running headless and managing through WebVirtCloud. Additionally I am running on Ubuntu 18.04 rather than 19.10, so it's likely that some of my packages aren't up to the task (QEMU and Libvirt might be too old).

I have my Raspberry Pi 4 set up as a compute node in WebVirtCloud along with an AMD64 system so I can virtualize an OS on either aarch64 or amd64 (as well as the 32 bit versions of arm and x86). So far I only have Debian Buster running on the aarch64 node.

Your setup sounds very unorthodox. This needs a 64-bit kernel and userland; you sound like you are running some kind of hybrid setup.

You just need a clean install of Ubuntu or Gentoo 64-bit with KVM; where's the Debian counterpart for that? With Gentoo you just have to run the following:

Code: Select all

emerge -av app-emulation/libvirt
emerge -av app-emulation/qemu

but qemu can take a while to build via portage - but there might be a way to select only aarch64 as a target.

You can then run the script providing all the files are in place.

[macromorgan]

It is present, and I have a KVM instance of Debian running.

Honestly aside from the steps to pull the ISO I don't think I followed any of your guide; I'm trying to get it installed through libvirt is the issue since I'm running headless and managing through WebVirtCloud. Additionally I am running on Ubuntu 18.04 rather than 19.10, so it's likely that some of my packages aren't up to the task (QEMU and Libvirt might be too old).

I have my Raspberry Pi 4 set up as a compute node in WebVirtCloud along with an AMD64 system so I can virtualize an OS on either aarch64 or amd64 (as well as the 32 bit versions of arm and x86). So far I only have Debian Buster running on the aarch64 node.

Your setup sounds very unorthodox. This needs a 64-bit kernel and userland; you sound like you are running some kind of hybrid setup.

You just need a clean install of Ubuntu or Gentoo 64-bit with KVM; where's the Debian counterpart for that? With Gentoo you just have to run the following:

Code: Select all

emerge -av app-emulation/libvirt
emerge -av app-emulation/qemu

but qemu can take a while to build via portage - but there might be a way to select only aarch64 as a target.

You can then run the script providing all the files are in place.

I'm running a full 64bit kernel/userspace (with multiarch 32 bit stuff in place) with KVM enabled, it's a stock 18.04.2 Ubuntu image with sakaki's kernel with KVM enabled. QEMU is either 2.1.1 (64 bit) or a hand compiled 4.1.0 (also 64 bit). I think the biggest issue is that I'm trying to do it through libvirt instead of through QEMU directly (in the past this has caused other issues too, namely with PPC hosts). I'm curious if you are able to get your Windows 10 ARM to work in virt-manager?

[gilius2k15]

Oh I see... You need to test in qemu first before you consider moving to some next level hypervisor setup. I tried virt-manager before but didn't work for me in this context; ARM64/KVM is still quite bleeding edge, and the drivers are not yet in a mature nor complete state, hence is only a hacker's tool right now. So you need to use qemu proper with a GUI, though there is some way of using the VNC built into Qemu but I not looked into it. RDP is for sure the way to go - but getting to that stage with KVM enabled is really tedious at this point in time.

BTW, instead of system.vhdx you can build up a windows inside a partition on the memory card. I'm sure there's a lot more interesting things that can be achieved, but this is only an introduction.

[macromorgan]

Confirmed it's an issue trying to create the instance via libvirt. When I launch from the command line and VNC into the QEMU instance it seems to be installing at least, which is progress.

edit: run directly from QEMU (via VNC) works successfully. It's only when I attempt to do the same via libvirt does it fail. I guess I'll try and figure out why eventually, for now I won't worry about it.

[gilius2k15]

BTW, here's the absolute quickest way of getting Windows 10 on ARM ISO to the system.vhdx using Windows Disk Management (for creating/attaching VHDX) and Windows command prompt:

create c:\wim and c:\mount and c:\drivers (latter containing the viostor and netkvm VirtIO drivers extracted from the VirtIO ISO):
12/04/2019 17:11 9,708 netkvm.cat
12/04/2019 12:46 16,292 netkvm.inf
12/04/2019 12:46 1,282,048 netkvm.pdb
12/04/2019 17:11 146,632 netkvm.sys
12/04/2019 17:13 1,072,840 netkvmco.dll
12/04/2019 12:46 12,161,024 netkvmco.pdb
12/04/2019 17:12 9,716 viostor.cat
12/04/2019 12:46 2,933 viostor.inf
12/04/2019 12:46 634,880 viostor.pdb
12/04/2019 17:12 65,736 viostor.sys

Extract Windows on ARM ISO using 7-Zip (or mount it) and copy the install.wim from the source folder to c:\wim

Code: Select all

diskpart
list disk
select disk x (after creating/attaching VHDX in disk management)
clean
convert gpt
create partition efi size=124
format fs=fat32
assign letter s
create partition primary
format fs=ntfs quick
assign letter w
exit
dism /mount-image /imagefile:c:\wim\install.wim /Index:1 /MountDir:c:\mount
dism /image:c:\mount /add-driver /driver:c:\drivers /recurse /forceunsigned
dism /unmount-wim /mountdir:c:\mount /commit
dism /apply-image /imagefile:c:\wim\install.wim /index:1 /applydir:W:
bcdboot W:\Windows /s s: /f UEFI
bcdedit /store s:\EFI\Microsoft\Boot\bcd /set {default} testsigning on
bcdedit /store s:\EFI\Microsoft\Boot\bcd /set {default} nointegritychecks on
bcdedit /store s:\EFI\Microsoft\Boot\bcd /dbgsettings NET HOSTIP:<ip address of the machine where you run windbg> PORT:50000 KEY:4.3.2.1
bcdedit /store s:\EFI\Microsoft\Boot\bcd /set {default} debug on
bcdedit /store s:\EFI\Microsoft\Boot\bcd /set {default} bootdebug on

[gilius2k15]

I had some problems with ich9 USB controller in software simulation mode for 1709 and 1803 - complained about source files - so I used NEC instead. I might need to update this guide sometime after doing some more testing.
https://en.wikibooks.org/wiki/QEMU/Devices/USB/Root

[gilius2k15]

I got 1709 to boot up at full CPU speed with 3GB of RAM on the Pi4 using Gentoo and driver1998's qemu fork:

1. Software simulation mode (fast Intel host) - cannot be ich9 usb controller - here using nec instead:

Code: Select all

qemu-system-aarch64.exe ^
-accel tcg,thread=multi ^
-M virt-2.12 ^
-smp 2 ^
-m 3G ^
-cpu cortex-a53 ^
-bios QEMU_EFI.fd ^
-device ramfb ^
-device nec-usb-xhci ^
-device usb-kbd ^
-device usb-mouse ^
-device usb-tablet ^
-device usb-storage,drive=install ^
-drive if=none,id=install,format=raw,media=cdrom,file=16299.15.170928-1534.RS3_RELEASE_CLIENTPRO_OEMRET_ARM64FRE_EN-GB.ISO ^
-device usb-storage,drive=drivers ^
-drive if=none,id=drivers,media=cdrom,file=virtio-win-0.1.171.iso ^
-device virtio-blk,drive=system ^
-drive if=none,id=system,format=raw,file=1709.vhd

2. After reboot continue in software simulation mode to create offline user account till the desktop appears then shut down.

3. Copy the 1709.vhd to the Pi4 and boot up with the following script in KVM mode:


To build up the qemu fork in Gentoo required several "emerges":

Code: Select all

sudo emerge -av app-emulation/libvirt
sudo emerge -av app-emulation/qemu
sudo emerge -av dev-libs/glib
sudo emerge -av media-libs/libsdl2
sudo emerge -av sys-devel/gcc
sudo emerge -av sys-devel/bison
sudo emerge -av sys-devel/flex
git clone https://github.com/driver1998/qemu
cd qemu
git checkout woa-kvm
mkdir build
cd build
sudo ../configure --target-list=aarch64-softmmu --disable-werror
sudo make
sudo make install

Above will probably work for 1803 and 1809 too.

EDIT: keyboard not working with NEC controller under KVM - so switched to qemu-xhci just for KVM

[gilius2k15]

The latest Insider (and other recent Windows 10 releases after 1903) are now working with network access!

NOTE: These newer releases of Windows do not currently work in Software Simulation Mode - but can be built up quite quickly in the latest versions of QEMU using KVM all the way (or part DISM). Remember: we do not need driver1998's fork here because that's only for the older versions of Windows 10 before it was made "KVM friendly".

So here's how I built the very latest Insider that came out only today (I think!):

1. Windows Command Prompt (ideally on an Intel host)

create c:\wim and c:\mount and c:\drivers (latter containing the viostor drivers only - extracted from the VirtIO ISO):
12/04/2019 17:12 9,716 viostor.cat
12/04/2019 12:46 2,933 viostor.inf
12/04/2019 12:46 634,880 viostor.pdb
12/04/2019 17:12 65,736 viostor.sys

Extract Windows on ARM ISO using 7-Zip (or mount it) and copy the install.wim from the source folder to c:\wim

Code: Select all

diskpart
list disk
select disk x (after creating/attaching insider.vhdx in Disk Management)
clean
convert gpt
create partition efi size=124
format fs=fat32
assign letter s
create partition primary
format fs=ntfs quick
assign letter w
exit
dism /mount-image /imagefile:c:\wim\install.wim /Index:1 /MountDir:c:\mount
dism /image:c:\mount /add-driver /driver:c:\drivers /recurse /forceunsigned
dism /unmount-wim /mountdir:c:\mount /commit
dism /apply-image /imagefile:c:\wim\install.wim /index:1 /applydir:W:
bcdboot W:\Windows /s s: /f UEFI
bcdedit /store s:\EFI\Microsoft\Boot\bcd /set {default} testsigning on
bcdedit /store s:\EFI\Microsoft\Boot\bcd /dbgsettings NET HOSTIP:<ip address of the machine where you run windbg> PORT:50000 KEY:4.3.2.1
bcdedit /store s:\EFI\Microsoft\Boot\bcd /set {default} debug on
bcdedit /store s:\EFI\Microsoft\Boot\bcd /set {default} bootdebug on

2. Detach insider.vhdx from Disk Management and copy over to the Pi4 then continue the Windows To Go installation process - entirely in KVM:

Code: Select all

sudo qemu-system-aarch64 \
-M virt-2.12 \
-smp 2 \
-m 3G \
-cpu host \ 
-enable-kvm \
-bios QEMU_EFI.fd \
-device ramfb \
-device qemu-xhci \
-device usb-kbd \
-device usb-mouse \
-device usb-tablet \
-hda insider.vhdx \
-netdev user,id=net0 -device virtio-net,disable-legacy=on,netdev=net0

Note: the guy from Microsoft/Azure also suggested using this alternative NIC device (works in debugging mode only as per virtio-net):

Code: Select all

-netdev user,id=net0 -device e1000,netdev=net0

And now that we have network access you could probably go one step further and setup RDP and a Samba share (see earlier instructions):

Code: Select all

-device virtio-net,disable-legacy=on,netdev=net0 \
-netdev user,id=net0,hostfwd=tcp::5555-:3389,smb=/home/demouser/Desktop

3. You can speed up windows 10 using the following scripts:
Run batch script as admin to disable superfetch, etc. (https://gist.github.com/BenNottelling/9 ... 01c5f5e455)
powershell -executionPolicy bypass -file decrapifier.ps1 -ClearStart -appaccess (see: https://community.spiceworks.com/script ... -1803-1809)
Adjust windows 10 for best performance
Disable windows defender

If you really want to fine tune Windows further - down to using a similar amount of RAM to Linux distros then see this old topic of mine:
https://discourse.pi64.win/t/optimizing ... ance/178/4
However, if with about 3.2 GB of RAM available you shouldn't need to go further than the scripts and disabling defender.

[gilius2k15]

Here I enabled RDP via Remote Settings as well as allowed blank passwords via local security policy and then replaced -device ramfb with -nographic - before remotely connecting to the Pi4 KVM guest via the network from my Intel PC.... PERFECT!!!!!

[ender`]

I installed Windows 10 (18362.10019) in KVM without problems, however I couldn't get virtio-net-pci to work - loading the netkvm driver always BSODs Windows for me. Luckily, I have a USB 3 hub with integrated network card, which has inbox drivers, so I could use that to get online.

[Gavinmc42]

Starting to make me wonder if someone at Microsoft has managed to get some Pi4's.
Is RDP fast enough for some older games?

This has convinced me VNC is now really worth looking at.
A Pi4 with two screens could control quite a few remote machines.
Gentoo64 is very easy to scroll through the default 4 desktops, too easy sometimes
That's 8 screens/RDP/VNC sessions

Now you are throwing KVM into the mix

[pooky]

Does anyone have compiled KVM enabled kernel for official Ubuntu 19.10 img ?

Current Ubuntu 19.10 kernel is 5.3.0-1008-raspi2 (Linux ubuntu 5.3.0-1008-raspi2 #9-Ubuntu SMP Fri Oct 18 13:26:35 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux)