Armbian guys are great in what they do with the sunxi64 kernel for the Pinebook. I have been experimenting with the Pinebook for 2 months now and so far I have used the native Ubuntu 18.04 aarch64 distro, NetBSD -current from Jared McNeill (http://www.invisible.ca/arm/) and the Armbian for Pinebook (https://www.armbian.com/pinebook-a64/)
I have so far best experience with the Armbian pinebook-a64 Linux distribution and here I would like to describe the process how to get full KVM support in the Armbian Bionic based on the 4.19.x kernel
Unfortunately the current 4.19.2-sunxi64 kernel does not have KVM virtualisation support enabled so any attempt on running aarch64 virtual machines in QEMU on the Pinebook will be horribly slow.
Installation of the Armbian Stretch 4.19.x distribution is straight forward, and I have used the image that I have transferred to a microSD card 64BG (100 mb/s as minimum speed). All this can be easily done on the current Linux machine via following command
x86_64 LAPTOP
$ wget https://dl.armbian.com/pinebook-a64/Debian_stretch_next.7z
$ su
# dd if=Armbian_5.69_Pinebook-a64_Debian_stretch_next_4.19.13.img of=/dev/sd(X) <- check what your USB microSD adaper is
Once the image is transferred, please follow the official documentation how to setup Networking and all other things, which I’m not going to cover here. It is straight forward.
So once we have the Pinebook booted and connected to the internet, make sure you get all packages updated
aarch64 PINEBOOK
# apt-get update
# apt-get upgrade
we can go on and install latest Qemu, so for this we need to enable the testing repository in /etc/apt/sources.list
aarch64 PINEBOOK
root@pinebook:/etc/apt# cat sources.list
deb http://httpredir.debian.org/debian testing main contrib non-free
#deb-src http://httpredir.debian.org/debian stretch main contrib non-free
deb http://httpredir.debian.org/debian stretch-updates main contrib non-free
#deb-src http://httpredir.debian.org/debian stretch-updates main contrib non-free
deb http://httpredir.debian.org/debian stretch-backports main contrib non-free
#deb-src http://httpredir.debian.org/debian stretch-backports main contrib non-free
deb http://security.debian.org/ stretch/updates testing main contrib non-free
#deb-src http://security.debian.org/ stretch/updates main contrib non-free
deb http://httpredir.debian.org/debian jessie main contrib non-free
So next we can install qemu-system which will pull all softmmu suport (x86,sparc,ppc,mips,arm) but for accelerated qemu emulation we obviously need only the qemu-system-arm (this includes both qemu-system-arm and qemu-system-aarch64) We will also need to install qemu-utils and qemu-efi-aarch64
aarch64 PINEBOOK
# apt-get install qemu-system qemu-efi-aarch64
So now we are ready to build our customized Armbian Stretch kernel 4.19.x based on mainline, since the stock one does not include KVM support.
So for this I have followed this guide https://docs.armbian.com/Developer-Guide_Build-Preparation/
What we will need is the following:
I have used the following config on my Linux Mint 19.1 – make sure you create a large enough VDI disk to hold all the build files, minimum is 20 GB, 50 GB is recommended.
![]()
Also Im using the following host-only-adapter network setup so that I can access the VM from my host and vice-versa easily via ssh/scp
![]()
For this to work I have a following network configuration script I run after I start VirtualBox manager (to get the vboxnet0 interface registered first)
x86_64 LAPTOP
root@panasonic:/home/user# cat networking.sh
#Setup tap and bridge
tunctl -t tap0 -u user
ifconfig tap0 up
brctl addbr br0
brctl addif br0 vboxnet0
brctl setfd br0 0
ifconfig br0 10.0.2.2 netmask 255.255.255.0 broadcast 10.0.2.255 up
brctl addif br0 tap0 vboxnet0
ifconfig tap0 0.0.0.0
sysctl net.ipv4.ip_forward=1
iptables --table nat -A POSTROUTING --out-interface wlan0 -j MASQUERADE
Internal IP address for the Guest VMs are always static in the following format
IP: 10.0.2.10-200
Gateway: 10.0.2.2 DNS: 8.8.8.8 or 1.1.1.1
Setup the OpenSSH server and configure the network on the Guest Ubuntu VM accordingly.
So once we have everything ready we can start preparing the Guest Ubuntu VM for the kernel build
Login, elevate to root and run
VirtualBox Ubuntu GUEST
# apt-get -y -qq install git
# git clone
# cd build
![]()
And run the compile.sh script
VirtualBox Ubuntu GUEST
# ./compile.sh
Select to build U-boot and Kernel packages
![]()
Choose to show Kernel options so that we can enable the KVM support
![]()
Choose Pinebook-a64
![]()
After a while you should come to the Kernel menuconfig
![]()
Make sure to set the following Virtualisation option on (space bar)
![]()
And the following subsection should be enabled as built-on and modules
![]()
After some time the kenrel packages would be made available in the following directory which we will scp out of the VirtualBox VM and transfer to the Pinebook via scp
![]()
![]()
Back on the Pinebook we need to install these packages via apt
aarch64 PINEBOOK
-rw-r--r-- 1 user user 44152 Jan 11 23:36 linux-dtb-next-sunxi64_5.70_arm64.deb
-rw-r--r-- 1 user user 10422904 Jan 11 23:35 linux-headers-next-sunxi64_5.70_arm64.deb
-rw-r--r-- 1 user user 16571572 Jan 11 23:35 linux-image-next-sunxi64_5.70_arm64.deb
-rw-r--r-- 1 user user 320580692 Jan 11 23:36 linux-source-next-sunxi64_5.70_all.deb
-rw-r--r-- 1 user user 246032 Jan 11 23:35 linux-u-boot-next-pinebook-a64_5.70_arm64.deb
# dpkg -i linux-dtb-next-sunxi64_5.70_arm64.deb
# dpkg -i linux-headers-next-sunxi64_5.70_arm64.deb
# dpkg -i linux-image-next-sunxi64_5.70_arm64.deb
Reboot the Pinebook to load the new KVM enabled Kernel, once up we should verify we are running the new Kernel
aarch64 PINEBOOK
root@pinebook:/home/user# uname -a
Linux pinebook 4.19.13-sunxi64 #5.70 SMP Sat Jan 12 00:10:12 CET 2019 aarch64 GNU/Linux
root@pinebook:/home/user#
Next we prepare the virtualisation environment where we want to run Centos 7.6
Before we actually load QEMU on the Pinebook we need to modify slightly the stock /etc/qemu-ifup script as follows
#! /bin/sh
# Script to bring a network (tap) device for qemu up.
# The idea is to add the tap device to the same bridge
# as we have default routing to.
# in order to be able to find brctl
PATH=$PATH:/sbin:/usr/sbin
ip=$(which ip)
if [ -n "$ip" ]; then
ip link set "$1" up
else
brctl=$(which brctl)
if [ ! "$ip" -o ! "$brctl" ]; then
echo "W: $0: not doing any bridge processing: neither ip nor brctl utility not found" >&2
exit 0
fi
ifconfig "$1" 0.0.0.0 up
fi
switch=$(ip route ls |
awk '/^default / {
for(i=0;i<NF;i++) { if ($i == "dev") { print $(i+1); next; } }
}'
)
switch=br0
# only add the interface to default-route bridge if we
# have such interface (with default route) and if that
# interface is actually a bridge.
# It is possible to have several default routes too
for br in $switch; do
if [ -d /sys/class/net/$br/bridge/. ]; then
if [ -n "$ip" ]; then
ip link set "$1" master "$br"
else
brctl addif $br "$1"
fi
exit # exit with status of the previous command
fi
done
echo "W: $0: no bridge for guest interface found" >&2
And prepare the following “network” helper script to enable internet connection sharing bethween Host and Quest on the Pinebook
You will however need to install some tools first for this to work
aarch64 PINEBOOK
# apt-get install uml-utilities
# apt-get install bridge-utils
And then finally save below shell script, make it executable and run it
#Setup tap and bridge
tunctl -t tap0 -u user
ifconfig tap0 up
brctl addbr br0
brctl setfd br0 0
ifconfig br0 10.0.2.2 netmask 255.255.255.0 broadcast 10.0.2.255 up
brctl addif br0 tap0
ifconfig tap0 0.0.0.0
sysctl net.ipv4.ip_forward=1
iptables --table nat -A POSTROUTING --out-interface wlan0 -j MASQUERADE
Load the above script each time you boot the Pinebook if you wish to run Qemu.
On the Pinebook prepare a directory to hold some files, we will be using the linaro UEFI image
aarch64 PINEBOOK
$ mkdir -p ~/KVM/centos
$ cd ~/KVM/centos
$ wget http://mirror.vpsnet.com/centos-altarch/7.6.1810/isos/aarch64/CentOS-7-aarch64-Minimal-1810.iso
$ mv CentOS-7-aarch64-Minimal-1810.iso centos.iso
$ wget http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/latest/QEMU-AARCH64/RELEASE_GCC5/QEMU_EFI.img.gz
$ gunzip QEMU_EFI.img.gz
$ qemu-img create -f qcow2 centos.img 50G
$ qemu-img create -f qcow2 varstore.img 64M
And next we prepare the qemu loader shell script for this simulation in the KVM/centos working directory and make it executable
/usr/bin/qemu-system-aarch64 \
-cpu host -M virt,accel=kvm -m 1024 -nographic \
-drive if=pflash,format=raw,file=QEMU_EFI.img \
-drive if=pflash,file=varstore.img \
-drive if=virtio,file=disk.img \
-drive if=virtio,format=raw,file=centos.iso \
-net nic -net tap
This is it, and you should be able to fully emulate with almost native speed CentOS 7.6 aarch64 on the Debian 9.6 aarch64 Pinebook
Configuration of CentOS is pretty standard.
FreeBSD 13 aarch64
The above method also works for FreeBSD 13 aarch64 ( http://ftp.freebsd.org/pub/FreeBSD/snapshots/arm64/aarch64/ISO-IMAGES/13.0/FreeBSD-13.0-CURRENT-arm64-aarch64-PINEBOOK-20190110-r342911.img.xz ) except the qemu loader is slightly different
/usr/bin/qemu-system-aarch64 \
-cpu host -M virt,accel=kvm -m 1024 -nographic \
-drive if=pflash,format=raw,file=QEMU_EFI.img \
-drive if=pflash,file=varstore.img \
-drive if=virtio,file=freebsd.img \
-net nic -net tap
Where the freebsd.img is the downloaded img file from the above URL
P.S to save up some space on the default image rm -rf /usr/lib/debug
To use secondary disk vtblk1 use this loader script (and create disk2.img qcow2 image first)
/usr/bin/qemu-system-aarch64 \
-cpu host -M virt,accel=kvm -m 1024 -nographic \
-drive if=pflash,format=raw,file=QEMU_EFI.img \
-drive if=pflash,file=varstore.img \
-drive if=virtio,bus=0,unit=0,format=raw,file=freebsd.img \
-drive if=virtio,bus=0,unit=1,format=raw,file=disk2.img \
-net nic -net tap
Here is a dmesg from FreeBSD 13 aarch64 running under KVM qemu-system-aarch64 on Debian 9.6 aarch64 Pinebook
root@pinebook:/home/user/KVM/FreeBSD# ssh 10.0.2.11
Password for root@freebsd:
Last login: Sun Jan 13 22:56:36 2019
FreeBSD 13.0-CURRENT r342911 GENERIC
Welcome to FreeBSD!
root@freebsd:~ # dmesg
---<<BOOT>>---
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2019 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.0-CURRENT r342911 GENERIC arm64
FreeBSD clang version 7.0.1 (tags/RELEASE_701/final 349250) (based on LLVM 7.0.1)
WARNING: WITNESS option enabled, expect reduced performance.
VT: init without driver.
KLD file umodem.ko is missing dependencies
FreeBSD/SMP: Multiprocessor System Detected: 1 CPUs
random: unblocking device.
random: entropy device external interface
MAP 785d0000 mode 2 pages 64
MAP 78610000 mode 2 pages 160
MAP 786b0000 mode 2 pages 160
MAP 7bc20000 mode 2 pages 400
MAP 7bdb0000 mode 2 pages 592
MAP 4000000 mode 0 pages 16384
MAP 9010000 mode 0 pages 1
kbd0 at kbdmux0
efirtc0: <EFI Realtime Clock>
efirtc0: registered as a time-of-day clock, resolution 1.000000s
acpi0: <BOCHS BXPCFACP>
acpi0: Power Button (fixed)
acpi0: Sleep Button (fixed)
psci0: <ARM Power State Co-ordination Interface Driver> on acpi0
gic0: <ARM Generic Interrupt Controller> iomem 0x8000000-0x8000fff,0x8010000-0x8010fff on acpi0
gic0: pn 0x2, arch 0x2, rev 0x1, implementer 0x43b irqs 288
gic0: frame: 0 8020000 1 64 80
gicv2m0: <ARM Generic Interrupt Controller MSI/MSIX> mem 0x8020000-0x8020fff on gic0
generic_timer0: <ARM Generic Timer> irq 34,35,36 on acpi0
Timecounter "ARM MPCore Timecounter" frequency 24000000 Hz quality 1000
Event timer "ARM MPCore Eventtimer" frequency 24000000 Hz quality 1000
cpu0: <ACPI CPU> on acpi0
uart0: <PrimeCell UART (PL011)> iomem 0x9000000-0x9000fff irq 0 on acpi0
uart0: console (9600,n,8,1)
virtio_mmio0: <VirtIO MMIO adapter> iomem 0xa000000-0xa0001ff irq 1 on acpi0
virtio_mmio1: <VirtIO MMIO adapter> iomem 0xa000200-0xa0003ff irq 2 on acpi0
virtio_mmio2: <VirtIO MMIO adapter> iomem 0xa000400-0xa0005ff irq 3 on acpi0
virtio_mmio3: <VirtIO MMIO adapter> iomem 0xa000600-0xa0007ff irq 4 on acpi0
virtio_mmio4: <VirtIO MMIO adapter> iomem 0xa000800-0xa0009ff irq 5 on acpi0
virtio_mmio5: <VirtIO MMIO adapter> iomem 0xa000a00-0xa000bff irq 6 on acpi0
virtio_mmio6: <VirtIO MMIO adapter> iomem 0xa000c00-0xa000dff irq 7 on acpi0
virtio_mmio7: <VirtIO MMIO adapter> iomem 0xa000e00-0xa000fff irq 8 on acpi0
virtio_mmio8: <VirtIO MMIO adapter> iomem 0xa001000-0xa0011ff irq 9 on acpi0
virtio_mmio9: <VirtIO MMIO adapter> iomem 0xa001200-0xa0013ff irq 10 on acpi0
virtio_mmio10: <VirtIO MMIO adapter> iomem 0xa001400-0xa0015ff irq 11 on acpi0
virtio_mmio11: <VirtIO MMIO adapter> iomem 0xa001600-0xa0017ff irq 12 on acpi0
virtio_mmio12: <VirtIO MMIO adapter> iomem 0xa001800-0xa0019ff irq 13 on acpi0
virtio_mmio13: <VirtIO MMIO adapter> iomem 0xa001a00-0xa001bff irq 14 on acpi0
virtio_mmio14: <VirtIO MMIO adapter> iomem 0xa001c00-0xa001dff irq 15 on acpi0
virtio_mmio15: <VirtIO MMIO adapter> iomem 0xa001e00-0xa001fff irq 16 on acpi0
virtio_mmio16: <VirtIO MMIO adapter> iomem 0xa002000-0xa0021ff irq 17 on acpi0
virtio_mmio17: <VirtIO MMIO adapter> iomem 0xa002200-0xa0023ff irq 18 on acpi0
virtio_mmio18: <VirtIO MMIO adapter> iomem 0xa002400-0xa0025ff irq 19 on acpi0
virtio_mmio19: <VirtIO MMIO adapter> iomem 0xa002600-0xa0027ff irq 20 on acpi0
virtio_mmio20: <VirtIO MMIO adapter> iomem 0xa002800-0xa0029ff irq 21 on acpi0
virtio_mmio21: <VirtIO MMIO adapter> iomem 0xa002a00-0xa002bff irq 22 on acpi0
virtio_mmio22: <VirtIO MMIO adapter> iomem 0xa002c00-0xa002dff irq 23 on acpi0
virtio_mmio23: <VirtIO MMIO adapter> iomem 0xa002e00-0xa002fff irq 24 on acpi0
virtio_mmio24: <VirtIO MMIO adapter> iomem 0xa003000-0xa0031ff irq 25 on acpi0
virtio_mmio25: <VirtIO MMIO adapter> iomem 0xa003200-0xa0033ff irq 26 on acpi0
virtio_mmio26: <VirtIO MMIO adapter> iomem 0xa003400-0xa0035ff irq 27 on acpi0
virtio_mmio27: <VirtIO MMIO adapter> iomem 0xa003600-0xa0037ff irq 28 on acpi0
virtio_mmio28: <VirtIO MMIO adapter> iomem 0xa003800-0xa0039ff irq 29 on acpi0
virtio_mmio29: <VirtIO MMIO adapter> iomem 0xa003a00-0xa003bff irq 30 on acpi0
virtio_mmio30: <VirtIO MMIO adapter> iomem 0xa003c00-0xa003dff irq 31 on acpi0
virtio_mmio31: <VirtIO MMIO adapter> iomem 0xa003e00-0xa003fff irq 32 on acpi0
pcib0: <Generic PCI host controller> on acpi0
pci0: <PCI bus> on pcib0
pcib0: pci_host_generic_core_alloc_resource FAIL: type=4, rid=16, start=0000000000000080, end=000000000000009f, count=0000000000000020, flags=1400
pcib0: pci_host_generic_core_alloc_resource FAIL: type=4, rid=16, start=0000000000000040, end=000000000000007f, count=0000000000000040, flags=1800
virtio_pci0: <VirtIO PCI Network adapter> mem 0x10042000-0x10042fff,0x8000000000-0x8000003fff at device 1.0 on pci0
vtnet0: <VirtIO Networking Adapter> on virtio_pci0
vtnet0: Ethernet address: 52:54:00:12:34:56
virtio_pci1: <VirtIO PCI Block adapter> mem 0x10041000-0x10041fff,0x8000004000-0x8000007fff at device 2.0 on pci0
vtblk0: <VirtIO Block Adapter> on virtio_pci1
vtblk0: 2560MB (5242880 512 byte sectors)
virtio_pci2: <VirtIO PCI Block adapter> mem 0x10040000-0x10040fff,0x8000008000-0x800000bfff at device 3.0 on pci0
vtblk1: <VirtIO Block Adapter> on virtio_pci2
vtblk1: 0MB (385 512 byte sectors)
acpi_button0: <Power Button> on acpi0
acpi0: Could not update all GPEs: AE_NOT_CONFIGURED
cryptosoft0: <software crypto>
Timecounters tick every 1.000 msec
usb_needs_explore_all: no devclass
CPU 0: ARM Cortex-A53 r0p4 affinity: 0
Instruction Set Attributes 0 = <AES+PMULL,SHA1,SHA2,CRC32>
Instruction Set Attributes 1 = <>
Processor Features 0 = <AdvSIMD,Float,EL3 32,EL2 32,EL1 32,EL0 32>
Processor Features 1 = <0>
Memory Model Features 0 = <4k Granule,64k Granule,MixedEndian,S/NS Mem,16bit ASID,1TB PA>
Memory Model Features 1 = <>
Memory Model Features 2 = <32b CCIDX,48b VA>
Debug Features 0 = <2 CTX Breakpoints,4 Watchpoints,6 Breakpoints,PMUv3,Debug v8>
Debug Features 1 = <0>
Auxiliary Features 0 = <0>
Auxiliary Features 1 = <0>
Trying to mount root from ufs:/dev/ufs/rootfs [rw]...
WARNING: WITNESS option enabled, expect reduced performance.
lo0: link state changed to UP
vtnet0: link state changed to UP
I have decided to cover my experience with NetBSD -current (thanks to Jared McNiell http://www.invisible.ca/arm/) which now runs very well off the microSD cards. Only drawback is that currently you need to use an external USB wireless interface, since the native is not supported by NetBSD yet.
Once in the Hercules prompt we do the following:
It is a good idea to read some documentation first before using the MVS :)
