Sharing my virtualization configurations and howtos here with others in case anybody would find them usable.
Currently a full 64bit support for KVM is only possible with Manjaro Linux (https://wiki.pine64.org/index.php/Pinebook_Pro_Software_Release#Manjaro_ARM) which I run off a microSD card (faster the better, use minimum of 100 mb/s read and 64 GB or more in size)
We are going to use mostly QEMU with KVM support for running the native arm64 systems, HYPERION-HERCULES for running mainframe simulator for z/OS or MVS and lastly the SIMH machine simulator for running VAX systems (OpenVMS and UNIX)
Qemu
git clone https://github.com/qemu/qemu.git cd qemu ./configure make -j5 su # make install
Hyperion – Hercules
git clone https://github.com/SDL-Hercules-390/hyperion git clone https://github.com/SDL-Hercules-390/crypto git clone https://github.com/SDL-Hercules-390/decNumber git clone https://github.com/SDL-Hercules-390/SoftFloat git clone https://github.com/SDL-Hercules-390/telnet
We first need to build crypto, decnumber, softfloat and telnet
mkdir crypto64.Release cd crypto64.Release cmake ../crypto make
cd .. mkdir decNumber64.Release cd decNumber64.Release cmake ../decNumber make
cd .. mkdir SoftFloat64.Release cd SoftFloat64.Release cmake ../SoftFloat make
cd .. mkdir telnet64.Release cd telnet64.Release cmake ../telnet make
Now we need to copy the compiled aarch64 libs to the proper hyperion directory
cd .. mkdir hyperion/telnet/lib/aarch64 mkdir hyperion/SoftFloat/lib/aarch64 mkdir hyperion/decNumber/lib/aarch64 mkdir hyperion/crypto/lib/aarch64 cp crypto64.Release/libcrypto64.a hyperion/crypto/lib/aarch64 cp decNumber64.Release/libdecNumber64.a hyperion/decNumber/lib/aarch64 cp SoftFloat64.Release/libsSoftFloat64.a hyperion/SoftFloat/lib/aarch64 cp telnet64.Release/libtelnet64.a hyperion/telnet/lib/aarch64
cd hyperion ./configure make -j4 su # make install
Next we need to compile x3270 terminal emulator
cd .. wget http://x3270.bgp.nu/download/03.06/suite3270-3.6ga8-src.tgz gunzip suite3270-3.6ga8-src.tgz tar -xvf suite3270-3.6ga8-src.tar cd suite3270-3.6 ./configure
Will fail because we have an old config.guess .. download a recent one from GCC
rm config.guess wget https://raw.githubusercontent.com/gcc-mirror/gcc/master/config.guess ./configure make -j4 # make install
SIMH
git clone https://github.com/simh/simh cd simh make -j5
All simh binaries will be in simh/BIN directory
Preparing the Pinebook-Pro Manjaro Linux for virtualization
Manjaro aarch64 kernel is compiled with KVM support so we can take full advantage of /dev/kvm acceleration via qemu-system-aarch64 or qemu-system-arm. Once we have the hypervizor and simulators compiled, here are some examples how I handle networking in the VMs
For the networking on the Arch Linux we need to download uml_utilities and compile them, since they do not ship on the Manjaro Linux (arch) since we need the tunctl command from there to setup the network
$ wget http://user-mode-linux.sourceforge.net/uml_utilities_20070815.tar.bz2
$ bunzip uml_utilities_20070815.tar.bz2
$ tar -xvf uml_utilities_20070815.tar
$ cd tools-20070815
$ make
The make will error on uml_net/hoct.c:215 undefined reference to ` makedev` But this does not really matter, the tunctl binary will be built which we can copy over to /usr/bin like this $ su # cd tunclt # cp tunctl /usr/bin
Also we would need to install the bridge-utils which are installable via pacman package manager like this
# pacman -S bridge-utils
Once we have all the above in place we can prepare the network sharing script for Qemu, just make sure we execute this after each system boot
#Setup tap and bridge #change the username below to match yours tunctl -t tap0 -u user ifconfig tap0 up brctl addbr br0 brctl setfd br0 0 ifconfig br0 10.0.2.2 netmask 255.255.255.0 broadcast 10.0.2.255 up brctl addif br0 tap0 ifconfig tap0 0.0.0.0 sysctl net.ipv4.ip_forward=1 iptables --table nat -A POSTROUTING --out-interface wlan0 -j MASQUERADE
Also make sure you have this script as /etc/qemu-ifup and executable
#! /bin/sh
# Script to bring a network (tap) device for qemu up.
# The idea is to add the tap device to the same bridge
# as we have default routing to.
# in order to be able to find brctl
PATH=$PATH:/sbin:/usr/sbin
ip=$(which ip)
if [ -n "$ip" ]; then
ip link set "$1" up
else
brctl=$(which brctl)
if [ ! "$ip" -o ! "$brctl" ]; then
echo "W: $0: not doing any bridge processing: neither ip nor brctl utility not found" >&2
exit 0
fi
ifconfig "$1" 0.0.0.0 up
fi
switch=$(ip route ls |
awk '/^default / {
for(i=0;i<NF;i++) { if ($i == "dev") { print $(i+1); next; } }
}'
)
switch=br0
# only add the interface to default-route bridge if we
# have such interface (with default route) and if that
# interface is actually a bridge.
# It is possible to have several default routes too
for br in $switch; do
if [ -d /sys/class/net/$br/bridge/. ]; then
if [ -n "$ip" ]; then
ip link set "$1" master "$br"
else
brctl addif $br "$1"
fi
exit # exit with status of the previous command
fi
done
echo "W: $0: no bridge for guest interface found" >&2
Running Virtual Machines on the PinebookPro
Here are some examples of installer/loader scripts of various systems I have used for KVM accelerated qemu
NetBSD -current arm64 ( http://www.invisible.ca/arm/)
wget https://releases.linaro.org/components/kernel/uefi-linaro/latest/release/qemu64/QEMU_EFI.fd/usr/local/bin/qemu-system-aarch64 \ -cpu host -M virt,accel=kvm -smp 2 \ -m 512 -nographic \ -bios QEMU_EFI.fd \ -drive file=netbsd.img,if=none,id=drive0,format=raw \ -device virtio-blk-device,drive=drive0 \ -serial tcp::4441,server,telnet,wait \ -net nic -net tap
Connect to the remote machine qemu console via telnet 4441
FreeBSD -current arm64 (http://ftp.freebsd.org/pub/FreeBSD/snapshots/arm64/aarch64/ISO-IMAGES/13.0/)
wget https://releases.linaro.org/components/kernel/uefi-linaro/latest/release/qemu64/QEMU_EFI.fd./qemu-system-aarch64 \ -cpu host -M virt,accel=kvm \ -m 2048 -nographic \ -bios QEMU_EFI.fd \ -drive file=freebsd.img,format=raw,id=drive1 \ -serial tcp::4441,server,telnet,wait \ -net nic -net tap
Connect to the remote machine qemu console via telnet 4441
CentOS 8 arm64 (http://isoredirect.centos.org/centos/8/isos/aarch64/)
wget https://releases.linaro.org/components/kernel/uefi-linaro/latest/release/qemu64/QEMU_EFI.fdInstall.sh
qemu-system-aarch64 \ -cpu host -M virt,accel=kvm -m 1024 -nographic \ -drive if=pflash,format=raw,file=QEMU_EFI.img \ -drive if=pflash,file=varstore.img \ -drive if=virtio,file=disk.img \ -drive if=virtio,format=raw,file=centos8.iso
Run.sh
qemu-system-aarch64 \ -cpu host -M virt,accel=kvm -m 1024 -nographic \ -drive if=pflash,format=raw,file=QEMU_EFI.img \ -drive if=pflash,file=varstore.img \ -drive if=virtio,file=disk.img \ -net nic -net tap
Windows 10 arm64
Getting the Win10 arm64 ISO https://www.woaproject.net/viewtopic.php?t=3
Then extract the zip file with the aria
mv 19541.1000_arm64_en-us_professional_7da33442_convert.zip ARIA/ unzip 19541.1000_arm64_en-us_professional_7da33442_convert.zip Archive: 19541.1000_arm64_en-us_professional_7da33442_convert.zip inflating: aria2_download_windows.cmd inflating: aria2_download_linux.sh inflating: ConvertConfig.ini extracting: files/convert_config_linux inflating: files/aria2c.exe inflating: files/convert.sh inflating: files/convert_ve_plugin inflating: files/7zr.exe inflating: files/uup-converter-wimlib.7z chmod +x aria2_download_linux.sh ./aria2_download_linux.sh this will take some time (faster on SSD drivers ofcourse) Total translation table size: 2048 Total rockridge attributes bytes: 0 Total directory bytes: 0 Path table size(bytes): 10 Max brk space used d5000 1854834 extents written (3622 MB)
Once downloaded rename it to win10.iso
mv 19541.1000_PROFESSIONAL_ARM64_EN-US.ISO win10.iso
Create a Win10 working directory and create empty disk image to hold the installation
wget https://releases.linaro.org/components/kernel/uefi-linaro/latest/release/qemu64/QEMU_EFI.fdwget https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.173-2/virtio-win-0.1.173.iso
qemu-img create -f qcow2 disk.img 20G
Install.sh
#!/bin/bash export QEMU_AUDIO_DRV=pa qemu-system-aarch64 \ -cpu host \ -enable-kvm \ -M virt-2.12 \ -smp 2 \ -m 2G \ -bios QEMU_EFI.img \ -device ramfb \ -device ich9-usb-ehci1 \ -device usb-kbd \ -device usb-mouse \ -device usb-tablet \ -device usb-storage,drive=windows \ -drive if=none,id=windows,media=cdrom,file=win10.iso \ -device usb-storage,drive=drivers \ -drive if=none,id=drivers,media=cdrom,file=virtio-win-0.1.173.iso \ -device virtio-blk,drive=system \ -drive if=none,id=system,format=qcow2,file=disk.img
Networking works best with a physical USB 1GB realtek dongle identified by Linux as productid=0x8153. During the installation make sure to load the virtio-win-0.1.173.iso drivers so that the disk.img gets detected by the installer !
Run.sh
#!/bin/bash export QEMU_AUDIO_DRV=pa qemu-system-aarch64 \ -cpu host \ -enable-kvm \ -M virt-2.12 \ -smp 2 \ -m 2G \ -bios QEMU_EFI.img \ -device ramfb \ -device ich9-usb-ehci1 \ -device usb-kbd \ -device usb-mouse \ -device usb-tablet \ -device usb-storage,drive=drivers \ -drive if=none,id=drivers,media=cdrom,file=virtio-win-0.1.173.iso \ -device virtio-blk,drive=system \ -drive if=none,id=system,format=qcow2,file=disk.img -vnc :1 \ -device usb-ehci,id=ehci -device usb-host,bus=ehci.0,vendorid=0x0bda,productid=0x8153
And use VNC connector to localhost :1 to finish the setup, please note that the virtio network drivers are causing BSOD on Win10 arm64, so its better to use a real USB ethernet dongle on the Pinebook-pro woth the Realtek chip identified as 0x8153, once you have the network up in the VM, use RemoteDesktop or freerdp2 to connect to Windows10 over the network for best performance.
NON aarch64 systems in simulators on the PinebookPro
Here are example loaders and installers for non arm/aarch64 systems which are fun to play around with (and of course you can run this on more powerful x86_64 platform ) I just wanted to show how powerful the Pinebookpro actually is
AmigaOS 4.1 qemu-system-ppc (search for Sam460InstallCD-53.58.iso on the internet)
qemu-system-ppc -machine sam460ex -rtc base=localtime -drive if=none,id=cd,file=./Sam460InstallCD-53.58.iso,format=raw -device ide-cd,drive=cd,bus=ide.1 -drive file=./disk.img,format=raw,id=disk -device ide-hd,drive=disk,bus=ide.0 -netdev user,id=mynet0 -net nic,model=rtl8139 -net tap
MorphOS 3.12 qemu-system-ppc
wget http://www.morphos-team.net/morphos-3.12.iso
Install.sh
qemu-system-ppc -machine mac99,via=pmu -m 512 \ -vga none -device sm501 \ -cdrom morphos-3.12.iso -boot d \ -hda disk.img \ -prom-env "boot-device=cd:,\mac_ppc32\boot.img" \ -bios openbios-qemu.elf -serial stdio \ -netdev user,id=mynet0 -device sungem,netdev=mynet0 -net nic -net tap
Run.sh
qemu-system-ppc -machine mac99,via=pmu -m 512 \ -vga none -device sm501 \ -cdrom morphos-3.12.iso -boot c \ -hda disk.img \ -prom-env "boot-device=hd:,boot.img" \ -bios openbios-qemu.elf -serial stdio \ -netdev user,id=mynet0 -device sungem,netdev=mynet0 -net nic -net tap
AIX 7.2 qemu-system-ppc64 (IBM prorietary OS)
You can get the System Diagnostics live CD from https://worthdoingbadly.com/aixqemu/
Or if you have AIX 7.2 you can use it onwards from Technology Level 3 and Service Pack 1 onwards in qemu-system-ppc64 since IBM added virtio-scsi support to the AIX kernel
Install.sh
qemu-system-ppc64 -cpu POWER8 -machine pseries -m 2048 -serial stdio -drive file=disk.img,if=none,id=drive-virtio-disk0 -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=drive-virtio-disk0 -cdrom aix.iso -prom-env "boot-command=dev / 0 0 s\" ibm,aix-diagnostics\" property boot cdrom:\ppc\chrp\bootfile.exe -s verbose" -net nic -net tap -display vnc=:1
Run.sh
qemu-system-ppc64 -cpu POWER8 -machine pseries -m 2048 -serial telnet::4441,server -drive file=disk.img,if=none,id=drive-virtio-disk0 -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=drive-virtio-disk0 -cdrom AIX71/cd1.iso -prom-env boot-command='boot disk: ' -net nic -net tap -vga none -nographic
Overall performance is not bad, bear in mind that 2 GB RAM is minimum to run AIX 7.2 under qemu-system-ppc64, it takes roughly about 15 minutes to boot on the Pinebook-pro and is quite usable afterwards.
HERCULES – HYPERION
Mainframe z/OS 1.10 (strictly experimental)
hercules.cnf
ARCHMODE z/Arch ALRF ENABLE cckd RA=2,RAQ=4,RAT=2,WR=2,GCINT=5,GCPARM=0,NOSTRESS=0,TRACE=0,FREEPEND=-1 CNSLPORT 3270 CONKPALV (3,1,10) CPUMODEL 3090 CPUSERIAL 012345 DIAG8CMD ENABLE ECPSVM YES LOADPARM 0A95DB.. LPARNAME HERCULES MAINSIZE 1024 MOUNTED_TAPE_REINIT DISALLOW NUMCPU 6 OSTAILOR Z/OS PANRATE 80 PGMPRDOS LICENSED SHCMDOPT NODIAG8 SYSEPOCH 1900 TIMERINT 50 TZOFFSET +1400 YROFFSET 0 HERCPRIO 0 TODPRIO -20 DEVPRIO 8 CPUPRIO 0 # Display Terminals 0700 3270 0701 3270 0702 3270 0703 3270 0704 3270 #******************************************************************** # CTCI COMMUNICATION DEVICES * #******************************************************************** 0E20.2 CTCI 192.168.11.101 192.168.11.102 # DASD Devices 0A81 3390 cckd/zares1.cckd sf=shadow/zares1_* 0A82 3390 cckd/zares2.cckd sf=shadow/zares2_* 0A83 3390 cckd/zadb81.cckd sf=shadow/zadb81_* 0A84 3390 cckd/zadb82.cckd sf=shadow/zadb82_* 0A85 3390 cckd/zadb83.cckd sf=shadow/zadb83_* 0A86 3390 cckd/zadb84.cckd sf=shadow/zadb84_* 0A87 3390 cckd/zadb91.cckd sf=shadow/zadb91_* 0A88 3390 cckd/zadb92.cckd sf=shadow/zadb92_* 0A89 3390 cckd/zacic1.cckd sf=shadow/zacic1_* 0A8A 3390 cckd/zadis1.cckd sf=shadow/zadis1_* 0A8B 3390 cckd/zadis2.cckd sf=shadow/zadis2_* 0A8C 3390 cckd/zadis3.cckd sf=shadow/zadis3_* 0A8D 3390 cckd/zadis4.cckd sf=shadow/zadis4_* 0A8E 3390 cckd/zadis5.cckd sf=shadow/zadis5_* 0A8F 3390 cckd/zadis6.cckd sf=shadow/zadis6_* 0A90 3390 cckd/zaims1.cckd sf=shadow/zaims1_* 0A91 3390 cckd/zaprd1.cckd sf=shadow/zaprd1_* 0A92 3390 cckd/zaprd2.cckd sf=shadow/zaprd2_* 0A93 3390 cckd/zaprd3.cckd sf=shadow/zaprd3_* 0A94 3390 cckd/zaprd4.cckd sf=shadow/zaprd4_* 0A95 3390 cckd/zasys1.cckd sf=shadow/zasys1_* 0A96 3390 cckd/zauss1.cckd sf=shadow/zauss1_* 0A97 3390 cckd/zawas1.cckd sf=shadow/zawas1_* 0A98 3390 cckd/zawas2.cckd sf=shadow/zawas2_* 0A99 3390 cckd/zawas3.cckd sf=shadow/zawas3_* 0A9A 3390 cckd/sares1.cckd sf=shadow/sares1_*
Execute Hercules
/usr/local/bin/hercules -f hercules.cnf
Open 2 x3270 terminals and connect them both to localhost so that Hercules logo shows up , below are some basic commands to run the simulator and the system, Im not goint to cover the z/OS here in deeper detail.
IPL the 0E20 device in Hercules by first doing
Press ESC to go to IPL menu in HerculesShift+l h During first run we need to initialize the system like so R 00,I Wait till you get either the TSO login or fix the TSO logon by running the following in the primary console window R 01,RETRY or R 02,RETRY
The network is tricky to get configured and the whole Mainframe simulation stuff needs a separate blog entry I believe, hopefully if I have time I will create a specific article about exactly that.
But at the end our Pinebook-pro runs Mainframe quite well under hercules :)