Arch Linux VFIO 2019

It’s good to be a guest, but being home is better

 Posted on October 12, 2019  |  13 minutes (2587 words)

An incredibly opinionated attempt at updating the Arch Linux VFIO install notes for late 2019 from a disparate collection of sources.

We strongly discourage blindly following this document. References have been provided if you are unsure what you are about to do.

Background

Ryzen2 3900X
Intel. That’s¹ the² joke³.
12 cores, 24 threads and 64MB L3 cache? What’s not to like?
NOTE: Ryzen2 suffered a showstopper that was fixed in AGESA 1.0.0.3ABA

X570
Too many bugs, keep an eye on BIOS updates and take backups. Annoyingly, settings are not recommended to be loaded between versions FFS

RX 570 / 5700XT
The pace of amdgpu, vulkan and dxvk development has made team green a laughing stock - more so since Pascal and later require signed drivers(pour one out for the nouveau homies).

systemd-boot
cause fuck grub

btrfs
Btrfs has been usable in raid0 for a few years as long as you aren’t storing DBs/VMs or expecting reasonable performance with quotas/qgroups enabled.
Obligatory old man warning included for the haters.
still waiting for kent to clean up those empty commit msgs before the end of the year lulz

xfs
solid and performant for mixed media and qcow2 images

vfio
cuz dual booting is so 1995 (s/o crash override)

NOTE: Linux kernel 5.3-rc2 has both¹ patches² for kvm regression while 5.2.5 only has one.

UPDATE: 5.2.14 works a charm

First Things First

BIOS

Update BIOS

We recommend finding the overclocking community for your board, they always have the hottest pro-tips, eg:

BIOS Settings

  • This will vary by board manufacturer, they are all confusing piles of trash

    Tweaker -> Advanced CPU Settings -> SVM Mode -> Enabled
Settings -> Miscellaneous -> IOMMU -> Enabled

Boot Installer

Download from www.archlinux.org/download/

  • Keymap:

    loadkeys us

  • Start network:

    dhcpcd

    or

    wifi-menu

Partitions & Filesystems

  • To encrypt the root partition, make sure to use the GPT partition table type instead of the default MSDOS type. Otherwise the GRUB2 boot loader may not have enough space for the second stage loader.

Initial Partitioning

  • Start gdisk:

    gdisk /dev/sda

  • EFI:

    n (Add a new partition)
Partition number 1
First sector 2048 (default)
Last sector +512M
Hex code EF00

  • Btrfs:

    n (Add a new partition)
Partition number 2
First sector 2099200 (default)
Last sector (press Enter to use remaining disk)
Hex code 8300

  • Format EFI Partition

    mkfs.fat -F32 -n EFI /dev/sda1

Encrypt System Parition

  • Check algo performance:

    cryptsetup benchmark

  • Encrypt the parition:

    cryptsetup luksFormat --align-payload=8192 -c aes-xts-plain64 -s 512 -h sha512 --use-random /dev/sda2

  • Open the encrypted partition:

    cryptsetup --allow-discards --persistent open /dev/sda2 buttah

Format System Partition

  • Make the filesystem:

    mkfs.btrfs -L btrfs /dev/mapper/buttah

  • Create sub-vols:

    mount /dev/mapper/buttah /mnt
btrfs subvolume create /mnt/@
btrfs subvolume create /mnt/@home
btrfs subvolume create /mnt/@swap
btrfs subvolume create /mnt/@snapshots
btrfs subvolume create /mnt/@var

  • Mount sub-vols:
    Avoid compress in 2019 (still dreaming of zstd:1)
    ALSO avoid: discard, space_cache=v2 (sigh)

    umount /mnt
mount -o subvol=@,ssd,compress=no,space_cache,noatime /dev/mapper/buttah /mnt
mkdir /mnt/{home,swap,.snapshots,var}
mount -o subvol=@home,ssd,compress=no,space_cache,noatime /dev/mapper/buttah /mnt/home
mount -o subvol=@swap,ssd,compress=no,space_cache,noatime /dev/mapper/buttah /mnt/swap
mount -o subvol=@snapshots,ssd,compress=no,space_cache,noatime /dev/mapper/buttah /mnt/.snapshots
mount -o subvol=@var,ssd,compress=no,space_cache,noatime /dev/mapper/buttah /mnt/var

  • Nested sub-vols:
    Creating nested subvolumes excludes these paths from top-level snapshots

    btrfs subvolume create /mnt/var/cache/pacman/pkg
btrfs subvolume create /mnt/var/tmp
btrfs subvolume create /mnt/srv

  • Create subvols after the fact
    See: https://www.spinics.net/lists/linux-btrfs/msg33258.html
    Some paths you probably want to avoid:

    /home/user/.cache/yay
/home/user/.local/share/Trash
/home/user/.local/share/Steam
/home/user/.ccache

Swap

  • From kernel 5.0+ btrfs natively supports swap files

    (must be fully allocated as NOCOW with no compression on one device)

    truncate -s 0 /mnt/swap/swapfile
chattr +C /mnt/swap/swapfile
dd if=/dev/zero of=/mnt/swap/swapfile bs=1M count=16384 status=progress
chmod 600 /mnt/swap/swapfile
mkswap /mnt/swap/swapfile
swapon /mnt/swap/swapfile

Boot Config

  • Mount boot parition

    mkdir /mnt/boot
mount /dev/sda1 /mnt/boot

  • Generate fstab

    mkdir /mnt/etc
genfstab -U /mnt > /mnt/etc/fstab

Auto-mount LUKS+xfs

  • First things first:

    gdisk /dev/sdd
Command (? for help): n
Partition number (1-128, default 1): 
First sector (34-15628053134, default = 2048) or {+-}size{KMGTP}: 
Last sector (2048-15628053134, default = 15628053134) or {+-}size{KMGTP}: 
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): 8e00
Changed type of partition to 'Linux LVM'

lvmdiskscan

pvcreate /dev/sdd1
vgcreate vault /dev/sdd1
lvcreate -l 100%FREE vault -n mydata /dev/sdd1

vgdisplay
lvdisplay

cryptsetup luksFormat --align-payload=8192 -c aes-xts-plain64 -s 512 -h sha512 --use-random /dev/mapper/vault-mydata  
cryptsetup luksOpen /dev/mapper/vault-mydata pool

dd if=/dev/urandom bs=1024 count=512 of=/etc/luks_vault.key
chmod 400 /etc/luks_vault.key
cryptsetup luksAddKey /dev/mapper/vault-mydata /etc/luks_vault.key
cryptsetup luksDump /dev/mapper/vault-mydata

lsblk -o name,uuid

  • vim /etc/crypttab:

    vault           UUID=<BLKID>    /etc/luks_vault.key

  • vim /etc/fstab:

    /dev/mapper/mydata    /mnt/data    xfs         rw,nodev,noexec,nosuid                                                  0 0 

mkfs.xfs /dev/mapper/mydata
mkdir /mnt/data
mount /dev/mapper/mydata /mnt/data

df /mnt/data

Bonus! Access veracrypt container with cryptsetup

  • mounting:

    mkdir /mnt/crypt
cryptsetup --type tcrypt --veracrypt open /mnt/tomb/stash/deaddrop/tomb vera-tomb --key-file /mnt/tomb/stash/deaddrop/pg19322.txt
mount /dev/mapper/vera-tomb /mnt/crypt
exa --tree /mnt/crypt

  • unmounting:

    umount /mnt/crypt
cryptsetup --type tcrypt --veracrypt close vera-tomb

Arch Install

  • Strap it!

    pacstrap /mnt base base-devel btrfs-progs efibootmgr sudo zsh zsh-completions amd-ucode neovim wireless_tools wpa_supplicant dialog

Arch chroot

  • Chroot:

    arch-chroot /mnt /bin/zsh
chsh -s /bin/zsh

initramfs

  • Edit /etc/mkinitcpio.conf:

    BINARIES=("/usr/bin/btrfs")
HOOKS=(base systemd autodetect modconf block sd-encrypt btrfs filesystems keyboard fsck)

  • Generate with:

    mkinitcpio -p linux

TZ & Localization

  • Set the tz:

    ln -sf /usr/share/zoneinfo/Region/City /etc/localtime

  • Generate /etc/adjtime:

    hwclock --systohc

  • Enable NTP:

    timedatectl set-ntp true

  • Localization:

    vim /etc/locale.gen (uncomment en_US.UTF-8 UTF-8)
locale-gen
echo LANG=en_US.UTF-8 > /etc/locale.conf

  • Keyboard layout:

    echo KEYMAP=us > /etc/vconsole.conf

  • Hostname:

    echo hostname > /etc/hostname

  • Edit /etc/hosts:

    127.0.0.1   localhost
::1     localhost

  • Set root password:

    passwd

Systemd-boot

  • Initialise systemd-boot:

    bootctl --path=/boot install

  • Get System UUID:

    lsblk -o name,uuid

  • Create /boot/loader/entries/arch.conf:

    title Arch Linux
linux /vmlinuz-linux
initrd /amd-ucode.img
initrd /initramfs-linux.img
options rd.luks.name=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx=buttah luks.options=discard root=/dev/mapper/buttah rw rootflags=subvol=@ quiet

  • Edit /boot/loader/loader.conf:

    timeout 1
default arch

Bootstrap

  • Reboot into Live System

    exit
umount -R /mnt
reboot

Essentials

  • Check for updates:

    pacman -Syyuu

  • Install git:

    pacman -S git
Improve Compile Time

  • Install ccache:

    pacman -S ccache

  • Edit /etc/makepkg.conf:

    MAKEFLAGS="-j$(nproc --ignore 1)”
BUILDDIR=/tmp/makepkg
PKGEXT='.pkg.tar'
BUILDENV=(fakeroot !distcc color ccache check !sign)
COMPRESSXZ=(xz -c -T 22 -z -)

  • Add the following line to enable for command line compilations:

    export PATH="/usr/lib/ccache/bin/:$PATH"
aria2 download manager

  • Read This

  • install

    yay -S aria2

  • vim /etc/pacman.conf:

    XferCommand = /usr/bin/aria2c --allow-overwrite=true --continue=true --file-allocation=none --log-level=error --max-tries=2 --max-connection-per-server=2 --max-file-not-found=5 --min-split-size=5M --no-conf --remote-time=true --summary-interval=60 --timeout=5 --dir=/ --out %o %u

Base System

cli

  • just the basics:

    pacman -S p7zip unrar unzip zip bind-tools openssh wget tmux htop ranger ncdu
xwindows

  • base:

    pacman -S xorg xorg-xinit xorg-server
pacman -S mesa xf86-video-amdgpu libva-mesa-driver nvidia nvidia-utils vulkan-radeon 
pacman -S ttf-{bitstream-vera,dejavu,droid,hack,iosevka,liberation,roboto,ubuntu-font-family} freetype2 
pacman -S alsa-utils alsa-plugins alsa-lib pulseaudio pulseaudio-alsa pulseaudio-bluetooth

  • wm:

    pacman -S caja cinnamon gnome-screenshot gnome-system-monitor xfce4-terminal

systemctl enable NetworkManager.service

  • greeter:

    pacman -S lightdm lightdm-gtk-greeter

systemctl enable lightdm
mouse config

  • Read This

  • Find device name

    egrep "Name|Handlers" /proc/bus/input/devices | egrep -B1 'Handlers.*mouse'

  • Create /etc/X11/xorg.d/10-mouse.conf:

    Section "InputDevice"
  Identifier      "Evdev Mouse"
  Driver          "evdev"
  Option          "Name" "Logitech USB Gaming Mouse"
  Option          "evBits"  "+1-2"
  Option          "keyBits" "~272-287"
  Option          "relBits" "~0-2 ~6 ~8"
  Option          "Pass"    "3"
  Option          "CorePointer"
EndSection
User

  • create a user

    useradd -m -g users -G wheel,storage,power,network -s /bin/zsh user
passwd user
sudo pacman -S sudo

  • edit /etc/sudoers to uncomment %wheel ALL=(ALL) ALL

  • Login as user

prezto
yay (AUR helper)

  • Install:

    mkdir src
cd src
git clone https://aur.archlinux.org/yay.git
cd yay
makepkg -si
yay --nodiffmenu --save
Userland Goodies

  • cli essentials:

    yay -S atool bat cmus exa fasd fd fzf gnupg httpie jq masscan mpv neofetch nmap gnu-netcat pakbak-git peco ripgrep rsync thefuck
pip3 install git+https://github.com/tehmaze/lolcat.git

    terminal theme(s) from: https://mayccoll.github.io/Gogh/#0

  • gui stuff:

    yay -S blueberry bluez-utils

    yay -S audacious bibata-cursor-theme bookworm capitaine-cursors celluloid cherrytree chromium code dosbox evince firefox geary gitkraken keepassxc kupfer lollypop netactview pinta plank redshift snapper-git snapper-gui-git spideroak-one spotify tela-icon-theme-git typora viewnior xpad

  • Hella Extras:

    yay -S spaceship-prompt-git powerline powerline-fonts-git noto-fonts-emoji

    yay -S fortune-mod-archer fortune-mod-arresteddevelopment fortune-mod-iasip fortune-mod-futurama fortune-mod-hitchhiker fortune-mod-metalocalypse fortune-mod-portal-game fortune-mod-rickandmorty fortune-mod-southpark

    Spaceship prompt reference

  • Winfonts

    Read This

    yay -G ttf-ms-win10 

7z e Win10_1709_English_x64.iso sources/install.wim
7z e install.wim 1/Windows/{Fonts/"*".{ttf,ttc},System32/Licenses/neutral/"*"/"*"/license.rtf} -ofonts/

mv fonts/* ttf-ms-win10

cd ttf-ms-win10
makepkg --skipchecksums
Snapper

  • setup

    umount /.snapshots
rm -rf /.snapshots 
snapper -c root create-config /
mount -a
chmod 750 /.snapshots

Add files to /etc/snapper/configs/* matching your desired volumes to snapshot

  • Enable and start with:

    systemctl enable snapper-timeline.timer
systemctl enable snapper-cleanup.timer
systemctl start snapper-timeline.timer
systemctl start snapper-cleanup.timer

  • Two-hourly snapshots:
    vim /usr/lib/systemd/system/snapper-timeline.timer:

    [Timer]
OnCalendar=0/2:00:00
pacman integration

  • install

    yay -S snap-pac

  • default config

    cp /etc/snap-pac/root.conf.example /etc/snap-pac/hostname.conf
mv cp /etc/snap-pac/root.conf.example /etc/snap-pac/root.conf

  • vim /etc/snap-pac/hostname.conf:

    CLEANUP_ALGORITHM="timeline"

  • vim /etc/snap-pac.conf:

    SNAPPER_CONFIGS="hostname"

  • vim /etc/snap-pac/root.conf:

    SNAPSHOT="no"
Backups

Tweaks & Hardening

  • Deadline scheduler:

    echo "block/sda/queue/scheduler = deadline" > /etc/sysctl.d/10-deadline.conf

  • Max files:

    echo "fs.file-max = 100000" > /etc/sysctl.d/20-fs.conf

  • /etc/sysctl.d/30-jit_harden.conf

    echo "net.core.bpf_jit_harden = 2" > /etc/sysctl.d/30-jit_harden.conf

  • /etc/sysctl.d/30-kexec-restrict.conf:

    echo "kernel.kexec_loaded_disabled = 1" > /etc/sysctl.d/30-kexec-restrict.conf

  • /etc/sysctl.d/40-tcp_harden.conf:

    net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_rfc1337 = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0

  • /etc/sysctl.d/40-network.conf:

    net.core.netdev_max_backlog = 100000
net.core.netdev_budget = 50000
net.core.netdev_budget_usecs = 5000
net.core.somaxconn = 1024  
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.optmem_max = 40960
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.udp_rmem_min = 8192
net.ipv4.udp_wmem_min = 8192
net.ipv4.tcp_fastopen = 3
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 10
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_keepalive_intvl = 10
net.ipv4.tcp_keepalive_probes = 6
net.ipv4.tcp_mtu_probing = 1
net.ipv4.ip_local_port_range = 10000 65000

  • /etc/sysctl.d/50-vm.conf:

    vm.swappiness = 10
vm.dirty_ratio = 3
vm.dirty_background_ratio = 2
vm.vfs_cache_pressure = 50
vm.min_free_kbytes = 131072

  • /etc/pam.d/system-login:

    auth       optional   pam_faildelay.so     delay=5000000
auth       required   pam_tally2.so        deny=3 unlock_time=600 onerr=succeed file=/var/log/tallylog

  • /etc/ssh/sshd_config:

    Protocol 2

HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key

KexAlgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256

Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr

MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com

AuthenticationMethods publickey
PermitRootLogin no
UsePrivilegeSeparation sandbox

LogLevel VERBOSE
Subsystem sftp  /usr/lib/ssh/sftp-server -f AUTHPRIV -l INFO

  • De-activate short moduli:

    awk '$5 >= 3071' /etc/ssh/moduli > /etc/ssh/moduli.tmp && mv /etc/ssh/moduli.tmp /etc/ssh/moduli

  • ~/.ssh/config:

    HashKnownHosts yes
HostKeyAlgorithms ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256

VFIO

  • Install deps:

    yay -S linux-vfio linux-vfio-headers qemu libvirt ovmf virt-manager bridge-utils dmidecode

  • Clone boot entry

    cp /boot/loader/entries/arch.conf /boot/loader/entries/vfio.conf
vim /boot/loader/entries/vfio.conf

  • Enable iommu (and some other tweaks)
    vim /boot/loader/entries/vfio.conf:

    iommu=pt kvm-amd.avic=1 rd.driver.pre=vfio-pci

  • Force secondary GPU
    vim /etc/xorg.conf.d/10-amdgpu.conf:

    Section "Device" 
Identifier "AMD" 
Driver "amdgpu" 
BusID "PCI:2:0:0" # omit first digit from `lspci -k` 0x:00.0 
EndSection

  • Enable hugepages
    Read This

    mkdir -p /dev/hugepages

    vim /etc/fstab:

    hugetlbfs       /dev/hugepages  hugetlbfs       mode=1770,gid=78        0 0

  • Interupt mapping

    echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
echo "options kvm report_ignored_msrs=N" >> /etc/modprobe.d/kvm.conf

  • reboot

  • Check iommu is enabled: dmesg | grep -e DMAR -e IOMMU

  • Inspect iommu groups:

    #!/bin/bash
shopt -s nullglob
for d in /sys/kernel/iommu_groups/*/devices/*; do
  n=${d#*/iommu_groups/*}; n=${n%%/*}
  printf 'IOMMU Group %s ' "$n"
  lspci -nns "${d##*/}"
done;

QEMU

  • Add user to libvirt group

    sudo usermod -aG libvirt username

  • Configure libvirt

    cp /usr/share/ovmf/x64/OVMF_VARS.fd /var/lib/libvirt/images/ovmf_vars_x64.bin

    vim /etc/libvirt/qemu.conf:

    nvram = [
    "/usr/share/ovmf/x64/OVMF_CODE.fd:/var/lib/libvirt/images/ovmf_vars_x64.bin"
]

  • Enable & Start libvirtd

    systemctl enable --now libvirtd.service
systemctl enable --now virtlogd.socket

  • Default libvirt network

    yay -Syu dnsmasq firewalld
sudo systemctl start firewalld
sudo systemctl enable firewalld
sudo systemctl restart libvirtd

sudo virsh net-autostart default
sudo virsh net-start default

  • Note: this will probably fuck up your docker containers
    vim /etc/docker/daemon.json

    {
      "bridge": "virbr0",
}

Bonus! mount raw images

  • using the loopback driver:

    mkdir /mnt/raw
partx -av /path/to/img/file
mount /dev/loop0p2 /mnt/raw 
umount /mnt/raw
partx -dv /dev/loop0

GPU passthru

  • acs overwride
    add pcie_acs_override=downstream to kernel boot line

  • get pci ids with lspci -nn

  • vim /etc/modprobe.d/vfio.conf:

    softdep nouveau pre: vfio-pci 
softdep nvidia pre: vfio-pci 
softdep nvidia* pre: vfio-pci
options vfio-pci ids=10de:1b81,10de:10f0

  • reboot one last time!

Note: virt-manager is so simple these days, fuck handcrafting xmls and those custom qemu shell scripts

EvDev

  • You can be lazy and just pass through the USB ports (or devices) or you can be brave and try evdev input swapping.
    Read This

Audio

  • Using HDMI audio for the time being, can’t hurt to try SCREAM

Extras

Hardware security

Nuff Said

Personal VPN

Start Here

  • Wireguard Client:

    yay -S wireguard-arch-dkms

    OR

    yay -S wireguard-arch-dkms networkmanager-wireguard-git 

CONF_FILE="~/algo/configs/1.3.3.7/wireguard/blackvolga.conf"
nmcli connection import type wireguard file "$CONF_FILE"

  • Setup auto-connect: Read This

Personal DNS

Docker

  • Read This

  • Install

    yay -S docker docker-compose docker-machine
Useful Images

  • Listed in no particular order

    archiveteam/warrior-dockerfile
jwilder/nginx-proxy
linuxserver/beets
linuxserver/couchpotato
linuxserver/headphones
linuxserver/jackett
linuxserver/mylar
linuxserver/plex
linuxserver/pyload
linuxserver/radarr
linuxserver/sabnzbd
linuxserver/sonarr
magicwormhole/magic-wormhole
portainer/portainer
timonier/webui-aria2
writl/pyload
v2tec/watchtower

Docker all the things

Reference

Don’t take our word for it…

Arch Stuff

Linux Tweaks

XFS

Btrfs

Butter Cheatsheet

Basic stuff

  • some useful commands:

    btrfs fi show

btrfs device stats /   

btrfs filesystem usage /

btrfs filesystem defragment -r /

btrfs scrub start / && watch -n 10 "btrfs scrub status /"
Recovery

Start with a scrub, usually takes a couple hours.

  • check the logged errors with:

    btrfs inspect-internal inode-resolve XXXX /mnt

  • use a backup of the root tree in read only mode:

    mount -o usebackuproot /dev/sdXY /mnt

  • see what you can recover if you can’t mount:

    btrfs restore -v -i -o /dev/sdXY /mnt

Bonus! you can try recovering snapshots using -s

  • restore a specified root

    btrfs-find-root /dev/sdX

btrfs restore -v -i -o -t <n> /dev/sdX /mnt/restore

where n matches the root objectid

  • mostly safe:

    btrfs rescue zero-log

  • not completely safe:

    btrfs rescue super-recover

  • even less safe:
    This will take a very looong time for large drives and can result in data being wrongly restored

    btrfs rescue chunk-recover

  • last resort
    seriously, forget this command unless you have tried everything else
    (multiple times, and even then you should probably forget this anyway)

    btrfs check --repair /dev/sdXY

VFIO / KVM / QEMU

Notes

  • Snapshot is supported with raw block device.

    qemu-img create -f qcow2 -b /path/to/base/image.qcow2 /dev/sdc

  • If you refuse to use images, LVM is much more flexible and easier to manage than raw block or partitions, with good performance.

Misc.

arch  vfio  kvm  qemu  amd  ryzen