Most of this blog has actually been about on-prem stuff, but hey, it is 2024 so let's do something in the cloud. Let's try to put Gentoo Linux on OCI. One way how to achieve that is to export a VM from VirtualBox to OCI. This blog however describes how to change an existing linux VM with Oracle Linux to Gentoo, closely following the Gentoo handbook.
The end result should be Gentoo on a btrfs file system. OpenRC will be used as the init system.
This first part will cover getting a Gentoo system running. Subsequent parts will be dealing with tidying up, and creating a custom image.
Create a VM and an extra block storage
The start is a standard Oracle Linux 8 VM, with an AMD or Intel shape. Make sure to be able to login over ssh.
Then create an extra block storage of 50 GB and attach it to the VM. The extra block storage will be used to park the Oracle Linux OS on.
Inspect the VM
Linux version
[opc@Gentoo ~]$ hostnamectl
Static hostname: Gentoo
Icon name: computer-vm
Chassis: vm
Machine ID: 5b526e3f9546492b9b69c39b504b5fd1
Boot ID: 8dfaa293d3794f559183bf49cd34037e
Virtualization: kvm
Operating System: Oracle Linux Server 8.9
CPE OS Name: cpe:/o:oracle:linux:8:9:server
Kernel: Linux 5.15.0-203.146.5.1.el8uek.x86_64
Architecture: x86-64
This is a KVM virtualized VM, with Oracle Linx 8.9 on it, and Linux kernel 5.15.
Disk layout:
[opc@Gentoo ~]$ lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINT
NAME SIZE TYPE FSTYPE MOUNTPOINT
sda 46.6G disk
├─sda1 100M part vfat /boot/efi
├─sda2 1G part xfs /boot
└─sda3 45.5G part LVM2_member
├─ocivolume-root 35.5G lvm xfs /
└─ocivolume-oled 10G lvm xfs /var/oled
sdb 50G disk
The VM is booted using an UEFI boot loader, with a separate boot partition, and LVM as logical volume manager. There are two logical volumes, both formatted with XFS:
- ocivolume-root: with the OL8 operating system
- ocivolume-oled: which is used to gather statistics with the PCP package. PCP is only available under OL8, so we do not need to retain it.
Our extra disk shows up as /dev/sdb.
Boot manager
[opc@OL-8 ~]$ ls /boot
config-4.18.0-513.11.0.1.el8_9.x86_64 loader
config-5.15.0-203.146.5.1.el8uek.x86_64 symvers-4.18.0-513.11.0.1.el8_9.x86_64.gz
efi symvers-5.15.0-203.146.5.1.el8uek.x86_64.gz
grub2 System.map-4.18.0-513.11.0.1.el8_9.x86_64
initramfs-0-rescue-c5779a24a7110341165b3f6f2fbec14a.img System.map-5.15.0-203.146.5.1.el8uek.x86_64
initramfs-4.18.0-513.11.0.1.el8_9.x86_64.img vmlinuz-0-rescue-c5779a24a7110341165b3f6f2fbec14a
initramfs-5.15.0-203.146.5.1.el8uek.x86_64.img vmlinuz-4.18.0-513.11.0.1.el8_9.x86_64
initramfs-5.15.0-203.146.5.1.el8uek.x86_64kdump.img vmlinuz-5.15.0-203.146.5.1.el8uek.x86_64
The boot manager is GRUB (2), the current version 2, not the old legacy version using an initramfs.
Move Oracle Linux to the additional block volume
All this work needs to be done as root.
Since OL8 is placed on a LVM, it is quite easy to move it to the extra block volume:
[root@Gentoo opc]# fdisk /dev/sdb
[root@Gentoo opc]# vgextend ocivolume /dev/sdb
[root@Gentoo opc]# time pvmove /dev/sda3
/dev/sda3: Moved: 0.05%
/dev/sda3: Moved: 0.89%
..
..
/dev/sda3: Moved: 99.01%
/dev/sda3: Moved: 99.77%
real 33m22.695s
user 0m0.086s
sys 0m0.073s
[root@Gentoo opc]# vgreduce ocivolume /dev/sda3
Removed "/dev/sda3" from volume group "ocivolume"
[root@Gentoo opc]# pvremove /dev/sda3
Labels on physical volume "/dev/sda3" successfully wiped.
[root@Gentoo opc]# pvdisplay
--- Physical volume ---
PV Name /dev/sdb
VG Name ocivolume
PV Size 50.00 GiB / not usable 4.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 12799
Free PE 1158
Allocated PE 11641
PV UUID iUXGmX-Njij-cVWt-wqjR-Y6Nl-xLxd-L0kKi7
Reformat the root disk
There is now 45 GB on /dev/sda3 for the Gentoo rootfs. OL-8 has a swapfile on XFS formatted the root file system. Swapfiles are not well supported on btrfs; it may be better to allocate a disk partition for this purpose:
[root@Gentoo opc]# fdisk /dev/sda
Command (m for help): p
Disk /dev/sda: 46.6 GiB, 50010783744 bytes, 97677312 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disklabel type: gpt
Disk identifier: 3A87099A-673A-4D5E-9C66-9D307A1A0FBF
Device Start End Sectors Size Type
/dev/sda1 2048 206847 204800 100M EFI System
/dev/sda2 206848 2303999 2097152 1G Linux filesystem
/dev/sda3 2304000 97675263 95371264 45.5G Linux LVM
Command (m for help): d
Partition number (1-3, default 3):
Partition 3 has been deleted.
Command (m for help): n
Partition number (3-128, default 3):
First sector (2304000-97677278, default 2304000):
Last sector, +sectors or +size{K,M,G,T,P} (2304000-97677278, default 97677278): +8G
Created a new partition 3 of type 'Linux filesystem' and of size 8 GiB.
Command (m for help): t
Partition number (1-3, default 3):
Partition type (type L to list all types): 19
Changed type of partition 'Linux filesystem' to 'Linux swap'.
Command (m for help): n
Partition number (4-128, default 4):
First sector (19081216-97677278, default 19081216):
Last sector, +sectors or +size{K,M,G,T,P} (19081216-97677278, default 97677278):
Created a new partition 4 of type 'Linux filesystem' and of size 37.5 GiB.
Command (m for help): p
Disk /dev/sda: 46.6 GiB, 50010783744 bytes, 97677312 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 1048576 bytes
Disklabel type: gpt
Disk identifier: 3A87099A-673A-4D5E-9C66-9D307A1A0FBF
Device Start End Sectors Size Type
/dev/sda1 2048 206847 204800 100M EFI System
/dev/sda2 206848 2303999 2097152 1G Linux filesystem
/dev/sda3 2304000 19081215 16777216 8G Linux swap
/dev/sda4 19081216 97677278 78596063 37.5G Linux filesystem
Command (m for help): w
The partition table has been altered.
Syncing disks.
Format and mount the swap and Gentoo partiton.
[root@Gentoo opc]# mkswap -L swap-sda3 /dev/sda3
[root@Gentoo opc]# swapon /dev/sda3
[root@Gentoo opc]# mkfs -t btrfs -L rootfs /dev/sda4
[root@Gentoo opc]# mkdir /mnt/rootfs /mnt/gentoo
[root@Gentoo opc]# mount /dev/sda4 /mnt/rootfs
[root@Gentoo opc]# btrfs subvol create /mnt/rootfs/gentoo
[root@Gentoo opc]# mount -o subvol=gentoo /dev/sda4 /mnt/gentoo
[root@Gentoo opc]# btrfs subvol create /mnt/rootfs/home
The result is that btrfs subvolume 'gentoo' on the /dev/sda41 partition is mounted on /mnt/gentoo, just like the Gentoo handbook prescribes. The complete partition is mounted on /mnt/rootfs too.
Install Gentoo
The starting point in the handbook would be the chapter 'the stage file'.
[root@Gentoo opc]# cd /mnt/gentoo
[root@Gentoo gentoo]# wget https://distfiles.gentoo.org/releases/amd64/autobuilds/20240329T230405Z/stage3-amd64-openrc-20240329T230405Z.tar.xz
[root@Gentoo gentoo]# tar xpf stage3-*.tar.xz --xattrs-include='*.*' --numeric-owner
Add the following to /mnt/gentoo/etc/portage/make.conf:
GRUB_PLATFORMS="efi-64"
ACCEPT_LICENSE="* -@EULA"
FEATURES="parallel-fetch clean-logs"
EMERGE_DEFAULT_OPTS="--jobs=4 --load-average=11"
MAKEOPTS="-j8"
Adjust MAKEOPTS according to the number of OCPUs, memory, and storage available.
Copy some configuration files
These will need to be updated later, but already gives a good starting point:
[root@Gentoo opc]# cp /etc/fstab /mnt/gentoo/etc
[root@Gentoo opc]# cp /etc/default/grub /mnt/gentoo/etc/default/
Chroot into the Gentoo environment
[root@Gentoo gentoo]# cp --dereference /etc/resolv.conf /mnt/gentoo/etc/
[root@Gentoo gentoo]# mount --types proc /proc /mnt/gentoo/proc
[root@Gentoo gentoo]# mount --rbind /sys /mnt/gentoo/sys
[root@Gentoo gentoo]# mount --rbind /dev /mnt/gentoo/dev
[root@Gentoo gentoo]# mount --bind /run /mnt/gentoo/run
[root@Gentoo gentoo]# chroot /mnt/gentoo /bin/bash
Gentoo / # source /etc/profile
Gentoo / # export PS1="(chroot) ${PS1}"
Repositories
(chroot) Gentoo / # mkdir --parents /etc/portage/repos.conf
(chroot) Gentoo / # cp /usr/share/portage/config/repos.conf /etc/portage/repos.conf/gentoo.conf
(chroot) Gentoo / # emerge-webrsync
(chroot) Gentoo / # eselect news read
Timezone
(chroot) Gentoo / # echo "Europe/Amsterdam" > /etc/timezone
(chroot) Gentoo / # emerge --config sys-libs/timezone-data
Locales
(chroot) Gentoo / # cat > /etc/locale.gen << AA
en_US ISO-8859-1
en_US.UTF-8 UTF-8
de_DE ISO-8859-1
de_DE.UTF-8 UTF-8
nl_NL ISO-8859-1
nl_NL.UTF-8 UTF-8
AA
(chroot) Gentoo / # locale-gen
* Generating 7 locales (this might take a while) with 12 jobs
* (3/7) Generating en_US.ISO-8859-1 ... [ ok ]
* (1/7) Generating de_DE.ISO-8859-1 ... [ ok ]
* (5/7) Generating nl_NL.ISO-8859-1 ... [ ok ]
* (7/7) Generating C.UTF-8 ... [ ok ]
* (4/7) Generating en_US.UTF-8 ... [ ok ]
* (2/7) Generating de_DE.UTF-8 ... [ ok ]
* (6/7) Generating nl_NL.UTF-8 ... [ ok ]
* Generation complete
* Adding locales to archive ... [ ok ]
(chroot) Gentoo / # eselect locale list
Available targets for the LANG variable:
[1] C
[2] C.utf8
[3] POSIX
[4] de_DE
[5] de_DE.iso88591
[6] de_DE.utf8
[7] en_US
[8] en_US.iso88591
[9] en_US.utf8
[10] nl_NL
[11] nl_NL.iso88591
[12] nl_NL.utf8
[13] C.UTF8 *
[ ] (free form)
(chroot) Gentoo / # eselect locale set 2
Setting LANG to C.utf8 ...
Run ". /etc/profile" to update the variable in your shell.
(chroot) Gentoo / # . /etc/profile && export PS1="(chroot) ${PS1}"
Firmware
(chroot) Gentoo / # emerge --ask sys-kernel/linux-firmware
Update /etc/fstab
Determine the UUIDs of /dev/sda*
(chroot) Gentoo / # lsblk /dev/sda -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINT,UUID
NAME SIZE TYPE FSTYPE MOUNT UUID
sdc 46.6G disk
├─sda1 100M part vfat EFB2-4487
├─sda2 1G part xfs 02a8feb2-328b-4743-8178-c163b8b72aa8
├─sda3 8G part swap eeff176a-5654-440c-9290-bf9ad12734ba
└─sda4 37.5G part btrfs / e7588fd9-ad29-4cf3-b732-6a32a3437b82
Update /etc/fstab with:
- the UUID of the rootfs for the rootfs whole partition
- and the same for the root mount point, and the home mount point
- change the swap file with the swap partition on /dev/sda3
When done /etc/fstab should look something like this:
UUID=EFB2-4487 /boot/efi vfat defaults,uid=0,gid=0,umask=077,shortname=winnt 0 2
UUID=02a8feb2-328b-4743-8178-c163b8b72aa8 /boot xfs defaults 0 0
UUID=e7588fd9-ad29-4cf3-b732-6a32a3437b82 / btrfs noatime,subvol=gentoo 0 0
UUID=e7588fd9-ad29-4cf3-b732-6a32a3437b82 /mnt/rootfs btrfs noatime 0 0
UUID=e7588fd9-ad29-4cf3-b732-6a32a3437b82 /home btrfs noatime,subvol=home 0 0
UUID=eeff176a-5654-440c-9290-bf9ad12734ba none swap sw,pri=10 0 0
tmpfs /dev/shm tmpfs defaults,nodev,nYosuid,noexec 0 0
Make sure to use the right UUIDs.
Set the hostname
(chroot) Gentoo / # echo "gentoo" > /etc/hostname
Install file system tools
(chroot) Gentoo / #emerge -v sys-fs/xfsprogs sys-fs/e2fsprogs sys-fs/dosfstools sys-fs/btrfs-progs sys-fs/lvm2
Install networking tools
(chroot) Gentoo / # emerge -v net-misc/dhcp net-misc/dhcpcd
Install some other tools
(chroot) Gentoo / # emerge -va busybox syslog-ng sys-process/cronie net-misc/chrony app-admin/logrotate app-editors/vim app-admin/sudo
Mount /boot/efi, /home, and /mnt/rootfs
(chroot) Gentoo / # mkdir /mnt/rootfs
(chroot) Gentoo / # mount /boot/efi
(chroot) Gentoo / # mount /home
Configure Grub
The standard parameters for for grub in /etc/default/grub were copied earlier from the original Oracle Linux root volume. These need to be adjusted:
- update the GRUB_DISTRIBUTOR parameter
- remove the LVM parameters from the commandline,
The file should look like this:
GRUB_TIMEOUT=5
# GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DISTRIBUTOR="Gentoo"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_DISABLE_RECOVERY="true"
GRUB_ENABLE_BLSCFG=true
GRUB_TERMINAL="console"
# GRUB_CMDLINE_LINUX="crashkernel=auto LANG=en_US.UTF-8 console=tty0 console=ttyS0,115200 rd.luks=0 rd.md=0 rd.dm=0 rd.lvm.vg=ocivolume rd.lvm.lv=ocivolume/root rd.net.timeout.carrier=5 netroot=iscsi:169.254.0.2:::1:iqn.2015-02.oracle.boot:uefi rd.iscsi.param=node.session.timeo.replacement_timeout=6000 net.ifnames=1 nvme_core.shutdown_timeout=10 ipmi_si.tryacpi=0 ipmi_si.trydmi=0 libiscsi.debug_libiscsi_eh=1 loglevel=4 ip=dhcp,dhcp6 rd.net.timeout.dhcp=10 crash_kexec_post_notifiers"
GRUB_CMDLINE_LINUX="crashkernel=auto LANG=en_US.UTF-8 console=tty0 console=ttyS0,115200 rd.luks=0 rd.md=0 rd.dm=0 rd.net.timeout.carrier=5 netroot=iscsi:169.254.0.2:::1:iqn.2015-02.oracle.boot:uefi rd.iscsi.param=node.session.timeo.replacement_timeout=6000 net.ifnames=1 nvme_core.shutdown_timeout=10 ipmi_si.tryacpi=0 ipmi_si.trydmi=0 libiscsi.debug_libiscsi_eh=1 loglevel=4 ip=dhcp,dhcp6 rd.net.timeout.dhcp=10 crash_kexec_post_notifiers"
Linux kernel
The new Gentoo distribution kernel is linked with a package installkernel. Intstallkernel will install the kernel in /boot, but can also generate an initramfs, and configure grub.
(chroot) Gentoo / # echo "sys-kernel/installkernel dracut grub" >> /etc/portage/package.use/installkernel
(chroot) Gentoo / # emerge -va gentoo-kernel
Update the EFI boot loader
(chroot) Gentoo / # grub-install --efi-directory=/boot/efi
Installing for x86_64-efi platform.
Installation finished. No error reported.
Start the neccessary services
Uncomment the serial console section in /etc/inittab:
# SERIAL CONSOLES
s0:12345:respawn:/sbin/agetty -L 115200 ttyS0 vt100
s1:12345:respawn:/sbin/agetty -L 115200 ttyS1 vt100
And enable the services for sshd, syslog-ng, chronie, crony:
(chroot) Gentoo / # rc-update add sshd default
(chroot) Gentoo / # rc-update add cronie default
(chroot) Gentoo / # rc-update add syslog-ng default
(chroot) Gentoo / # rc-update add chronyd default
Create a user
The OCI default user opc and provide initial passwd:
(chroot) Gentoo / # useradd -g users -G wheel -m -u 2000 -N opc
(chroot) Gentoo / # passwd opc
It is given the uid of 2000, to not interfere with other uids that one may have on other systems.
Set root password
(chroot) Gentoo / # passwd
Update /etc/sudoers
So that anyone in group wheel can sudo:
(chroot) Gentoo / # visudo
And update /etc/sudoers as follows:
## Uncomment to allow members of group wheel to execute any command
# %wheel ALL=(ALL:ALL) ALL
%wheel ALL=(ALL:ALL) ALL
Prevent interactive logins over ssh
We only allow login with ssh keys. Update /etc/ssh/sshd_config:
#KbdInteractiveAuthentication yes
KbdInteractiveAuthentication no
Copy opc's ssh's authorized_keys to the new opc user
(chroot) Gentoo / # exit
[root@Gentoo opc]# mkdir /mnt/gentoo//home/opc/.ssh
[root@Gentoo opc]# cp /home/opc/.ssh/authorized_keys /mnt/gentoo//home/opc/.ssh/
[root@Gentoo opc]# chown 2000:users /mnt/gentoo//home/opc/.ssh/
[root@Gentoo opc]# chown 2000:users /mnt/gentoo//home/opc/.ssh/authorized_keys
[root@Gentoo opc]# chmod 700 /mnt/gentoo//home/opc/.ssh/
Reboot
And hope it works...
Refer to the troubleshooting section below in case it does not.
Troubleshooting
OL8 may be booted in case the machine was stopped and started at a later time.
This can be fixed by chrooting into the gentoo partition at /dev/sda4,
and run another grub-install --efi-directory=/boot/efi.
Afterwards reboot.
In case ssh does not work, then it may be possible to use a console connection to enter the Gentoo instance.
In case something else does not work then it may be possible to fix the issue by:
- stopping the compute instance
- detaching the boot volume from the compute instance
- attaching it to another running compute instance
- chroot into the boot volume and fix the issue.
- exit the chroot
- detach the storage
- attach it to, and boot the compute instance