Install Ubuntu onto ZFS

ZFS is an open source file system developed by Oracle Inc. that provides enhanced data integrity, compression and data snapshots. It combines a logical volume manager and a software RAID into the one filesystem. ZFS volumes can span multiple drives, like a traditional LVM setup. ZFS can effectively reduce disk space consumption and enhance performance when compressing data.

This article details the process of installing Ubuntu 16 into a ZFS root and if installed correctly, will boot natively from a ZFS pool. This article might be adapted for use with the following operating systems


 * Linux Mint - Virtually the same as Ubuntu
 * Zorin OS - Same architecture as Ubuntu
 * Ubuntu Server

For installing Arch Linux into a ZFS root, refer to Install Arch into ZFS Root

Introduction
Ubuntu 16 can be installed to a ZFS dataset and subsequently booted from that dataset. Doing this provides additional advantages to installing Ubuntu 16:


 * Improved performance due to on-the-fly compression of data
 * Reduced disk space usage due to compression
 * Easier to spin up a clone of the installed instance

Moreover, since ZFS supports a RAID setup, the data set where Ubuntu is installed is transparent to the actual disk where Ubuntu is installed. This allows for redundancy, for instance, a RAID mirroring setup would allow the failure of a disk without impacting the running of the system.

ZFS Overview
Main Article: ZFS Filesystem

The ZFS filesystem can support up to 256 zerabytes and, if enabled, transparently compresses and decompresses data on the fly, thereby, conserving disk space consumed. In addition, ZFS has various different RAID configurations, which can enhance performance through stripping or provide data redundancy through RAID-Z or mirroring. Redundancy ensures that the server remains operational, even when a critical disk fails. While the system remains up and running, the system enters a degraded state. The situation is mitigated by swapping out the disk with a replacement. However, even when a RAID configuration is not used, ZFS is still powerful because its snapshot, compression and data integrity. Although, data loss on a ZFS system is still possible, the filesystem will not become inconsistent due to corrupt or incomplete writes. This means tools like chkdisk are not required.

OS Support
Ubuntu 16.04 has added ZFS support to its operating system. This means that ZFS can be easily installed using a few installation commands.

sudo apt-get update sudo apt install zfs

An additional set of ZFS utilities are required if intending to boot from a ZFS provisioned data set.

sudo apt-get install zfs-initramfs

System Requirements
Running ZFS has a minimal set of system requirements. Moreover, it is possible for pool corruption to occur if these requirements are not met. A general rule of thumb is that for a minimum of 8 GB of RAM is required and that for each terabyte of disk space, there should be approximately 1GB of RAM in addition to the 8GB minimum.

Error Correcting Memory (ECC) is also commonly recommended, but it is not strictly necessary for home or small office environments.

Setup Live Environment
Installing Ubuntu is usually carried out by booting from an Ubuntu Live CD, which can be downloaded from here.

In most cases, the user can simply run Ubiquity to start a graphical install of Ubuntu from the live environment. However, Ubiquity does not natively support installation to a ZFS data set.

Consequently, ZFS needs to be installed in the live environment, so that the ZFS pool and data sets can be created for housing the Ubuntu instance. The Live CD, in many ways, is very similar to an installed copy of Ubuntu and as such, the tools can be used in a similar fashion. Once the Live CD is booted to the desktop, the terminal can be opened to start entering commands.

Add / Update Installation Repositories
When Ubuntu is installed, it adds necessary repositories to the sources file, so that the installation commands know where to look for the packages to download and install. The repositories that contain the zfs packages do not exist on the live CD. Therefore, it is necessary to add a repository to install the ZFS tools.

Installation of ZFS packages requires Internet connectivity, so the live environment must have Internet access. Without Internet connectivity, Ubuntu will not be able to contact the repository, so will not be able to download necessary files. You may need to install software-properties-common to use the apt-add command.

Once Internet connectivity is established, the commands below are used to add the universe repository, which is used to install ZFS. When ZFS is not found on the live CD, the package manager will search this repository for a package named 'zfs'. When it finds it, it will ask the user if they want to continue.

sudo apt-get install -y software-properties-common sudo apt-add-repository universe sudo apt update

These commands are not installing ZFS. They are simply adding and updating the said repository. It is very important that no errors occur during repository updating. If an error occurs, it could be due to lack of Internet connectivity.

Install ZFS
You can install ZFS in the Ubuntu Live environment using these commands.

sudo apt-get install -y zfs zfs-initramfs

The package, zfs-initramfs is necessary to create a modified initramfs image that can boot from the ZFS data set.

Install Partitioning Tools
Prior to provisioning a ZFS data pool, it is first necessary to appropriately create and format partitions. The ZFS data pool is created on top of a so-called 'root data partition'. The Ubuntu Live CD comes with a graphical partitioning tool called GParted, which may be used to create appropriate partitions. However, a guide on GitHub recommends using a command line tool called 'sgdisk'. It can be installed from the terminal.

sudo apt-get install -y gdisk

Note: GParted cannot be used on non-graphical systems such as Ubuntu Server (which lacks a desktop environment).

Disk Partitioning
Partitioning one or more local storage mediums is necessary to prepare a ZFS data pool for this kind of application. Even though, an entire disk can be used for ZFS, this setup will not be bootable. Partitioning a disk offers a number of advantages:-


 * The ability to specify an UFI partition for booting UEFI-enabled systems
 * The ability to use LUKS encryption to protect the ZFS installation.
 * The ability to have a dual booted setup where partition X is ZFS and partition Y is another system such as Windows

In theory, partitions from multiple physical disks can be added to a single ZFS pool to provide enhanced performance, increased space or redundancy. However, the long term implications of this setup remain to be tested.

Create ZFS Filesystem
After creating the necessary partitions, the ZFS file system must be created. If you intend to encrypt the ZFS data, a LUKS partition is required.

Setup up ZFS Pool
The ZFS pool is created using our data partition as the so called source disk. The pool will be stored within the data partition, which allows other partitions to simultaneously reside on the disk. If encryption is being used, the data partition is the LUKS container set up in the previous section. GitHub recommends using the ashift=12 flag to support 4K sector sized disks.

The command below needs to know to which block device (disk, partition or virtual device) ZFS is being set up on. This will vary, according to your system and needs.


 * If using encryption, you will want to find the mapped LUKS device.
 * If not using encryption, you will want to find the partition that is mapped to a device alias. Aliases can be listed using

sudo ls -lhsa /dev/disk/by-id

In some cases, the appropriate device may not appear in the aliases.

sudo zpool create -o ashift=12 -O atime=off -O canmount=off -O compression=lz4 -O normalization=formD -O mountpoint=/ -R /mnt 


 * The   specifies the block device, representing the appropriate partition. The disk is referenced by an identifier. Typically, issuing 'ls' on the directory /dev/disk/by-id/ can be used to find the disk and partition.


 *   is the desired name of the pool. A useful name, such as rpool helps identify that it is the root pool for the Ubuntu installation instance.

Create Data Sets
A new root data set needs to be created inside the new ZFS pool to house the Ubuntu installation instance. You can vary the following command as required to create a valid ZFS data set.

sudo zfs create -o canmount=off -o mountpoint=none /ROOT sudo zfs create -o canmount=noauto -o mountpoint=/ /ROOT/ubuntu sudo zfs mount /ROOT/ubuntu

You may want to additionally create more data sets, for instance, for storing user data in a separate data set from your main system. This is important when using snapshots because reverting a broken system will not accidentally erase user data.

For example, to create a data set for storing home directories, you could do,

sudo zfs create -o setuid=off /home

You may want to create a cache directory that does not have snapshots. You can disable snapshots on a created ZFS data set with this command.

sudo zfs create -o com.sun:auto-snapshot=false /cache

You may use these commands to create data sets for your var and log directories.

<pre style="font-weight:bold;">sudo zfs create -o canmount=off -o setuid=off -o exec=off <pool-name>/var sudo zfs create <pool-name>/var/log sudo zfs create <pool-name>/var/spool sudo zfs create -o com.sun:auto-snapshot=false -o exec=on <pool-name>/var/tmp

Install Base System
The current ZFS data sets are empty and do not have any files on them. The Ubuntu installer does not support directly installing to a ZFS data set. Consequently, one approach is to copy an Ubuntu installation to the ZFS data set. For this approach to work, Ubuntu must first be installed somewhere and then copied over to the ZFS dataset.

A second approach is to install a base system of Ubuntu directly into the mounted directory. Typically, if the Ubuntu system does not have a graphical environment, then you can use this method. Otherwise, you can use the Ubuntu installer to install Ubuntu to an alternate location.

Graphic Install
If the Ubuntu live environment has graphics avaliable, the Ubuntu graphical installer can be used to install Ubuntu to an alternate location, and then moved over to the ZFS data set.

A small 20GB ZFS volume (known as a ZVOL) is created within the ZFS pool i.e. <pool-name>. You can use this command to create a ZVOL with 20GB of provisioned disk space. You may adjust the space allocation as desired

<pre style="font-weight:bold;">sudo zfs create -V 20G <pool-name>/ubuntu-installation

You might be curious to know, whether or not, you can simply boot Ubuntu by installing it here and rebooting. The problem is that the installer fails to install the boot loader. This is not a problem when moving the installation to a ZFS dataset, but the ZVOL itself is not bootable.

The graphical installer will detect zvol as though it is another hard disk drive. As such, you may use this installer to select that ZVOL device and install to it. This process will copy the files and configure Ubuntu. Since these files will be later copied, it it unnecessary to create additional partitions and mount points. Ubuntu can be installed to the root of the device.

The wizard will allow you to correctly complete timezone and username information without needing to manually edit text files. The error about the failed boot loader can be safely ignored, since the boot loader will have to be manually installed.

The ZVOL should be mounted, so files stored on it can be copied to the new ZFS data set. The /mnt directory is likely already in use due to having previously mounted the ZFS root data set. Therefore, the zvol should be mounted elsewhere

<pre style="font-weight:bold;">sudo mount /dev/zd0p1 /media

The rsync utility can be used to copy the Ubuntu installation to the ZFS data set. The command below can be used. You may need to change the directories to correspond with the correct mount points.

<pre style="font-weight:bold;">sudo rsync -avPX /media/. /mnt

Once the files have been copied, chroot can be used to log in to the new Ubuntu instance. This is necessary to configure the boot loader and install ZFS tools within the new Ubuntu instance. Currently, changes have been made to the currently running live environment and not to the Ubuntu installation.

Command Line Install
When installing from the command line, a minimal base install of Ubuntu is carried out using debootstrap. This tool can be installed from the terminal.

<pre style="font-weight:bold;">sudo apt-get install -y debootstrap

You can use the debootstrap utility to install a very minimal version of Ubuntu. Packages can then be installed as desired on top of the minimal environment, including a desktop environment. Debootstrap does not install a desktop environment and leaves configuration to the user.

You can use these commands to install Ubuntu into the /mnt directory ,

<pre style="font-weight:bold;">sudo chmod 1777 /mnt/var/tmp sudo debootstrap xenial /mnt sudo zfs set devices=off rpool

The system is unconfigured, so it is advisable to configure it to ensure smooth running of the system. Configuration changes need to be made to the config file in /mnt/etc/hosts. The IP address 127.0.1.1 should be bound to the local host name, or to the DNS name. Likewise, the network interface needs to be configured to use a valid IP address, or DHCP.

Chroot Into Ubuntu
A bash command can be used to automatically mount necessary aliases, so they can be accessed from within the chroot environment.

<pre style="font-weight:bold;">sudo su for d in proc sys dev; do mount --bind /$d /mnt/$d; done chroot /mnt echo "nameserver 8.8.8.8" | tee -a /etc/resolv.conf

The chroot environment now has Internet access, allowing packages to be installed directly to the newly installed instance of Ubuntu.

System Configuration (Non-Graphical Install)
If you used the non-graphical install method, you need to configure your system. Otherwise, Ubuntu will not run properly when it is booted. The graphical installer already completes this configuration, so in that case, you should skip to the next section, Install Boot Images.

Add Source Repositories
You need to add appropriate repositories to your /etc/sources.list file, so that software can be located and installed by the package manager. An Ubuntu configuration file generator is located here. You can copy and paste the configuration into your /etc/sources.list file.

After adding new packages, you should run apt-get update so the new packages can be used.

<pre style="font-weight:bold;">ln -s /proc/self/mounts /etc/mtab apt update

Set System Language
You need to set the appropriate language for your system. GitHub recommends having United States English always available, even if you live outside the United States.

<pre style="font-weight:bold;">locale-gen en_US.UTF-8 echo 'LANG="en_US.UTF-8"' > /etc/default/locale dpkg-reconfigure tzdata

Install Minimal Packages
You should install the minimal set of packages for your system

<pre style="font-weight:bold;">apt install --yes ubuntu-minimal apt install -y --no-install-recommends linux-image-generic

Setup Groups
System groups need to be configured on the newly installed system,

<pre style="font-weight:bold;">addgroup --system lpadmin addgroup --system sambashare

Root Password
A root password should be set for security purposes. It is advisable to use a password different from your username, since a compromised user password could provide an attacker with root access.

<pre style="font-weight:bold;">passwd

Install Boot Images
The ZFS and kernel boot images in the chroot environment need to be updated to support booting from ZFS. This requires the installation of the ZFS intramfs package.

<pre style="font-weight:bold;">apt install -y zfs-initramfs

Install Boot Loader
The boot loader (i.e. GRUB) is required to be boot the new installation. The boot files do not reside inside the ZFS dataset, but on a small boot partition. The BIOS cannot see ZFS directly and as such, a separate partition is needed to kick start the boot process. The boot files in Ubuntu are located in /boot/. This directory, therefore, needs to be mapped to the correct boot partition, so that the files are physically created and updated on the correct partition.

There are two options for installing the boot loader depending on your needs:-


 * Legacy Booting - The old MBR setup that lacks EFI support
 * UEFI - The more modern approach to booting. The system boots by looking for a small FAT32 formatted partition. The MBR is not used.

Legacy Booting
Legacy booting is simply a matter of installing GRUB using the following commands,

<pre style="font-weight:bold;">sudo apt install -y grub-pc update-initramfs -c -k all grub-install

Replace with the path to the main disk alias e.g. /dev/sda or /dev/disk/by-id/scsi-SATA_disk1

UEFI
UEFI booting requires a small partition formatted with the FAT32 filesystem and appropriate boot files added to it. This can be achieved by mounting the physical boot partition to /boot, so that Ubuntu appropriately adds the files. If the partition is not formatted with FAT32 filesystem, you will need to do this first. Otherwise, you can configure the chroot environment, so that the boot partition is mounted.

The /boot/ directory does not contain a required subdirectory, efi by default. Therefore, it is necessary to first create this directory,

<pre style="font-weight:bold;">sudo mkdir /boot/efi

Next, changes should be made to the /etc/fstab configuration file, so the boot partition is automatically mounted by the system at path /boot. This is the canonical path used by Ubuntu to look for boot files. It is important to do this to ensure that the correct boot files are updated. Replace <boot-partition> with the correct boot partition alias.

<pre style="font-weight:bold;">echo PARTUUID=$(blkid -s PARTUUID -o value /dev/disk/by-id/<boot-partition>) /boot/efi vfat defaults 0 1 >> /etc/fstab sudo mount /boot/efi sudo apt install -y grub-efi-amd64

Note:  This may fail if the Live environment is not booted in UEFI mode.

To install UEFI-enabled GRUB to the EFI partition, you should run the following commands,

<pre style="font-weight:bold;">update-initramfs -c -k all grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=ubuntu --recheck --no-floppy