Install Ubuntu onto ZFS

ZFS is an open source file system developed by Oracle Inc. that provides enhanced data integrity, compression and data snapshots. It combines a logical volume manager and a software RAID into the one filesystem. ZFS volumes can span multiple drives, like a traditional LVM setup. ZFS can effectively reduce disk space consumption and enhance performance when compressing data.

This article details the process of installing Ubuntu 16 into a ZFS root and if installed correctly, will boot natively from a ZFS pool. This article might be adapted for use with the following operating systems


 * Linux Mint - Virtually the same as Ubuntu
 * Zorin OS - Same architecture as Ubuntu
 * Ubuntu Server

For installing Arch Linux into a ZFS root, refer to Install Arch into ZFS Root

Introduction
Ubuntu 16 can be installed to a ZFS dataset and booted from that dataset. Doing this provides additional advantages to installing Ubuntu 16:


 * Improved performance due to on-the-fly compression of data
 * Reduced disk space usage due to compression
 * Easier to spin up a clone of the installed instance

Moreover, since ZFS supports a RAID setup, the data set where Ubuntu is installed is transparent to the actual disk where Ubuntu is installed. This allows for redundancy, for instance, a RAID mirroring setup would allow the failure of a disk without impacting the running of the system.

ZFS Overview
Main Article: ZFS Filesystem

The ZFS filesystem supports up to 256 zerabytes and, if enabled, transparently compresses and decompresses data on the fly, reducing disk space usage. ZFS has various different RAID configurations, which can enhance performance through stripping or provide data redundancy through RAID-Z or mirroring. Redundancy ensures that the server remains operational, even when a critical disk fails. While the system remains up and running, the system enters a degraded state. The situation is mitigated by swapping out the disk with a replacement. However, even when a RAID configuration is not used, ZFS is still powerful because its snapshot, compression and data integrity. Although, data loss on a ZFS system is still possible, the filesystem will not become inconsistent due to corrupt or incomplete writes. This means tools like chkdisk are not required.

OS Support
Ubuntu 16.04 has added ZFS support to its operating system. This means that ZFS can be easily installed using a few installation commands specified in the next section.

System Requirements
Running ZFS has a minimal set of system requirements. Moreover, it is possible for pool corruption to occur if these requirements are not met. A general rule of thumb is that for a minimum of 8 GB of RAM is required and that for each terabyte of disk space, there should be approximately 1GB of RAM in addition to the 8GB minimum.

Error Correcting Memory (ECC) is also commonly recommended, but it is not strictly necessary for home or small office environments.

Overview
The process of installing Ubuntu into a ZFS volume involves a multi-part process


 * 1) Prepare Live Environment by adding ZFS support
 * 2) Provision ZFS volume by formatting an existing hard disk.
 * 3) Optionally encrypt the partition housing the ZFS pool for additional confidentiality
 * 4) Create new ZFS pool and filesystems for housing new installation
 * 5) Install Ubuntu into a (non-booteable) ZFS volume
 * 6) Copy the installed Ubuntu instance onto a mounted ZFS root
 * 7) Configure Ubuntu
 * 8) Reconfigure the boot loader to boot off the new ZFS pool

When installing Ubuntu normally, many of these cumbersome steps are completed automatically by a scripted utility called Ubiquity, however, Ubiquity is older than ZFS support for Linux and so doesn't support the additional steps needed to install Ubuntu there. This means that Ubuntu needs to be installed differently.

Setup Live Environment
Setup the live environment by following this guide

Install ZFS
The package, zfs-initramfs is necessary to create a modified initramfs image that can boot from the ZFS data set. Please type these commands to install ZFS utilities needed for accessing ZFS volumes on a hard disk:

sudo apt-get install -y zfs zfs-initramfs

Partition and Format Hard Disks
Prior to provisioning a ZFS data pool, it is first necessary to appropriately create and format the partitions. The ZFS data pool will be created on top of a so-called root data partition. Partitioning one or more local storage mediums is necessary to prepare a ZFS data pool for this kind of application. Even though, an entire disk can be used for ZFS, this setup will not be bootable. Partitioning a disk offers a number of advantages:-


 * The ability to specify an UFI partition for booting UEFI-enabled systems
 * The ability to use LUKS encryption to protect the ZFS installation.
 * The ability to have a dual booted setup where partition X is ZFS and partition Y is another system such as Windows

In theory, partitions from multiple physical disks can be added to a single ZFS pool to provide enhanced performance, increased space or redundancy. However, the long term implications of this setup remain to be tested.

1. Install SGDisk Utility
The Ubuntu Live CD comes with a graphical partitioning tool called GParted, which may be used to create appropriate partitions. However, to reduce problems, the sgdisk utility should be used.

Type:

sudo apt-get install -y gdisk

Please Note: GParted cannot be used on non-graphical systems such as Ubuntu Server (which lack a desktop environment).

2. Wipe Partition Table on physical medium
Ideally, the existing partition table (and any data currently residing on the disk) should be destroyed before attempting to create a valid partition setup. Doing so will render data inaccessible with no guarrenteed chance of successful recovery. You should back any sensitive data before proceeding. Otherwise, it will be deleted and may become corrupted. If the disk is new and has never been formatted or used before, then you can skip this step, since no partition table will be present.

Type the command below to wipe the existing partition table.

sudo sgdisk --zap-all /dev/disk/by-id/

3A. Create Partition for UEFI Booting
Modern computer systems, including many tablet-based computers such as the Microsoft Surface support Unified Extensible Firmware Interface designed to standardize disk booting. This is option is recommended if your computer supports it.

To create a UEFI-enabled boot partition, type:

sudo sgdisk -n3:1M:+512M -t3:EF00 /dev/disk/by-id/

3B. Create Partition for Legacy Booting
If your computer does not have support for UEFI or if you booted the Ubuntu live CD in legacy mode, then you should provision a boot partition that supports booting in legacy mode.

sudo sgdisk -a1 -n2:34:2047 -t2:EF02 /dev/disk/

Setup Data Partitions
In this step, a data partition will be created that will house the ZFS pool. There are are two different types of data partitions that can be created:


 * 1) Encrypted ZFS Pool: A ZFS pool is created on a LUKS-encrypted data partition
 * 2) Unencrypted ZFS Pool: A ZFS pool is created on an unencrypted partition

Method 1: Encrypted ZFS Pool using LUKS
''Warning: Encryption is not a license for criminal, immoral activity prohibited by law. If your computer is seized by law enforcement, LUKS encryption will not suffice to limit access to the data. Please also note that LUKS encryption is vulnerable to physical access attacks such as cold boot attacks

A): The first step is to create a new, LUKS partition, which will house the ZFS pool information. To do this, type the following commands:

sudo sgdisk -n4:0:+512M -t4:8300 /dev/disk/by-id/ sudo sgdisk    -n1:0:0      -t1:8300 /dev/disk/by-id/scsi-SATA_disk1

B): Format the new partition using the LUKS filesystem:

sudo cryptsetup luksFormat -c aes-xts-plain64 -s 256 -h sha256 \ /dev/disk/by-id/-part1

Note: You may be prompted to enter a passphrase. This is a personal "password" designed to protect the volume. You will also have to re-enter it every time the machine boots. For more information and alternate solutions, please see Full Disk Encryption.

C): Open the newly formatted LUKS volume and map to /dev/mapper/luksVol:

sudo cryptsetup luksOpen /dev/disk/by-id/-part1 luksVol

D):  Enter the same passphrase used to create the volume

Setup up ZFS Pool
The ZFS pool is created using our data partition as the so called source disk. The pool will be stored within the data partition, which allows other partitions to simultaneously reside on the disk. If encryption is being used, the data partition is the LUKS container set up in the previous section. GitHub recommends using the ashift=12 flag to support 4K sector sized disks.

1. Find Appropriate Block Device
The command below needs to know to which block device (disk, partition or virtual device) ZFS is being set up on. This will vary, according to your system and needs.


 * If using encryption, you will want to find the mapped LUKS device.
 * If not using encryption, you will want to find the partition that is mapped to a device alias. Aliases can be listed using:

sudo ls -lhsa /dev/disk/by-id

2. Provision ZFS Pool
Setup the ZFS pool by typing:

sudo zpool create -o ashift=12 -O atime=off -O canmount=off -O compression=lz4 -O normalization=formD -O mountpoint=/ -R /mnt <pool-name>

3. Create Data Sets
A new root data set needs to be created inside the new ZFS pool to house the Ubuntu installation instance. You can vary the following command as required to create a valid ZFS data set.

<pre style="font-weight:bold;">sudo zfs create -o canmount=off -o mountpoint=none <pool-name>/ROOT sudo zfs create -o canmount=noauto -o mountpoint=/ <pool-name>/ROOT/ubuntu sudo zfs mount <pool-name>/ROOT/ubuntu

You may want to additionally create more data sets, for instance, for storing user data in a separate data set from your main system. This is important when using snapshots because reverting a broken system will not accidentally erase user data.

For example, to create a data set for storing home directories, you could do:

<pre style="font-weight:bold;">sudo zfs create -o setuid=off <pool-name>/home

You may want to create a cache directory that does not have snapshots. You can disable snapshots on a created ZFS data set with this command.

<pre style="font-weight:bold;">sudo zfs create -o com.sun:auto-snapshot=false <pool-name>/cache

You may use these commands to create data sets for your var and log directories.

<pre style="font-weight:bold;">sudo zfs create -o canmount=off -o setuid=off -o exec=off <pool-name>/var sudo zfs create <pool-name>/var/log sudo zfs create <pool-name>/var/spool sudo zfs create -o com.sun:auto-snapshot=false -o exec=on <pool-name>/var/tmp

Install Base System
The current ZFS data sets are empty and do not have any files on them. The Ubuntu installer does not support directly installing to a ZFS data set. There are two approaches you can take to install Ubuntu:


 * 1) Install Ubuntu to an alternate location:  Ubuntu can be installed to an alternate location, such as a supported unbootable ZFS volume and later copied over to the newly created ZFS data set.
 * 2) Install a minimal, base system of Ubuntu directly into ZFS: On non-graphical environments, Ubuntu can be installed directly into the ZFS data set but only a minimal system will be installed.

Typically, the second method can be used when a graphical environment is not avaliable, for instance, when installing Ubuntu server. The first method is slower than the second method but is easier to perform for novices since Ubuntu will configured and nearly ready to go once copied over. Both methods are documented in this article.

Method 1 - Graphic Install
If the Ubuntu live environment has graphics avaliable, the Ubuntu graphical installer can be used to install Ubuntu to an alternate location, and then moved over to the ZFS data set.

A small 20GB ZFS volume (known as a ZVOL) is created within the ZFS pool i.e. <pool-name>. You can use this command to create a ZVOL with 20GB of provisioned disk space. You may adjust the space allocation as desired

<pre style="font-weight:bold;">sudo zfs create -V 20G <pool-name>/ubuntu-installation

You might be curious to know, whether or not, you can simply boot Ubuntu by installing it here and rebooting. The problem is that the installer fails to install the boot loader. This is not a problem when moving the installation to a ZFS dataset, but the ZVOL itself is not bootable.

The graphical installer will detect zvol as though it is another hard disk drive. As such, you may use this installer to select that ZVOL device and install to it. This process will copy the files and configure Ubuntu. Since these files will be later copied, it it unnecessary to create additional partitions and mount points. Ubuntu can be installed to the root of the device.

The wizard will allow you to correctly complete timezone and username information without needing to manually edit text files. The error about the failed boot loader can be safely ignored, since the boot loader will have to be manually installed.

The ZVOL should be mounted, so files stored on it can be copied to the new ZFS data set. The /mnt directory is likely already in use due to having previously mounted the ZFS root data set. Therefore, the zvol should be mounted elsewhere

<pre style="font-weight:bold;">sudo mount /dev/zd0p1 /media

The rsync utility can be used to copy the Ubuntu installation to the ZFS data set. The command below can be used. You may need to change the directories to correspond with the correct mount points.

<pre style="font-weight:bold;">sudo rsync -avPX /media/. /mnt

Method 2 - Command Line Install
When installing from the command line, a minimal base install of Ubuntu is carried out using debootstrap. This tool can be installed from the terminal.

<pre style="font-weight:bold;">sudo apt-get install -y debootstrap

You can use the debootstrap utility to install a very minimal version of Ubuntu. Packages can then be installed as desired on top of the minimal environment, including a desktop environment. Debootstrap does not install a desktop environment and leaves configuration to the user.

You can use these commands to install Ubuntu into the /mnt directory ,

<pre style="font-weight:bold;">sudo chmod 1777 /mnt/var/tmp sudo debootstrap xenial /mnt sudo zfs set devices=off rpool

The system is unconfigured, so it is advisable to configure it to ensure smooth running of the system. Configuration changes need to be made to the config file in /mnt/etc/hosts. The IP address 127.0.1.1 should be bound to the local host name, or to the DNS name. Likewise, the network interface needs to be configured to use a valid IP address, or DHCP.

Configure New Ubuntu Instance
Once the files have been copied over, the chroot command is used to log in to our new Ubuntu instance. This is necessary to configure the boot loader and install ZFS tools within the new Ubuntu instance. If Ubuntu was booted in its current state, it would not boot.

1. Setup aliases
Chroot changes the root of the filesystem so the new Ubutu instance, instead of the live environment, will be updated. To facilitate network access and access to hardware, a binding needs to be created to redirect to the live CD's files. Currently, after using chroot, you will find that you cannot access the Internet from within that environment. To fix this problem, type the following commands:

<pre style="font-weight:bold;">sudo su for d in proc sys dev; do mount --bind /$d /mnt/$d; done chroot /mnt echo "nameserver 8.8.8.8" | tee -a /etc/resolv.conf

The chroot environment now has Internet access, allowing packages to be installed directly to the newly installed instance of Ubuntu. The commands above automatically switched into the new environment.

A. System Configuration (Non-Graphical Install)
If you used the non-graphical install method, you need to configure your system. Otherwise, Ubuntu will not run properly when it is booted. The graphical installer already completes this configuration, so in that case, you should skip to the next section, Install Boot Images.

1. Add Source Repositories
You need to add appropriate repositories to your /etc/sources.list file, so that software can be located and installed by the package manager. An Ubuntu configuration file generator is located here. You can copy and paste the configuration into your /etc/sources.list file.

After adding new packages, you should run apt-get update so the new packages can be used.

<pre style="font-weight:bold;">ln -s /proc/self/mounts /etc/mtab apt update

2. Set System Language
You need to set the appropriate language for your system. GitHub recommends having United States English always available, even if you live outside the United States.

<pre style="font-weight:bold;">locale-gen en_US.UTF-8 echo 'LANG="en_US.UTF-8"' > /etc/default/locale dpkg-reconfigure tzdata

3. Install Minimal Packages
You should install the minimal set of packages for your system

<pre style="font-weight:bold;">apt install --yes ubuntu-minimal apt install -y --no-install-recommends linux-image-generic

4. Setup Groups
System groups need to be configured on the newly installed system,

<pre style="font-weight:bold;">addgroup --system lpadmin addgroup --system sambashare

5. Root Password
A root password should be set for security purposes. It is advisable to use a password different from your username, since a compromised user password could provide an attacker with root access.

<pre style="font-weight:bold;">passwd

B. Install Boot Images
The ZFS and kernel boot images in the chroot environment need to be updated to support booting from ZFS. This requires the installation of the ZFS intramfs package.

<pre style="font-weight:bold;">apt install -y zfs-initramfs

Reconfigure & Install Boot Loader
The boot loader (i.e. GRUB) is required to be boot the new installation. The boot files do not reside inside the ZFS dataset, but on a small boot partition. The BIOS cannot see ZFS directly and so a separate partition is needed to kick start the boot process. The boot files in Ubuntu are located in /boot/. This directory, therefore, needs to be mapped to the correct boot partition, so that the files are physically created and updated on the correct partition.

There are two options for installing the boot loader depending on your needs:


 * Legacy Booting - The old MBR setup that lacks EFI support
 * UEFI - The more modern approach to booting. The system boots by looking for a small EFI partition. The MBR is not used.

Method 1: Legacy Booting
Legacy booting is simply a matter of installing GRUB using the following commands:

<pre style="font-weight:bold;">sudo apt install -y grub-pc update-initramfs -c -k all grub-install

Please Note:  Replace with the path to the main disk alias e.g. /dev/sda or /dev/disk/by-id/scsi-SATA_disk1''

Method 2: UEFI
UEFI booting requires a small partition formatted with the FAT32 filesystem and the appropriate boot files added to it. This can be achieved by mounting the physical boot partition to /boot, so that Ubuntu appropriately adds the files.

The /boot/ directory does not contain a required subdirectory, efi by default. Therefore, it is necessary to first create this directory by typing these commands:

<pre style="font-weight:bold;">sudo mkdir /boot/efi

Next, changes should be made to the /etc/fstab configuration file, so the boot partition is automatically mounted by the system at path /boot. This is the canonical path used by Ubuntu to look for boot files. It is important to do this to ensure that the correct boot files are updated. Replace <boot-partition> with the correct boot partition alias.

<pre style="font-weight:bold;">echo PARTUUID=$(blkid -s PARTUUID -o value /dev/disk/by-id/<boot-partition>) /boot/efi vfat defaults 0 1 >> /etc/fstab sudo mount /boot/efi sudo apt install -y grub-efi-amd64

To install UEFI-enabled GRUB to the EFI partition, you should run the following commands:

<pre style="font-weight:bold;">update-initramfs -c -k all grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=ubuntu --recheck --no-floppy