How to expand ZFS on FreeBSD

Running out of disk space is one of the leading causes of IT outages. In this blog post, I will show you how to expand storage on FreeBSD with ZFS. ZFS works as volume manager and filesystem.

Current State 

I have VMware based Virtual Machine with FreeBSD Operating System. Virtual Machine has 10 GB vDisk as clearly visible in geom report ...

 root@iredmail:~/iRedMail-1.7.4 # geom disk list da0  
 Geom name: da0  
 Providers:  
 1. Name: da0  
   Mediasize: 10737418240 (10G)  
   Sectorsize: 512  
   Stripesize: 1048576  
   Stripeoffset: 0  
   Mode: r2w2e3  
   descr: VMware Virtual disk  
   ident: (null)  
   rotationrate: 0  
   fwsectors: 63  
   fwheads: 255  
 root@iredmail:~/iRedMail-1.7.4 #  

Let's check partitions in da0 ...

 root@iredmail:~/iRedMail-1.7.4 # gpart show da0  
 =>     40 20971440 da0  GPT (10G)  
        40     1024   1  freebsd-boot (512K)  
      1064      984      - free - (492K)  
      2048  4194304   2  freebsd-swap (2.0G)  
   4196352 16773120   3  freebsd-zfs (8.0G)  
  20969472     2008      - free - (1.0M)  
 root@iredmail:~/iRedMail-1.7.4 #   

Size of ZFS partition is 8 GB.

Let's check ZFS status ...

 root@iredmail:~/iRedMail-1.7.4 # zpool status zroot  
   pool: zroot  
  state: ONLINE  
 config:  
         NAME      STATE   READ WRITE CKSUM  
         zroot     ONLINE     0     0     0  
           da0p3   ONLINE     0     0     0  
 errors: No known data errors  
 root@iredmail:~/iRedMail-1.7.4 # zfs list zroot  
 NAME   USED  AVAIL  REFER  MOUNTPOINT  
 zroot 7.27G     0B    96K  /zroot  
 root@iredmail:~/iRedMail-1.7.4 #   

We have 0B available capacity on /zroot mount point where is our FreeBSD operating system, therefore we need expand the storage.

Storage Expansion

Let's start with disk expansion on VMware. We will expand our 10 GB virtual disk to 20 GB and update the disk size within FreeBSD by command camcontrol reprobe da0

 root@iredmail:~/iRedMail-1.7.4 # camcontrol reprobe da0  
 root@iredmail:~/iRedMail-1.7.4 # geom disk list da0  
 Geom name: da0  
 Providers:  
 1. Name: da0  
   Mediasize: 21474836480 (20G)  
   Sectorsize: 512  
   Stripesize: 1048576  
   Stripeoffset: 0  
   Mode: r3w3e4  
   descr: VMware Virtual disk  
   ident: (null)  
   rotationrate: 0  
   fwsectors: 63  
   fwheads: 255  
 root@iredmail:~/iRedMail-1.7.4 #   

After disk expansion we can double check partitions on da0.

 root@iredmail:~/iRedMail-1.7.4 # gpart show da0  
 =>     40 41942960 da0 GPT (20G)  
        40     1024   1 freebsd-boot (512K)  
      1064      984     - free - (492K)  
      2048  4194304   2 freebsd-swap (2.0G)  
   4196352 16773120   3 freebsd-zfs (8.0G)  
  20969472 20973528     - free - (10G)  
 root@iredmail:~/iRedMail-1.7.4 #   

Now we have additional 10 GB free space on da0 and we need to extend partition da0p3 which participate in ZFS pool zroot. This is where gpart resize come in to play.

 root@iredmail:~/iRedMail-1.7.4 # gpart resize -i 3 -a 4k da0  
 da0p3 resized  
 root@iredmail:~/iRedMail-1.7.4 # gpart show da0  
 =>     40 41942960 da0 GPT (20G)  
        40     1024   1 freebsd-boot (512K)  
      1064      984     - free - (492K)  
      2048  4194304   2 freebsd-swap (2.0G)  
   4196352 37746648   3 freebsd-zfs (18G)  
 root@iredmail:~/iRedMail-1.7.4 #   

Here is explanation of gpart resize options 

• -i 3 
• Specifies the partition index to resize. In your case, it's 3.
• -a 4k 
• This specifies the alignment. ZFS works best with 4k sector alignment, so it's a good practice to include it.
• da0
• The disk device name
• Important Note 
• You don't need to specify the new size (-s option) because by omitting it, gpart will automatically expand the partition to fill all the available free space at the end of the disk

Perfect, partition da0p3 is expanded, but we need to tell ZFS to expand the pool to use the newly available space on its underlying device. The command to do this is zpool online with the -e flag.

 root@iredmail:~/iRedMail-1.7.4 # zpool online -e zroot da0p3  
 root@iredmail:~/iRedMail-1.7.4 # zfs list zroot  
 NAME    USED  AVAIL  REFER  MOUNTPOINT  
 zroot  7.27G  9.69G    96K  /zroot  
 root@iredmail:~/iRedMail-1.7.4 #   

Now we see that our zroot dataset has almost 10 GB available space. That's exactly what we wanted to achieve. Job done.