linux:zfs:raid

ZFS offers powerful software RAID features built into its storage pool (zpool) layer. Unlike traditional software RAID tools like mdadm, ZFS RAID is tightly integrated with the filesystem, offering data integrity, easier management, and better recovery options.

Here's a breakdown of the software RAID options ZFS supports:

ZFS RAID Options

1. Mirror (RAID-1)

Description: Data is duplicated across two or more disks. Best for redundancy and read performance.

sudo zpool create mymirror mirror /dev/sdX /dev/sdY

You can use more than two disks (mirror /dev/sdX /dev/sdY /dev/sdZ) – ZFS will mirror across all.
If one disk fails, data is safe.
Read speed is improved (ZFS reads in parallel from mirrors).

2. RAID-Z (Single Parity, RAID-5 Equivalent)

Description: Uses parity to recover from a single disk failure.

sudo zpool create myraid raidz /dev/sd{b,c,d}

Minimum 3 disks required.
If one disk fails, ZFS reconstructs data using parity.
Write performance is better than mdadm RAID-5 but not as good as mirrors.

3. RAID-Z2 (Double Parity, RAID-6 Equivalent)

Description: Tolerates two disk failures.

sudo zpool create myraid2 raidz2 /dev/sd{b,c,d,e,f}

Minimum 4 disks.
Parity is spread across the disks.
More redundancy but lower usable storage.

4. RAID-Z3 (Triple Parity)

Description: Tolerates three disk failures.

sudo zpool create myraid3 raidz3 /dev/sd{b,c,d,e,f,g}

Minimum 5 disks.
Extremely fault-tolerant. Useful for archival, enterprise systems, or critical NAS.

Comparison of RAID Levels in ZFS

Mirror	2	1 (or N-1)	50% (if 2 disks)	Fast reads, writes may scale linearly
RAID-Z	3	1	N - 1	Efficient, safe for moderate setups
RAID-Z2	4	2	N - 2	High safety, good for medium/large arrays
RAID-Z3	5	3	N - 3	Max safety, best for critical data

Important RAID Concepts in ZFS

`ashift`

-o ashift=12

Sets the sector size to 2^12 (4096 bytes). Best for modern disks (especially SSDs). You cannot change this later.

Add Hot Spares

ZFS can keep extra disks idle, ready to take over when another fails:

sudo zpool add mypool spare /dev/sdX

ZFS will automatically replace a failed disk with a spare.

🚨 Avoid This: Striped Pools Across Multiple Disks

sudo zpool create badpool /dev/sdX /dev/sdY

This creates a striped pool with no redundancy.
If any one disk fails, the entire pool is lost.

Never use multiple disks in a single pool without mirroring or RAID-Z.

—

🛡️ Replacing a Failed Disk in ZFS RAID

Check pool:

sudo zpool status

Offline the failed disk:

sudo zpool offline mypool /dev/sdX

Replace it:

sudo zpool replace mypool /dev/sdX /dev/sdY

Monitor resilvering:

sudo zpool status

ZFS will rebuild the data on the new disk automatically.

Test a RAID-Z Pool

sudo zpool create -o ashift=12 \
  -O compression=lz4 \
  -O atime=off \
  testpool raidz /dev/sd{b,c,d}

sudo zfs create testpool/files
echo "Hello ZFS RAID-Z" | sudo tee /testpool/files/test.txt
sudo zfs snapshot testpool/files@first

Then simulate a failure by detaching or offlining a disk and using zpool status, replace, and resilver.

snippet.cpp

by:
▖     ▘▖▖
▌ ▌▌▛▘▌▙▌
▙▖▙▌▙▖▌ ▌
 
edited: June 2025