Chapter 5 — Storage: filesystems, LVM, LUKS, RAID, capacity

Identify before you touch

Block-device identity discipline comes first, because every destructive storage command starts with a device path (lsblk(8)):

# [READ-ONLY] topology/signatures vs on-disk tags vs actual mounts
lsblk -o NAME,TYPE,SIZE,FSTYPE,UUID,MOUNTPOINTS   # check available columns: lsblk --help
blkid
findmnt
findmnt --target /path          # which filesystem serves this path
findmnt --fstab                 # configured mounts (vs kernel's current view)
findmnt --fstab --verify --verbose   # parsability/usability, where supported

lsblk reads sysfs and udev for relationships; blkid is the more authoritative identifier of filesystem and RAID tags; recently changed hardware may require waiting for udev to settle. Default lsblk columns can change between versions, so scripts and runbooks should request explicit columns. A filesystem is not always a one-device/one-mount relationship — MOUNTPOINTS/SOURCES preserve that distinction (lsblk(8); findmnt(8)).

For persistent references, use filesystem UUID/LABEL or partition UUID rather than assuming /dev/sdX names remain stable, and inspect for duplicate tags before relying on them — findmnt --evaluate warns that duplicate tags can map to multiple devices (findmnt(8); systemd.mount(5)).

fstab and systemd mount units

systemd converts /etc/fstab entries into mount units at boot and whenever the system manager reloads configuration; a typo or wrong device/option can delay or disrupt boot (systemd.mount(5)). Inspection first, backup of the file, validate the table (findmnt --fstab --verify), test the intended mount, and only then reboot. On systemd hosts, reload the manager after editing fstab and inspect the generated mount unit/status. Note that nofail, noauto, _netdev, and x-systemd.* alter dependency/boot behavior and are not generic mount options guaranteed by every init system (systemd.mount(5)).

LVM, LUKS, MD RAID — inventory, then respect the warnings

Storage-layer inventory is read-only and layered (LVM2; mdadm(8); cryptsetup(8)):

# [READ-ONLY] layer by layer
pvs; vgs; lvs -a -o +devices        # LVM: physical volumes, volume groups, logical volumes
cat /proc/mdstat                    # MD RAID array status
mdadm --detail /dev/mdX
cryptsetup status NAME              # active dm-crypt mapping
cryptsetup luksDump DEVICE          # LUKS header: key slots, metadata

LVM distinguishes PVs, VGs, and LVs; RAID arrays are virtual block devices assembled from component devices; LUKS adds a metadata header and key slots. Do not conflate these layers when diagnosing.

HIGH-RISK · MUTATING · root — LVM mutation and resizing. pvcreate, pvremove, vgremove, vgreduce, lvremove, lvreduce, lvresize change storage metadata or capacity; shrinking an LV without first ensuring the filesystem/data fits can truncate the filesystem (LVM2). Preflight: capture pvs/vgs/lvs reports and VG metadata backup; identify every dependent filesystem/application; confirm free extents and the supported resize direction; make and verify a separate backup. Rollback: the verified backup. Success check: filesystem mounts, data intact, application works.

HIGH-RISK · MUTATING · root — LUKS format, erase, slot removal, header restore. luksFormat initializes a LUKS device; erase wipes every key slot, making the container inaccessible until a valid header is restored; luksKillSlot destroys a particular credential slot. A damaged header without a matching backup can permanently deny access to the encrypted data (cryptsetup(8)). Preflight: positive device identification (serial/WWN, size, mount state — never trust a placeholder like /dev/sdX as literal); confirm the intended contents; protect the passphrase/recovery material. Rollback: an offline header backup — make one, keep it protected. Success check: cryptsetup status opens the container and data is readable. Encryption does not replace backups.

HIGH-RISK · MUTATING · root — MD RAID create/reshape/fail/force/zero-superblock. mdadm --create writes array metadata and starts a resync; --grow can reshape; --fail/--remove change member state; --zero-superblock erases component signatures; --force assembly can start a degraded array whose data may be corrupt (mdadm(8)). Preflight: confirm which physical drives are in the array; preserve member order/metadata details and array state; avoid simultaneous drive replacement/reshape. Rollback: documented array state and member order. Success check: /proc/mdstat shows the expected state; rebuild progress monitored; data read back. Never use forced assembly as a routine fix.

RAID is not backup. RAID can provide availability for some device failures, but it provides no independent historical copies and cannot protect against deletion, corruption, theft, ransomware, or a site-wide event — it is a redundant block device, not a versioned backup system (mdadm(8); restic docs).

Capacity: bytes, inodes, and the files that hide

# [READ-ONLY]
df -hT /path        # filesystem block capacity + type ([df(1)](https://manpages.debian.org/bookworm/coreutils/df.1.en.html))
df -ih /path        # inode capacity
du -xhd1 /path      # [GNU] top-level usage on ONE filesystem
du --inodes         # [GNU] inode counts ([du(1)](https://manpages.debian.org/bookworm/coreutils/du.1.en.html))

Apparent size can differ from allocated usage, including because sparse files have holes (du(1)). If df says full but du cannot account for the space, check for deleted-but-open files and hidden data under mount points before deleting more: /proc/PID/fd exposes a process's open file descriptors (proc(5)), and lsof +L1 finds deleted-but-open files if lsof is installed (optional package; availability/output vary — lsof). Distinguish bytes, inodes, quotas, reserved space, and a full underlying filesystem from an application-specific limit.

Filesystem repair: identify the filesystem first

Filesystem repair is filesystem-specific and usually requires an unmounted filesystem. For ext2/3/4, e2fsck normally replays a journal when appropriate; running it on a mounted filesystem is generally unsafe and results invalid — even its nominally read-only -n output is not valid as a diagnosis while mounted (e2fsck(8)). Do not use fsck as a magical generic repair command: first identify the filesystem, then follow its own documentation.

HIGH-RISK · MUTATING · root — filesystem repair. e2fsck -y answers yes to every repair question and can rewrite metadata; -p only preens conditions judged safe without intervention (e2fsck(8)). Preflight: stop writes; identify the correct device; unmount as required; preserve an image/backup where possible; record tool version/output. Rollback: the image/backup. Success check: review exit status; mount read-only first if possible; verify data. A generalizing recipe: identify filesystem → unmount → consult that tool's manual → take an image/backup if data is valuable → repair. (This edition deliberately does not prescribe XFS/Btrfs/ZFS repair commands — see the source index's verification note.)