Note the prompt. Most of this has to be done as root (the '#' prompt) but where root isn't necessary I'm doing it as a normal user (the '$' prompt).
Frankly this is as much for my own reference a for anyone else so if what I've said above is gibberish feel free to skip this post and wait for my next one. Ready? Here comes the geekery...
- Run parted and create a new disklabel.
- parted /dev/sdX (where 'X'... aww, hell, if I have to explain that you shouldn't be here)
- # parted /dev/sdhGNU Parted 3.2Using /dev/sdhWelcome to GNU Parted! Type 'help' to view a list of commands.(parted) pError: /dev/sdh: unrecognised disk labelModel: US HDD Docking (scsi)Disk /dev/sdh: 4001GBSector size (logical/physical): 512B/512BPartition Table: unknownDisk Flags:(parted) mklabel gpt(parted) pModel: US HDD Docking (scsi)Disk /dev/sdh: 4001GBSector size (logical/physical): 512B/512BPartition Table: gptDisk Flags:Number Start End Size File system Name Flags(parted)
- Create a partition on the disk starting at 1049kB. Why there? You want to leave a little bit of space for slight variances in makes and models of disk. You can leave a bigger buffer, maybe even 1GB, but from what I've seen and read 1MB is enough. Also, I name the partition after the manufacturer, model and serial number. e.g: zraid-HGST_HUS696969ALAC64_PASTASTFU.
- (parted) mkpart zraid-HGST_HUS696969ALAC64_PASTASTFU 1049kb 100%Warning: failed to translate partition name(parted) pModel: US HDD Docking (scsi)Disk /dev/sdh: 4001GBSector size (logical/physical): 512B/512BPartition Table: gptDisk Flags:Number Start End Size File system Name Flags1 1049kB 4001GB 4001GB zraid-HGST_HUS696969ALAC64_PASTASTFU
- Generate an encryption key for LUKS and format your new partition with it. I'm using the same key for all of my z-RAID disks but you can use different ones if you want. I figure if someone gets a hold of one of my keys they'll get all of them so why overly complicate things? Make sure you save this key someone other than on the system in question. Got a password keeper? Good, put it there. (If not, why not?!) Or wrap it in an encrypted zip file, PGP it, etc and put it somewhere you trust and won't forget. Have faith in a cloud vendor? Well, you're a braver soul than I, but you can use that. I keep meaning to look into something like tarsnap. Suggestions?
- # dd if=/dev/urandom of=/root/your.key bs=1k count=64# chmod 600 /root/your.key
- # cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha512 /dev/disk/by-id/ata-HGST_HUS696969ALAC64_PASTASTFU-part1 /root/luks-zraid.key
========This will overwrite data on /dev/disk/by-id/ata-HGST_HUS696969ALAC64_PASTASTFU-part1 irrevocably.Are you sure? (Type uppercase yes): YES#
- Find the UUID for your new partition in /dev/disk/by-id then use that to mount it. You could mount it by its ID but you'll need the UUID for /etc/crypttab so it's better to verify everything works this way. The last parameter is what will appear in /dev/mapper and how ZFS will identify it. That doesn't have to match the partition label but self-documentation can be a sanity saver.
- $ ls -l /dev/disk/by-uuid | grep sdh1lrwxrwxrwx 1 root root 10 Jul 25 19:43 aa173e06-8675-309-abbababba -> ../../sdh1
- # cryptsetup open --type luks /dev/disk/by-uuid/aa173e06-8675-309-abbababba --key-file/root/your.key zraid-HGST_HUS696969ALAC64_PASTASTFU
- $ ls -l /dev/mapper
total 0crw------- 1 root root 10, 236 Jul 25 00:50 controllrwxrwxrwx 1 root root 7 Jul 25 19:49 zraid-HGST_HUS696969ALAC64_PASTASTFU -> ../dm-0
- Update /etc/crypttab so that the volume mounts at boot-time:
- $ cat /etc/crypttabzraid-HGST_HUS696969ALAC64_PASTASTFU UUID=aa173e06-8675-309-abbababba /root/your.key luks$
- Replace the old (failing, unencrypted, small, whatever) disk with the new one, using the new disk's encrypted volume, which is now mounted in /dev/mapper. 'ashift=12' tells ZFS to use 4k blocks instead of 512k, since most disks still lie about their blocksize.
- # zpool replace -o ashift=12 tank ata-ST3000DM001-1ER169_BADHDD-part1 /dev/mapper/zraid-HGST_HUS696969ALAC64_PASTASTFU#
- Monitor progress as a user other than root. My old disk is failing hard and fast so resilvering is glacial. I don't mind if it takes a few days, but at the current clip it's looking more like three months. If that doesn't improve soon I'll fail the old disk manually and let ZFS rebuild the data from parity. It's a RAID-Z2 so as long as I don't lose two more disks before recon completes my data is safe.
- $ zpool statuspool: tankstate: ONLINEstatus: One or more devices is currently being resilvered. The pool willcontinue to function, possibly in a degraded state.action: Wait for the resilver to complete.scan: resilver in progress since Wed Jul 25 20:13:03 201827.8M scanned out of 9.15T at 241K/s, (scan is slow, no estimated time)3.32M resilvered, 0.00% doneconfig:NAME STATE READ WRITE CKSUMtank ONLINE 0 0 0raidz2-0 ONLINE 0 0 0ata-ST3000DM001-NEXT_DEADDISK3E ONLINE 0 0 0ata-TOSHIBA_MADCOWA400_YYZRGBBLAHA ONLINE 0 0 0zraid-HGST_HDN23455VALE614_HVYMETAL ONLINE 0 0 0ata-ST4000VN000-2AH302_NEWSGHDD ONLINE 0 0 0replacing-4 ONLINE 0 0 0ata-ST3000DM001-1ER169_BADHDD-part1 ONLINE 0 0 4zraid-HGST_HUS696969ALAC64_PASTASTFU ONLINE 0 0 0 (resilvering)logsmirror-1 ONLINE 0 0 0ata-OCZ-AGILITY4_OCZ-ZIPPYZIPPYBOOTUP-part4 ONLINE 0 0 0wwn-0x255c302351400460-part4 ONLINE 0 0 0cacheata-OCZ-AGILITY4_OCZ-ZIPPYZIPPYBOOTUP-part3 ONLINE 0 0 0ata-ST240HM000-1G5152_SGTSSDOK-part3 ONLINE 0 0 0$