Wednesday, July 25, 2018

Encrypting ZFS drives on Linux using LUKS

This isn't really relevant to iPhones but then again neither was my last post on Kindle Fire sticks.  Today we're going to replace a failing hard drive in a ZFS pool, but this time the disk will be encrypted underneath ZFS, using LUKS.  I should have done this when I created the pool in the first place but I didn't know LUKS well enough back then.  With a couple years of ZFS-on-LUKS backup experience under my belt now I'm much more confident, so all new disks are getting this treatment, so that if I ever have to send a disk to the manufacturer for a warranty claim I won't worry about any of my data being exposed.  The rest of my HDDs are out of warranty anyway so there's no need to go back and redo them.

Note the prompt.  Most of this has to be done as root (the '#' prompt) but where root isn't necessary I'm doing it as a normal user (the '$' prompt).

Frankly this is as much for my own reference a for anyone else so if what I've said above is gibberish feel free to skip this post and wait for my next one.  Ready? Here comes the geekery...

  1. Run parted and create a new disklabel.  
    1. parted /dev/sdX (where 'X'... aww, hell, if I have to explain that you shouldn't be here)
    2. # parted /dev/sdh
      GNU Parted 3.2
      Using /dev/sdh
      Welcome to GNU Parted! Type 'help' to view a list of commands.
      (parted) p                                                                
      Error: /dev/sdh: unrecognised disk label
      Model: US HDD Docking (scsi)                                              
      Disk /dev/sdh: 4001GB
      Sector size (logical/physical): 512B/512B
      Partition Table: unknown
      Disk Flags: 
      (parted) mklabel gpt
      (parted) p                                                                
      Model: US HDD Docking (scsi)
      Disk /dev/sdh: 4001GB
      Sector size (logical/physical): 512B/512B
      Partition Table: gpt
      Disk Flags: 

      Number  Start  End  Size  File system  Name  Flags

      (parted)

  2. Create a partition on the disk starting at 1049kB. Why there? You want to leave a little bit of space for slight variances in makes and models of disk.  You can leave a bigger buffer, maybe even 1GB, but from what I've seen and read 1MB is enough.  Also, I name the partition after the manufacturer, model and serial number.  e.g: zraid-HGST_HUS696969ALAC64_PASTASTFU.
    1. (parted) mkpart zraid-HGST_HUS696969ALAC64_PASTASTFU 1049kb 100%
      Warning: failed to translate partition name
      (parted) p                                                                
      Model: US HDD Docking (scsi)
      Disk /dev/sdh: 4001GB
      Sector size (logical/physical): 512B/512B
      Partition Table: gpt
      Disk Flags: 

      Number  Start   End     Size    File system  Name                                       Flags
       1      1049kB  4001GB  4001GB               zraid-HGST_HUS696969ALAC64_PASTASTFU
      (parted)

  3. Generate an encryption key for LUKS and format your new partition with it. I'm using the same key for all of my z-RAID disks but you can use different ones if you want.  I figure if someone gets a hold of one of my keys they'll get all of them so why overly complicate things?  Make sure you save this key someone other than on the system in question.  Got a password keeper?  Good, put it there.  (If not, why not?!)  Or wrap it in an encrypted zip file, PGP it, etc and put it somewhere you trust and won't forget.  Have faith in a cloud vendor?   Well, you're a braver soul than I, but you can use that.  I keep meaning to look into something like tarsnap.  Suggestions? 
    1. # dd if=/dev/urandom of=/root/your.key bs=1k count=64
      # chmod 600 /root/your.key

    2. # cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha512 /dev/disk/by-id/ata-HGST_HUS696969ALAC64_PASTASTFU-part1 /root/luks-zraid.key
      WARNING!
    1. ========
      This will overwrite data on /dev/disk/by-id/ata-HGST_HUS696969ALAC64_PASTASTFU-part1 irrevocably.

      Are you sure? (Type uppercase yes): YES



  4. Find the UUID for your new partition in /dev/disk/by-id then use that to mount it.  You could mount it by its ID but you'll need the UUID for /etc/crypttab so it's better to verify everything works this way.  The last parameter is what will appear in /dev/mapper and how ZFS will identify it.  That doesn't have to match the partition label but self-documentation can be a sanity saver. 
    1. $ ls -l /dev/disk/by-uuid | grep sdh1
      lrwxrwxrwx 1 root root 10 Jul 25 19:43 aa173e06-8675-309-abbababba -> ../../sdh1

    2. cryptsetup open --type luks /dev/disk/by-uuid/aa173e06-8675-309-abbababba --key-file/root/your.key zraid-HGST_HUS696969ALAC64_PASTASTFU

    3. $ ls -l /dev/mapper
    4. total 0
      crw------- 1 root root 10, 236 Jul 25 00:50 control
      lrwxrwxrwx 1 root root       7 Jul 25 19:49 zraid-HGST_HUS696969ALAC64_PASTASTFU -> ../dm-0
  5. Update /etc/crypttab so that the volume mounts at boot-time:
    1. $ cat /etc/crypttab
      zraid-HGST_HUS696969ALAC64_PASTASTFU UUID=aa173e06-8675-309-abbababba /root/your.key luks
      $

  6. Replace the old (failing, unencrypted, small, whatever) disk with the new one, using the new disk's encrypted volume, which is now mounted in /dev/mapper.  'ashift=12' tells ZFS to use 4k blocks instead of 512k, since most disks still lie about their blocksize.
    1. # zpool replace -o ashift=12 tank ata-ST3000DM001-1ER169_BADHDD-part1 /dev/mapper/zraid-HGST_HUS696969ALAC64_PASTASTFU
      #
  7. Monitor progress as a user other than root.  My old disk is failing hard and fast so resilvering is glacial.  I don't mind if it takes a few days, but at the current clip it's looking more like three months.  If that doesn't improve soon I'll fail the old disk manually and let ZFS rebuild the data from parity.  It's a RAID-Z2 so as long as I don't lose two more disks before recon completes my data is safe.
    1. $ zpool status
        pool: tank
       state: ONLINE
      status: One or more devices is currently being resilvered.  The pool will
              continue to function, possibly in a degraded state.
      action: Wait for the resilver to complete.
        scan: resilver in progress since Wed Jul 25 20:13:03 2018
          27.8M scanned out of 9.15T at 241K/s, (scan is slow, no estimated time)
          3.32M resilvered, 0.00% done
      config:

              NAME                                             STATE     READ WRITE CKSUM
              tank                                             ONLINE       0     0     0
                raidz2-0                                       ONLINE       0     0     0
                  ata-ST3000DM001-NEXT_DEADDISK3E              ONLINE       0     0     0
                  ata-TOSHIBA_MADCOWA400_YYZRGBBLAHA           ONLINE       0     0     0
                  zraid-HGST_HDN23455VALE614_HVYMETAL          ONLINE       0     0     0
                  ata-ST4000VN000-2AH302_NEWSGHDD              ONLINE       0     0     0
                  replacing-4                                  ONLINE       0     0     0
                    ata-ST3000DM001-1ER169_BADHDD-part1        ONLINE       0     0     4
                    zraid-HGST_HUS696969ALAC64_PASTASTFU       ONLINE       0     0     0  (resilvering)
              logs
                mirror-1                                       ONLINE       0     0     0
                  ata-OCZ-AGILITY4_OCZ-ZIPPYZIPPYBOOTUP-part4  ONLINE       0     0     0
                  wwn-0x255c302351400460-part4                 ONLINE       0     0     0
              cache
                ata-OCZ-AGILITY4_OCZ-ZIPPYZIPPYBOOTUP-part3    ONLINE       0     0     0
                ata-ST240HM000-1G5152_SGTSSDOK-part3           ONLINE       0     0     0
      $
  8. Celebrate!

No comments:

Post a Comment

Comments and discussion are warmly welcomed.
The admins have enabled moderation to ward off evil and spammers (but I repeat myself) so if you’re a spammer your posts will never stick. Please don’t waste your time or ours. Thanks.