11 ms·

FreeBSD Full Disk Encryption, with an External Boot Drive, GELI, and UFS (2015)

84 points by eatonphil 11y ago

616c 11y ago
Sounds very cool, as an almost life-long Linux user. Important question, that makes me suffer in other *nix platforms: does anyone run a rig like this AND it work properly with sleep/hibernation and resume?
- cperciva 11y ago
  Works fine with suspend-to-RAM. FreeBSD doesn't have software suspend-to-disk support (and it's almost never implemented by the BIOS).
  - yrro 11y ago
    Is BIOS support required for hibernation? I don't see why it's necessary; or rather I don't see why the firmware cares whether or not the operating system saved its state to swap before it powered the system off.
    - cperciva 11y ago
      When ACPI S4 was first created, there were two versions: "S4OS", and "S4BIOS". In "S4BIOS", the dumping of RAM to disk is handled by the BIOS rather than the OS; this makes the implementation much easier since it means that the OS disk driver can be suspended rather than needing to figure out how to dump itself (and worse, restore itself) to/from disk.
      - yrro 11y ago
        Thanks for that, I had no idea! It sounds like I've been using S4OS with Linux. Do you know what the difference is between S4BIOS and Intel Rapid Start? They sound very similar to me. I'm wondering how the BIOS identifies, when it powers on, that it should resume and where it should read the contents of RAM from.
        cperciva 11y ago
        Yes, S4BIOS hasn't really been a thing for a long time -- once Windows supported S4OS (and didn't use S4BIOS even if it existed, since they quite reasonably didn't trust BIOS authors to get it right) all the BIOS authors said "this seems like a useless feature" and stopped providing it. No idea about Intel Rapid Start. I don't really deal with that side of the system.
  - deleted 11y ago
    [deleted]
  - 616c 11y ago
    > FreeBSD doesn't have software suspend-to-disk support (and it's almost never implemented by the BIOS). Linux has spoied me so bad I never noticed the former, and I am shocked by the latter. Nice to get a response from a completely ignorant BSD and security notice, by the way. :-) I guess I could ego trip and say "how can I trust this user" but I suspect others will chuckle like me.
    - cperciva 11y ago
      I'm having trouble understanding this comment. Possibly due to typos?
      - chongli 11y ago
        I think this person is impressed that the founder of Tarsnap would respond to the question of a novice.
- andrey_utkin 11y ago
  Yes. I use similar setup with Linux' LUKS disk encryption and suspend/resume with encrypted swap partition works. You need to use initramfs with your kernel, so that kernel you boot from external drive is not alone but has some tools. Then you need to tell to your initramfs scripts to take care of encryption of your data and _swap_ partitions. Both dracut (redhat initramfs tool) and genkernel (gentoo initramfs tool) handle that. With genkernel, this is what I currently have in /etc/default/grub. That's all I need, really. GRUB_CMDLINE_LINUX="crypt_root=UUID=7e2c0396-7cc2-4468-9431-c85f81a4c9b2 root=UUID=d098a3ad-ff13-4a44-b4e6-d9e83cddd7ad crypt_swap=UUID=976f44a3-25c2-4869-b3b5-61808f3bc20d swap=UUID=5adcdf03-abaf-4cd3-8902-45228be2abe5 root_key=key swap_key=key root_keydev=UUID=d5878b0f-1997-467a-8f3b-65e78b16fb67 swap_keydev=UUID=d5878b0f-1997-467a-8f3b-65e78b16fb67 rootfstype=ext4 rootflags=rw,relatime,data=ordered resume=UUID=5adcdf03-abaf-4cd3-8902-45228be2abe5 " With dracut, you want to look at options having "luks" in their names.
  - 616c 11y ago
    I am aware of how to do this on Linux. I do exactly as you subscribe. But running the /boot on external USB seems to cause more issues than when I merely have /boot partition and a partition with LVM inside LUKS volume on the same physical disk. I want to know if FreeBSD works that way, as I hear some FreeBSD on FreeBSDTalk and other podcasts say they have made big strides on laptop support (jokes abound with resume/sleep issues) on different laptops, but I am curious how far these improvements go.
    - andrey_utkin 11y ago
      External boot enables me to carry encryption key on it secretly so that I don't need to enter it manually. With on-machine unencrypted boot, you cannot do like that, because it's pointless. Anyway, another important issue is Evil Maid Attack. To recap, if you leave your device alone for a moment, an adversary may edit your bootpath (BIOS/EFI, or just your boot partition) to insert keylogger. Then, as long as you enter your key once, the game is over. BIOS/EFI hacking is not possible if you have trusted stuff setup properly. Hacking an unencrypted boot partition is just piece of cake. I believe Linux or BSD can do whatever you want, just be careful with what you want.
- deleted 11y ago
  [deleted]
snvzz 11y ago
I kinda like how Dragonfly does implement dm-crypt such as on Linux (compatible) instead :).
CIPHERSTONE 11y ago
Great write up!
keithpeter 11y ago
"Optionally, at this point you might want to overwrite any preexisting data on your hard drives with zeroes or, preferably, random values." Am I being silly, or could a hard drive full of random data be confused with a hard drive full of encrypted data by some member of the security services in the future? Would zeros not be a better bet? The Debian installer attempts to write random data to the whole drive when you elect to encrypt the hard drive (except for the small /boot partition as mentioned in OA). I always quit out of that. My security needs are of the 'laptop left on bus' variety, not enterprise level.
- Terribledactyl 11y ago
  At that point, you are trying to remove the ability to read the old files on the disk you are about to use for new encrypted data. Wiping with all zeros will make it extremely difficult to recover files and random will be even more so. But by writing down zeros, you leak metadata about the encrypted parts of your drive. Zeros almost certainly don't contain data, but the noise will. If I can see you have 30gb of noise on a drive, rest zeros then I know something I probably shouldn't, namely that you have written at least 30gb to that drive. But that same 30gb of noise wouldn't be discernible in a drive full of random data. What you might be thinking of is when decommissioning the drive after having encrypted it. At this point, I don't think it matters. I'm not sure drives come with all zeroed status from the manufacturer anyways.
  - Someone 11y ago
    If you are concerned about leaking 'this was written earlier', you have to mimic what a factory-new disk looks like. It seems likely that is either all zeroes, all ones, or uniform noise, but is that true, and if so, which is it? I would not know. Does anybody on HN?
  - buy_used 11y ago
    Buy a used platter of off eBay or from your favorite reseller of refurbished equipment (and keep the receipt). Replace the disk's controller with one of your own (the one it came with was broken). Ummm, I dunno what the previous owner of this drive has done with it!
mioelnir 11y ago
I've switched back from such a setup to having an unencrypted UFS boot partition. The external boot medium works nicely, but then you think about a USB stick being your SPOF too much, then you create backup sticks. Which means you have to secure them at least as good as the one "you are always carrying". But you can not simply carry the backup as well, as that would subject it to the same wear and tear. Which spawns the second backup. It is a rabbit hole. That said, I am really looking forward to Allan Jude's last patches hitting CURRENT, at which point I will rebuild that Laptop with UEFI booted, full disk encrypted root on ZFS without any unencrypted bootpartition. There will still be the issue of the unencrypted bootloader that does the initial decryption to load the kernel, but that will likely end up an irrelevant attack vector compared to the Laptop's BIOS anyway...
- simonebrunozzi 11y ago
  Which laptop do you use? I love FreeBSD but have always failed at installing it properly on a laptop (Macbook dual boot, or Dell XPS 13).
  - mioelnir 11y ago
    It's a Dell Latitude E6430 aka whatever my workplace gave out at the time I got a new one. Which means I can not claim that any sort of research went into choosing this one or that that it runs FreeBSD dis-proportionally well - which it does not. It just runs it, with the occasional hickup here and there. For example on boot, it always prints that the boot device is invalid, since it gets confused by finding GPT partitioning without UEFI. Then you press enter and it happily continues to boot... I never fully configured the UMTS modem in it I think, that entire class of hardware is just soul crushingly awful. And as I am running CURRENT, every once in a while there is commit that throws i915kms into a suspend/resume tantrum (refusing to shut down, not coming back up). But since I do actually power down the laptop instead of suspending it, so that the disk encryption actually has a chance to help me out, I am not too bothered by it. My use case for the laptop is mainly being a glorified ssh terminal multiplexer. It just needs to provide a comfortable, familiar runtime that I can navigate blindly at 3am, still partially asleep and without any coffee. All the tools need to be there, in the correct drawer, with the correct label and all the right marker labels on the man pages. I tried running a Linux laptop/workstation combination at work since that is what our boxes predominantly run, but it just feels like running around in somebody else's slippers all day. Sorry for oversharing.
xenophonf 11y ago
And here's how you do the same thing, only with ZFS: https://web.archive.org/web/20120606231347/https://web.irtnog.org/Members/xenophon/blog/revised-freebsd-root-zfs-geli https://web.archive.org/web/20120606231347/https://web.irtno... (Sorry---I'd posted this to my now-defunct blog back in 2012.)
- xenophonf 11y ago
  Here's how you do something similar, only in a dual-boot setup with Mac OS X (FileVault2) and Ubuntu (OpenZFS/LUKS): https://gist.github.com/xenophonf/2e2d1a1550b0fb8dae98 https://gist.github.com/xenophonf/2e2d1a1550b0fb8dae98
z0xcd 11y ago
Question: when installing FreeBSD 11, I can automatically encrypt the disk in the install if I use ZFS, using UFS doesn't work, I would have to do that manually. Why didn't they implement this already?
- mioelnir 11y ago
  I think the encrypted ZFS installer bits were upstreamed by PCBSD which is ZFS-only; so it would make sense for them not to implement the UFS version. Apart from that, probably nobody needed it bad enough to sit down and write it so far.

84 points by eatonphil 11y ago

616c 11y ago

Sounds very cool, as an almost life-long Linux user. Important question, that makes me suffer in other *nix platforms: does anyone run a rig like this AND it work properly with sleep/hibernation and resume?

cperciva 11y ago
Works fine with suspend-to-RAM. FreeBSD doesn't have software suspend-to-disk support (and it's almost never implemented by the BIOS).
- yrro 11y ago
  Is BIOS support required for hibernation? I don't see why it's necessary; or rather I don't see why the firmware cares whether or not the operating system saved its state to swap before it powered the system off.
  - cperciva 11y ago
    When ACPI S4 was first created, there were two versions: "S4OS", and "S4BIOS". In "S4BIOS", the dumping of RAM to disk is handled by the BIOS rather than the OS; this makes the implementation much easier since it means that the OS disk driver can be suspended rather than needing to figure out how to dump itself (and worse, restore itself) to/from disk.
    - yrro 11y ago
      Thanks for that, I had no idea! It sounds like I've been using S4OS with Linux. Do you know what the difference is between S4BIOS and Intel Rapid Start? They sound very similar to me. I'm wondering how the BIOS identifies, when it powers on, that it should resume and where it should read the contents of RAM from.
      - cperciva 11y ago
        Yes, S4BIOS hasn't really been a thing for a long time -- once Windows supported S4OS (and didn't use S4BIOS even if it existed, since they quite reasonably didn't trust BIOS authors to get it right) all the BIOS authors said "this seems like a useless feature" and stopped providing it. No idea about Intel Rapid Start. I don't really deal with that side of the system.
- deleted 11y ago
  [deleted]
- 616c 11y ago
  > FreeBSD doesn't have software suspend-to-disk support (and it's almost never implemented by the BIOS). Linux has spoied me so bad I never noticed the former, and I am shocked by the latter. Nice to get a response from a completely ignorant BSD and security notice, by the way. :-) I guess I could ego trip and say "how can I trust this user" but I suspect others will chuckle like me.
  - cperciva 11y ago
    I'm having trouble understanding this comment. Possibly due to typos?
    - chongli 11y ago
      I think this person is impressed that the founder of Tarsnap would respond to the question of a novice.
andrey_utkin 11y ago
Yes. I use similar setup with Linux' LUKS disk encryption and suspend/resume with encrypted swap partition works. You need to use initramfs with your kernel, so that kernel you boot from external drive is not alone but has some tools. Then you need to tell to your initramfs scripts to take care of encryption of your data and _swap_ partitions. Both dracut (redhat initramfs tool) and genkernel (gentoo initramfs tool) handle that. With genkernel, this is what I currently have in /etc/default/grub. That's all I need, really. GRUB_CMDLINE_LINUX="crypt_root=UUID=7e2c0396-7cc2-4468-9431-c85f81a4c9b2 root=UUID=d098a3ad-ff13-4a44-b4e6-d9e83cddd7ad crypt_swap=UUID=976f44a3-25c2-4869-b3b5-61808f3bc20d swap=UUID=5adcdf03-abaf-4cd3-8902-45228be2abe5 root_key=key swap_key=key root_keydev=UUID=d5878b0f-1997-467a-8f3b-65e78b16fb67 swap_keydev=UUID=d5878b0f-1997-467a-8f3b-65e78b16fb67 rootfstype=ext4 rootflags=rw,relatime,data=ordered resume=UUID=5adcdf03-abaf-4cd3-8902-45228be2abe5 " With dracut, you want to look at options having "luks" in their names.
- 616c 11y ago
  I am aware of how to do this on Linux. I do exactly as you subscribe. But running the /boot on external USB seems to cause more issues than when I merely have /boot partition and a partition with LVM inside LUKS volume on the same physical disk. I want to know if FreeBSD works that way, as I hear some FreeBSD on FreeBSDTalk and other podcasts say they have made big strides on laptop support (jokes abound with resume/sleep issues) on different laptops, but I am curious how far these improvements go.
  - andrey_utkin 11y ago
    External boot enables me to carry encryption key on it secretly so that I don't need to enter it manually. With on-machine unencrypted boot, you cannot do like that, because it's pointless. Anyway, another important issue is Evil Maid Attack. To recap, if you leave your device alone for a moment, an adversary may edit your bootpath (BIOS/EFI, or just your boot partition) to insert keylogger. Then, as long as you enter your key once, the game is over. BIOS/EFI hacking is not possible if you have trusted stuff setup properly. Hacking an unencrypted boot partition is just piece of cake. I believe Linux or BSD can do whatever you want, just be careful with what you want.
deleted 11y ago
[deleted]

snvzz 11y ago

I kinda like how Dragonfly does implement dm-crypt such as on Linux (compatible) instead :).

CIPHERSTONE 11y ago

Great write up!

keithpeter 11y ago

"Optionally, at this point you might want to overwrite any preexisting data on your hard drives with zeroes or, preferably, random values." Am I being silly, or could a hard drive full of random data be confused with a hard drive full of encrypted data by some member of the security services in the future? Would zeros not be a better bet? The Debian installer attempts to write random data to the whole drive when you elect to encrypt the hard drive (except for the small /boot partition as mentioned in OA). I always quit out of that. My security needs are of the 'laptop left on bus' variety, not enterprise level.

Terribledactyl 11y ago
At that point, you are trying to remove the ability to read the old files on the disk you are about to use for new encrypted data. Wiping with all zeros will make it extremely difficult to recover files and random will be even more so. But by writing down zeros, you leak metadata about the encrypted parts of your drive. Zeros almost certainly don't contain data, but the noise will. If I can see you have 30gb of noise on a drive, rest zeros then I know something I probably shouldn't, namely that you have written at least 30gb to that drive. But that same 30gb of noise wouldn't be discernible in a drive full of random data. What you might be thinking of is when decommissioning the drive after having encrypted it. At this point, I don't think it matters. I'm not sure drives come with all zeroed status from the manufacturer anyways.
- Someone 11y ago
  If you are concerned about leaking 'this was written earlier', you have to mimic what a factory-new disk looks like. It seems likely that is either all zeroes, all ones, or uniform noise, but is that true, and if so, which is it? I would not know. Does anybody on HN?
- buy_used 11y ago
  Buy a used platter of off eBay or from your favorite reseller of refurbished equipment (and keep the receipt). Replace the disk's controller with one of your own (the one it came with was broken). Ummm, I dunno what the previous owner of this drive has done with it!

mioelnir 11y ago

I've switched back from such a setup to having an unencrypted UFS boot partition. The external boot medium works nicely, but then you think about a USB stick being your SPOF too much, then you create backup sticks. Which means you have to secure them at least as good as the one "you are always carrying". But you can not simply carry the backup as well, as that would subject it to the same wear and tear. Which spawns the second backup. It is a rabbit hole. That said, I am really looking forward to Allan Jude's last patches hitting CURRENT, at which point I will rebuild that Laptop with UEFI booted, full disk encrypted root on ZFS without any unencrypted bootpartition. There will still be the issue of the unencrypted bootloader that does the initial decryption to load the kernel, but that will likely end up an irrelevant attack vector compared to the Laptop's BIOS anyway...

simonebrunozzi 11y ago
Which laptop do you use? I love FreeBSD but have always failed at installing it properly on a laptop (Macbook dual boot, or Dell XPS 13).
- mioelnir 11y ago
  It's a Dell Latitude E6430 aka whatever my workplace gave out at the time I got a new one. Which means I can not claim that any sort of research went into choosing this one or that that it runs FreeBSD dis-proportionally well - which it does not. It just runs it, with the occasional hickup here and there. For example on boot, it always prints that the boot device is invalid, since it gets confused by finding GPT partitioning without UEFI. Then you press enter and it happily continues to boot... I never fully configured the UMTS modem in it I think, that entire class of hardware is just soul crushingly awful. And as I am running CURRENT, every once in a while there is commit that throws i915kms into a suspend/resume tantrum (refusing to shut down, not coming back up). But since I do actually power down the laptop instead of suspending it, so that the disk encryption actually has a chance to help me out, I am not too bothered by it. My use case for the laptop is mainly being a glorified ssh terminal multiplexer. It just needs to provide a comfortable, familiar runtime that I can navigate blindly at 3am, still partially asleep and without any coffee. All the tools need to be there, in the correct drawer, with the correct label and all the right marker labels on the man pages. I tried running a Linux laptop/workstation combination at work since that is what our boxes predominantly run, but it just feels like running around in somebody else's slippers all day. Sorry for oversharing.

xenophonf 11y ago

And here's how you do the same thing, only with ZFS: https://web.archive.org/web/20120606231347/https://web.irtnog.org/Members/xenophon/blog/revised-freebsd-root-zfs-geli https://web.archive.org/web/20120606231347/https://web.irtno... (Sorry---I'd posted this to my now-defunct blog back in 2012.)

xenophonf 11y ago
Here's how you do something similar, only in a dual-boot setup with Mac OS X (FileVault2) and Ubuntu (OpenZFS/LUKS): https://gist.github.com/xenophonf/2e2d1a1550b0fb8dae98 https://gist.github.com/xenophonf/2e2d1a1550b0fb8dae98

z0xcd 11y ago

Question: when installing FreeBSD 11, I can automatically encrypt the disk in the install if I use ZFS, using UFS doesn't work, I would have to do that manually. Why didn't they implement this already?

mioelnir 11y ago
I think the encrypted ZFS installer bits were upstreamed by PCBSD which is ZFS-only; so it would make sense for them not to implement the UFS version. Apart from that, probably nobody needed it bad enough to sit down and write it so far.