5 ms·

Looking at the article, I like the Drop trait, as a way to ensure memory are cleaned before being released. What is the preferred way in Rust to say "If at all

Looking at the article, I like the Drop trait, as a way to ensure memory are cleaned before being released. What is the preferred way in Rust to say "If at all possible, please do not allow this value/object/whatever to be paged out of RAM?"

Grollicus 8y ago
With security mindset, "if at all possible do not allow" is the basically the same as "always allow". As mlock / VirtualProtect are OS dependent I strongly suspect this is not part of Rust itself. Additionally, I suspect there would be some design problems for example putting this protected thing on the stack (protect the whole stack? Or just parts? Bookkeeping wether a stack lock can be released after a function exits?). How do you move values into / out of such an object? So I don't think this exists, but I've been pleasantly surprised by Rust before. My general solution for this case is to either mlockall the whole process if it is small / short-living or to have no swap in the machine, for example long running crypto servers. Who runs servers with swap anyways?
- deleted 8y ago
  [deleted]
monocasa 8y ago
Those system calls don't really work like that; there isn't really a public way to say "don't save a copy of this page on disk" on Windows, OSX, or Linux. For instance see Raymond Chen's explanation here > Is VirtualLock safe to use for cryptographic purposes – that is, to avoid sensitive memory being written to disk? > ... > [There does not appear to be any guarantee that the memory won't be written to disk while locked. As you noted, the machine may be hibernated, or it may be running in a VM that gets snapshotted. -Raymond] VirtualLock and the equivalents only guarantee that the canonical copy of that page stays in RAM, not that other copies won't make their way out to disk. Edit: add link https://blogs.msdn.microsoft.com/oldnewthing/20140207-00/?p=1833 https://blogs.msdn.microsoft.com/oldnewthing/20140207-00/?p=...
codys 8y ago
One thing to note is that it is not guaranteed that values are ever dropped. In rust, it is "safe" to leak memory (it wasn't possible to make it unsafe in rust).
- zozbot123 8y ago
  AIUI, this should only come up in rare cases, mostly involving RC cycles. The "should" proviso is there because the operation of "forgetting" that a value has to be dropped (thus de-facto leaking it) is in fact allowed in Safe Rust, and it's not immediately clear to me where that might come up in practice.
  - lilyball 8y ago
    std::mem::forget() is allowed in Safe Rust specifically because it's always possible to reconstruct it safely using Rc. As for where it comes up, it's usually because you're handing ownership of the memory to something outside of Rust's normal memory management. In most cases you'll be using a higher-level API like Box::into_raw(), but if you look at the implementation of that you'll see it uses mem::forget() internally.
vardump 8y ago
> ... "If at all possible, please do not allow this value/object/whatever to be paged out of RAM?" With the precondition you're running on bare metal (no virtualization), I think you'd need a kernel mode driver for that to cover all scenarios, including system hibernation and suspend. You can reliably zero out the secret data before hibernation or even suspend by implementing power management calls – mlock or VirtualProtect aren't going to protect you from that leak.
- blattimwind 8y ago
  You don't really need a driver for that. On Linux you could use a systemd unit with Before=sleep.target (or probably a udev rule). Windows has PBT_APMSUSPEND (which some applications use to quick-save before entering stand-by mode).
  - vardump 8y ago
    I think the systemd case is ok. But PBT_APMSUSPEND is not necessarily. There's a 2 second timeout before Windows suspends or hibernates regardless. It's possible for the event handling thread not to run until after the timeout has passed, that is, after the system has returned from suspend or hibernate. That's important considering the adversary might induce this kind of scenario to steal the secret. Kernel mode driver does not have that limitation, because it can't be pre-empted.
wyldfire 8y ago
There's no target neutral way in Rust IIRC but there is a standard way among unix-style targets. It's page granularity, and requires superuser privileges though. See mlock(2) for details [1]. [1] https://man.openbsd.org/mlock.2 https://man.openbsd.org/mlock.2
- the8472 8y ago
  At least on linux unprivileged processes can lock a limited amount of memory. > Since Linux 2.6.9, no limits are placed on the amount of memory that a privileged process can lock and the RLIMIT_MEMLOCK soft resource limit instead defines a limit on how much memory an unprivileged process may lock.
orf 8y ago
How would that work at the operating system level, which is where page in and outs occur? You might be able to tell some OS's not to page out a specific page, but with variable length data that becomes impossible
- charleslmunger 8y ago
  http://man7.org/linux/man-pages/man2/mlock.2.html http://man7.org/linux/man-pages/man2/mlock.2.html at least for linux.
  - monocasa 8y ago
    mlock, like VirtualLock, doesn't guarantee that a copy won't go out to disk, only that a copy will always be in RAM.
    - charleslmunger 8y ago
      The notes section of the page I linked addresses this - it'll only be paged in a hibernate state where memory is powered down. It won't be paged without that.
      - monocasa 8y ago
        That's just one of many examples. For instance if you're running in a VM, all bets are off, beacuse the host can and will page you out guest phys memory.
- loeg 8y ago
  Have a separate allocation pool for such objects that uses the mlock syscall on memory ranges it allocates from the OS. As you note, you can't just mlock memory handed out by the ordinary allocator, as you never know when it will be free to munlock.
TheDong 8y ago
It's pretty much impossible to zero out memory in all cases in any language. See [0]. In rust, there's the zeroize crate [1], which uses compiler fences and volitile memory to have a shot at it, but it's still not a sure thing. That library doesn't help with the paging issue you mention, but what you can do for that is entirely platform dependent (up to the kernel's paging mechanism), and so rust isn't going to make it any easier or harder than just making the libc calls/syscalls yourself. [0]: http://www.daemonology.net/blog/2014-09-06-zeroing-buffers-is-insufficient.html http://www.daemonology.net/blog/2014-09-06-zeroing-buffers-i... [1]:https://github.com/iqlusioninc/crates/tree/zeroize/v0.5.2/zeroize#about https://github.com/iqlusioninc/crates/tree/zeroize/v0.5.2/ze...
- kccqzy 8y ago
  Your first reference refers to the problem in C. The C standard has given C compilers too much leeway in managing memory (spilling things to the stack, copying values etc). That doesn't mean it's impossible to zero out memory in all cases in any language. Your rust crate similarly uses a combination of tricks to ensure zeroing buffers is not optimized away. The core problem here is compiler optimizations. If you write in, say, assembler, you will have precise control in zeroing the buffers and not spilling sensitive register values to the stack, and also zero out XMM registers. In fact that the very article you linked to outlined a solution in the form of a proposed language extension disagrees with your characterization that this problem cannot be solved in any languages. Clearly, the author thinks that it's still possible to zero buffers correctly in a language, but just that C is not such a language unless it gains his proposed extension.
  - ncmncm 8y ago
    This is an overoptimistic view. Yes, people are trying to create ways to zero things reliably, but all the machinery meant to improve performance is doing its level best to undermine that. It has to be solved at all levels; each level gets a chance to sabotage it, and will.
  - TheDong 8y ago
    Rust is an llvm based language, which is to say the caveats that apply to optimizing c compilers almost all apply identically to rust since llvm at its core is more an optimizing c compiler than anything else.
pornel 8y ago
There is no such method in the standard library. You'd get a pointer to the underlying memory (there's `Vec.as_ptr()` for example), and make whatever (C) system calls you want.

Grollicus 8y ago

With security mindset, "if at all possible do not allow" is the basically the same as "always allow". As mlock / VirtualProtect are OS dependent I strongly suspect this is not part of Rust itself. Additionally, I suspect there would be some design problems for example putting this protected thing on the stack (protect the whole stack? Or just parts? Bookkeeping wether a stack lock can be released after a function exits?). How do you move values into / out of such an object? So I don't think this exists, but I've been pleasantly surprised by Rust before. My general solution for this case is to either mlockall the whole process if it is small / short-living or to have no swap in the machine, for example long running crypto servers. Who runs servers with swap anyways?

deleted 8y ago
[deleted]

monocasa 8y ago

Those system calls don't really work like that; there isn't really a public way to say "don't save a copy of this page on disk" on Windows, OSX, or Linux. For instance see Raymond Chen's explanation here > Is VirtualLock safe to use for cryptographic purposes – that is, to avoid sensitive memory being written to disk? > ... > [There does not appear to be any guarantee that the memory won't be written to disk while locked. As you noted, the machine may be hibernated, or it may be running in a VM that gets snapshotted. -Raymond] VirtualLock and the equivalents only guarantee that the canonical copy of that page stays in RAM, not that other copies won't make their way out to disk. Edit: add link https://blogs.msdn.microsoft.com/oldnewthing/20140207-00/?p=1833 https://blogs.msdn.microsoft.com/oldnewthing/20140207-00/?p=...

codys 8y ago

One thing to note is that it is not guaranteed that values are ever dropped. In rust, it is "safe" to leak memory (it wasn't possible to make it unsafe in rust).

zozbot123 8y ago
AIUI, this should only come up in rare cases, mostly involving RC cycles. The "should" proviso is there because the operation of "forgetting" that a value has to be dropped (thus de-facto leaking it) is in fact allowed in Safe Rust, and it's not immediately clear to me where that might come up in practice.
- lilyball 8y ago
  std::mem::forget() is allowed in Safe Rust specifically because it's always possible to reconstruct it safely using Rc. As for where it comes up, it's usually because you're handing ownership of the memory to something outside of Rust's normal memory management. In most cases you'll be using a higher-level API like Box::into_raw(), but if you look at the implementation of that you'll see it uses mem::forget() internally.

vardump 8y ago

> ... "If at all possible, please do not allow this value/object/whatever to be paged out of RAM?" With the precondition you're running on bare metal (no virtualization), I think you'd need a kernel mode driver for that to cover all scenarios, including system hibernation and suspend. You can reliably zero out the secret data before hibernation or even suspend by implementing power management calls – mlock or VirtualProtect aren't going to protect you from that leak.

blattimwind 8y ago
You don't really need a driver for that. On Linux you could use a systemd unit with Before=sleep.target (or probably a udev rule). Windows has PBT_APMSUSPEND (which some applications use to quick-save before entering stand-by mode).
- vardump 8y ago
  I think the systemd case is ok. But PBT_APMSUSPEND is not necessarily. There's a 2 second timeout before Windows suspends or hibernates regardless. It's possible for the event handling thread not to run until after the timeout has passed, that is, after the system has returned from suspend or hibernate. That's important considering the adversary might induce this kind of scenario to steal the secret. Kernel mode driver does not have that limitation, because it can't be pre-empted.

wyldfire 8y ago

There's no target neutral way in Rust IIRC but there is a standard way among unix-style targets. It's page granularity, and requires superuser privileges though. See mlock(2) for details [1]. [1] https://man.openbsd.org/mlock.2 https://man.openbsd.org/mlock.2

the8472 8y ago
At least on linux unprivileged processes can lock a limited amount of memory. > Since Linux 2.6.9, no limits are placed on the amount of memory that a privileged process can lock and the RLIMIT_MEMLOCK soft resource limit instead defines a limit on how much memory an unprivileged process may lock.

orf 8y ago

How would that work at the operating system level, which is where page in and outs occur? You might be able to tell some OS's not to page out a specific page, but with variable length data that becomes impossible

charleslmunger 8y ago
http://man7.org/linux/man-pages/man2/mlock.2.html http://man7.org/linux/man-pages/man2/mlock.2.html at least for linux.
- monocasa 8y ago
  mlock, like VirtualLock, doesn't guarantee that a copy won't go out to disk, only that a copy will always be in RAM.
  - charleslmunger 8y ago
    The notes section of the page I linked addresses this - it'll only be paged in a hibernate state where memory is powered down. It won't be paged without that.
    - monocasa 8y ago
      That's just one of many examples. For instance if you're running in a VM, all bets are off, beacuse the host can and will page you out guest phys memory.
loeg 8y ago
Have a separate allocation pool for such objects that uses the mlock syscall on memory ranges it allocates from the OS. As you note, you can't just mlock memory handed out by the ordinary allocator, as you never know when it will be free to munlock.

TheDong 8y ago

It's pretty much impossible to zero out memory in all cases in any language. See [0]. In rust, there's the zeroize crate [1], which uses compiler fences and volitile memory to have a shot at it, but it's still not a sure thing. That library doesn't help with the paging issue you mention, but what you can do for that is entirely platform dependent (up to the kernel's paging mechanism), and so rust isn't going to make it any easier or harder than just making the libc calls/syscalls yourself. [0]: http://www.daemonology.net/blog/2014-09-06-zeroing-buffers-is-insufficient.html http://www.daemonology.net/blog/2014-09-06-zeroing-buffers-i... [1]:https://github.com/iqlusioninc/crates/tree/zeroize/v0.5.2/zeroize#about https://github.com/iqlusioninc/crates/tree/zeroize/v0.5.2/ze...

kccqzy 8y ago
Your first reference refers to the problem in C. The C standard has given C compilers too much leeway in managing memory (spilling things to the stack, copying values etc). That doesn't mean it's impossible to zero out memory in all cases in any language. Your rust crate similarly uses a combination of tricks to ensure zeroing buffers is not optimized away. The core problem here is compiler optimizations. If you write in, say, assembler, you will have precise control in zeroing the buffers and not spilling sensitive register values to the stack, and also zero out XMM registers. In fact that the very article you linked to outlined a solution in the form of a proposed language extension disagrees with your characterization that this problem cannot be solved in any languages. Clearly, the author thinks that it's still possible to zero buffers correctly in a language, but just that C is not such a language unless it gains his proposed extension.
- ncmncm 8y ago
  This is an overoptimistic view. Yes, people are trying to create ways to zero things reliably, but all the machinery meant to improve performance is doing its level best to undermine that. It has to be solved at all levels; each level gets a chance to sabotage it, and will.
- TheDong 8y ago
  Rust is an llvm based language, which is to say the caveats that apply to optimizing c compilers almost all apply identically to rust since llvm at its core is more an optimizing c compiler than anything else.

pornel 8y ago

There is no such method in the standard library. You'd get a pointer to the underlying memory (there's `Vec.as_ptr()` for example), and make whatever (C) system calls you want.