5 ms·

I'd love some further explanation here: what does locking memory mean in this context? For what purpose do security sensitive programs lock their memory? what

I'd love some further explanation here: what does locking memory mean in this context? For what purpose do security sensitive programs lock their memory? what is the performance degradation that happens with low-spec computers?

SteveNuts 8y ago
It tells the OS to never swap memory allocated to the process to disk. You should always lock memory if you're going to be storing crypto keys, etc. since once the pages are swapped to disk you're vulnerable to someone pulling the swap partition out and reading it.
- rkeene2 8y ago
  It is significant to note that: 1. You don't have to lock all your memory (although it may be hard to capture all the intermediate buffers if you don't). 2. You still need to clear the memory buffers after they're no longer going to be used, otherwise other processes can read /proc/kcore, etc (or cool your RAM and extract it and put it another system) 3. It is possible to encrypt the swap partition with a randomly generated key at boot
richard_todd 8y ago
One locks memory to keep it from being swapped to disk. You can imagine if you have sensitive data, you want to keep it as ephemeral as possible. Do even security-focused go programs typically lock _all_ of their virtual memory, or just the sensitive pages? I don’t know. But if a bunch of programs hogged hundreds of megabytes of unused RAM each, that’s the problem alluded to on low resource systems.
deathanatos 8y ago
> what does locking memory mean in this context? I think the person is referring to the mlock() and mlockall() functions (or equivalents on other OS), which keep pages resident / prevents pages from being paged out. Forces them to remain in RAM. It can be used to, e.g., prevent an encryption key or password from being swapped out to disk, where it might then be recoverable. (Personally, this is why I encrypt swap.) > what is the performance degradation that happens with low-spec computers? Locking a larger portion of RAM means less room for the OS to page out unused pages and free up the space for other programs. While one can try to selectively lock buffers with sensitive data with mlock(), you have to be sure they aren't copied into other buffers that aren't locked (and could thus be subsequently paged out). If you're writing a UI program that displays or receives those in a widget, this might be harder (you might not have access to the internal buffer of the widget, as it is an "implementation detail" of your library), and locking the entire process might be a simpler solution (albeit being a bigger hammer).
- rkeene2 8y ago
  Linux cgroups and Solaris containers atleast also provide a way to avoid paging to disk, without modifying the program.
- scott_s 8y ago
  Good explanations of what and why one might lock memory. But, in the context of this overall discussion, I think it's important to keep in mind that when a process allocates a large amount of virtual memory, it does not automatically allocate any physical memory. So a process allocating a lot of virtual memory up front should not impact other processes which have locked some of their memory into physical memory.
  - deathanatos 8y ago
    > I think it's important to keep in mind that when a process allocates a large amount of virtual memory, it does not automatically allocate any physical memory. It will, if you mlock() it, I believe. The manual page notes "real-time processes" as a main user of mlock() (the other being the cryptographic uses I hinted at); it cites their use case as locking the page to avoid delays due to paging during critical sections. In order for that to work, the OS would need to bring the pages in, at the time of locking; so at that point, a large virtual allocation becomes equivalent to a physical one.
    - scott_s 8y ago
      Correct, but my point was that a process with allocates a lot of physical memory will not interfere with the other processes that have locked pages.
- monocasa 8y ago
  My understanding was that mlock doesn't really keep the page from being paged out, since it can't even begin to do that in the hibernation case, or if you're running as a VM who's guest RAM wasn't mlocked. What it does is keep a canonical version of the page in memory. That's useful being able to deterministically touch a piece of memory, but it doesn't really help you as far as making sure the page never touches disk.

SteveNuts 8y ago

It tells the OS to never swap memory allocated to the process to disk. You should always lock memory if you're going to be storing crypto keys, etc. since once the pages are swapped to disk you're vulnerable to someone pulling the swap partition out and reading it.

rkeene2 8y ago
It is significant to note that: 1. You don't have to lock all your memory (although it may be hard to capture all the intermediate buffers if you don't). 2. You still need to clear the memory buffers after they're no longer going to be used, otherwise other processes can read /proc/kcore, etc (or cool your RAM and extract it and put it another system) 3. It is possible to encrypt the swap partition with a randomly generated key at boot

richard_todd 8y ago

One locks memory to keep it from being swapped to disk. You can imagine if you have sensitive data, you want to keep it as ephemeral as possible. Do even security-focused go programs typically lock _all_ of their virtual memory, or just the sensitive pages? I don’t know. But if a bunch of programs hogged hundreds of megabytes of unused RAM each, that’s the problem alluded to on low resource systems.

deathanatos 8y ago

> what does locking memory mean in this context? I think the person is referring to the mlock() and mlockall() functions (or equivalents on other OS), which keep pages resident / prevents pages from being paged out. Forces them to remain in RAM. It can be used to, e.g., prevent an encryption key or password from being swapped out to disk, where it might then be recoverable. (Personally, this is why I encrypt swap.) > what is the performance degradation that happens with low-spec computers? Locking a larger portion of RAM means less room for the OS to page out unused pages and free up the space for other programs. While one can try to selectively lock buffers with sensitive data with mlock(), you have to be sure they aren't copied into other buffers that aren't locked (and could thus be subsequently paged out). If you're writing a UI program that displays or receives those in a widget, this might be harder (you might not have access to the internal buffer of the widget, as it is an "implementation detail" of your library), and locking the entire process might be a simpler solution (albeit being a bigger hammer).

rkeene2 8y ago
Linux cgroups and Solaris containers atleast also provide a way to avoid paging to disk, without modifying the program.
scott_s 8y ago
Good explanations of what and why one might lock memory. But, in the context of this overall discussion, I think it's important to keep in mind that when a process allocates a large amount of virtual memory, it does not automatically allocate any physical memory. So a process allocating a lot of virtual memory up front should not impact other processes which have locked some of their memory into physical memory.
- deathanatos 8y ago
  > I think it's important to keep in mind that when a process allocates a large amount of virtual memory, it does not automatically allocate any physical memory. It will, if you mlock() it, I believe. The manual page notes "real-time processes" as a main user of mlock() (the other being the cryptographic uses I hinted at); it cites their use case as locking the page to avoid delays due to paging during critical sections. In order for that to work, the OS would need to bring the pages in, at the time of locking; so at that point, a large virtual allocation becomes equivalent to a physical one.
  - scott_s 8y ago
    Correct, but my point was that a process with allocates a lot of physical memory will not interfere with the other processes that have locked pages.
monocasa 8y ago
My understanding was that mlock doesn't really keep the page from being paged out, since it can't even begin to do that in the hibernation case, or if you're running as a VM who's guest RAM wasn't mlocked. What it does is keep a canonical version of the page in memory. That's useful being able to deterministically touch a piece of memory, but it doesn't really help you as far as making sure the page never touches disk.