7 ms·

> "Clever" memory use is frowned upon in Rust. In C, anything goes. For example, in C I'd be tempted to reuse a buffer allocated for one purpose for another pur

> "Clever" memory use is frowned upon in Rust. In C, anything goes. For example, in C I'd be tempted to reuse a buffer allocated for one purpose for another purpose later (a technique known as HEARTBLEED). This made me laugh

viraptor 6y ago
It's not trivial to write a funny and clever burn, but this just hits the spot...
- waterhouse 6y ago
  That is nice, although I think Heartbleed was due to a missing bounds check enabling the reading of adjacent memory, not due to reusing the same buffer...
  - NobodyNada 6y ago
    If my memory is correct: yes, the root cause was a missing bounds check, but the vulnerability was much worse than it could have been because OpenSSL tended to allocate small blocks of memory and aggressively reuse them — meaning the exploited buffer was very likely to be close in proximity to sensitive information. I don’t have time right now to research the full details, but the Wikipedia article gives a clue: > Theo de Raadt, founder and leader of the OpenBSD and OpenSSH projects, has criticized the OpenSSL developers for writing their own memory management routines and thereby, he claims, circumventing OpenBSD C standard library exploit countermeasures, saying "OpenSSL is not developed by a responsible team." Following Heartbleed's disclosure, members of the OpenBSD project forked OpenSSL into LibreSSL.
    - waterhouse 6y ago
      Ah, good point; at the very least things like zeroing out buffers upon deallocation would have helped. Yes, I was a fan of the commits showing up at opensslrampage.org. One of the highlights was when they found it would use private keys as an entropy source: https://opensslrampage.org/post/83007010531/well-even-if-time-isnt-random-your-rsa https://opensslrampage.org/post/83007010531/well-even-if-tim...
    - saagarjha 6y ago
      Until very recently, memory allocators were more than happy to return you the thing you just deallocated if you asked for another allocation of the same size. It makes sense, too: if you're calling malloc/free in a loop, which is pretty common, this is pretty much the best thing you can do for performance. Countless heap exploits later (mostly attacking heap metadata rather than stale data, to be honest) allocators have begun to realize that predictable allocation patterns might not be the best idea, so they're starting to move away from this.
      - rcxdude 6y ago
        True of the more common ones, but it should be acknowledged that OpenBSD was doing this kind of thing (and many other hardening techniques) before heartbleed, which was the main reason Theo de Raadt was so upset that they decided to circumvent this, because OpenBSD's allocator could have mitigated the impact otherwise.
        loeg 6y ago
        Even higher-performance mallocs like jemalloc had heap debugging features (poisoning freed memory) before Heartbleed, which -- if enabled -- would catch use-after-frees, so long as libraries and applications didn't circumvent malloc like OpenSSL did (and Python still does AFAIK).
        supergarfield 6y ago
        > and Python still does AFAIK Don't you sort of have to do that if you're writing your own garbage collector, though? I guess for a simple collector you could maintain lists of allocated objects separately, but precisely controlling where the memory is allocated is important for any kind of performant implementation.
        loeg 6y ago
        Python does refcount-based memory management. It's not a GC design. You don't have to retain objects in an internal linked list when the refcount drops to zero, but CPython does, purely as a performance optimization. Type-specific free lists (just a few examples; there are more): * https://github.com/python/cpython/blob/master/Objects/floatobject.c#L231-L245 https://github.com/python/cpython/blob/master/Objects/floato... * https://github.com/python/cpython/blob/master/Objects/tupleobject.c#L268-L290 https://github.com/python/cpython/blob/master/Objects/tupleo... And also just wrapping malloc in general. There's no refcounting reason for this, they just assume system malloc is slow (which might be true, for glibc) and wrap it in the default build configuration: https://github.com/python/cpython/blob/master/Objects/obmalloc.c#L733-L771 https://github.com/python/cpython/blob/master/Objects/obmall... So many layers of wrapping malloc, just because system allocators were slow in 2000. Defeats free() poisoning and ASAN. obmalloc can be disabled by turning off PYMALLOC, but that doesn't disable the per-type freelists IIRC. And PYMALLOC is enabled by default.
    - fulafel 6y ago
      That's what happens by using normal malloc/free anyway, no? Implementations of malloc have a strong performance incentive to allocate from the cache hot most recently freed blocks.
      - ric129 6y ago
        Yes, all allocators (except perhaps OpenBSDs from what I see in this thread) do this. It is also why `calloc` exists - because zero-initializing every single allocation is really, really expensive.
  - gameswithgo 6y ago
    iirc both issues caused the problem. Buffer overlow let the memory get read, re-use meant there was important data in the buffer.
- eps 6y ago
  That's not a good burn though.
- Blikkentrekker 6y ago
  It's incorrect, however. Heartbleed wasn't caused by reusing buffers; it was caused by not properly sanitizing the length of the buffer from entrusted input, and reading over it's allocated size, thus allowing the attacker to read into memory that wasn't meant for him.
  - pornel 6y ago
    OpenSSL had its own memory-recycling allocator, which made the bug guarantee leaking OpenSSL's own data. Of course leaking random process memory wouldn't be safe either, but the custom allocator added that extra touch.
    - Blikkentrekker 6y ago
      OpenSSL, like many other software indeed used a custom allocator, but this hasn't much to do with this to anything at all, as the system allocator also strongly favors giving back memory that once belonged to the same process, as it has to zero memory that belonged to other processes first. This is of course a kernel feature when the lower level primitives are used that ask for blocks of memory from the kernel, which zeroes them if they had belonged to another process prior, and does not when they had not, and thus strongly favors giving back own memory. — allocators, such as the standard library's one or any custom ones, are built on top of these primitives.
sivizius 6y ago
This was actually a somewhat significant reason I shared this article. (^.^)
gridspy 6y ago
> in C I'd be tempted to reuse a buffer allocated for one purpose ... In rust I'd just declare an enum for this. Enums in Rust can store data. In this way they are like a safe union.
secondcoming 6y ago
It was quite funny but it's quite likely you'll be reusing memory anyway whether it's on the stack or the heap, no? The issue with this is that 'clever' compilers can optimise out any memset calls you do.
- pornel 6y ago
  Rust's safety rules also forbid access to uninitialized memory, even if it's just a basic array of bytes. This is an extra protection against accidentally disclosing data from a previous "recycled" allocation.

viraptor 6y ago

It's not trivial to write a funny and clever burn, but this just hits the spot...

waterhouse 6y ago
That is nice, although I think Heartbleed was due to a missing bounds check enabling the reading of adjacent memory, not due to reusing the same buffer...
- NobodyNada 6y ago
  If my memory is correct: yes, the root cause was a missing bounds check, but the vulnerability was much worse than it could have been because OpenSSL tended to allocate small blocks of memory and aggressively reuse them — meaning the exploited buffer was very likely to be close in proximity to sensitive information. I don’t have time right now to research the full details, but the Wikipedia article gives a clue: > Theo de Raadt, founder and leader of the OpenBSD and OpenSSH projects, has criticized the OpenSSL developers for writing their own memory management routines and thereby, he claims, circumventing OpenBSD C standard library exploit countermeasures, saying "OpenSSL is not developed by a responsible team." Following Heartbleed's disclosure, members of the OpenBSD project forked OpenSSL into LibreSSL.
  - waterhouse 6y ago
    Ah, good point; at the very least things like zeroing out buffers upon deallocation would have helped. Yes, I was a fan of the commits showing up at opensslrampage.org. One of the highlights was when they found it would use private keys as an entropy source: https://opensslrampage.org/post/83007010531/well-even-if-time-isnt-random-your-rsa https://opensslrampage.org/post/83007010531/well-even-if-tim...
  - saagarjha 6y ago
    Until very recently, memory allocators were more than happy to return you the thing you just deallocated if you asked for another allocation of the same size. It makes sense, too: if you're calling malloc/free in a loop, which is pretty common, this is pretty much the best thing you can do for performance. Countless heap exploits later (mostly attacking heap metadata rather than stale data, to be honest) allocators have begun to realize that predictable allocation patterns might not be the best idea, so they're starting to move away from this.
    - rcxdude 6y ago
      True of the more common ones, but it should be acknowledged that OpenBSD was doing this kind of thing (and many other hardening techniques) before heartbleed, which was the main reason Theo de Raadt was so upset that they decided to circumvent this, because OpenBSD's allocator could have mitigated the impact otherwise.
      - loeg 6y ago
        Even higher-performance mallocs like jemalloc had heap debugging features (poisoning freed memory) before Heartbleed, which -- if enabled -- would catch use-after-frees, so long as libraries and applications didn't circumvent malloc like OpenSSL did (and Python still does AFAIK).
        supergarfield 6y ago
        > and Python still does AFAIK Don't you sort of have to do that if you're writing your own garbage collector, though? I guess for a simple collector you could maintain lists of allocated objects separately, but precisely controlling where the memory is allocated is important for any kind of performant implementation.
        loeg 6y ago
        Python does refcount-based memory management. It's not a GC design. You don't have to retain objects in an internal linked list when the refcount drops to zero, but CPython does, purely as a performance optimization. Type-specific free lists (just a few examples; there are more): * https://github.com/python/cpython/blob/master/Objects/floatobject.c#L231-L245 https://github.com/python/cpython/blob/master/Objects/floato... * https://github.com/python/cpython/blob/master/Objects/tupleobject.c#L268-L290 https://github.com/python/cpython/blob/master/Objects/tupleo... And also just wrapping malloc in general. There's no refcounting reason for this, they just assume system malloc is slow (which might be true, for glibc) and wrap it in the default build configuration: https://github.com/python/cpython/blob/master/Objects/obmalloc.c#L733-L771 https://github.com/python/cpython/blob/master/Objects/obmall... So many layers of wrapping malloc, just because system allocators were slow in 2000. Defeats free() poisoning and ASAN. obmalloc can be disabled by turning off PYMALLOC, but that doesn't disable the per-type freelists IIRC. And PYMALLOC is enabled by default.
  - fulafel 6y ago
    That's what happens by using normal malloc/free anyway, no? Implementations of malloc have a strong performance incentive to allocate from the cache hot most recently freed blocks.
    - ric129 6y ago
      Yes, all allocators (except perhaps OpenBSDs from what I see in this thread) do this. It is also why `calloc` exists - because zero-initializing every single allocation is really, really expensive.
- gameswithgo 6y ago
  iirc both issues caused the problem. Buffer overlow let the memory get read, re-use meant there was important data in the buffer.
eps 6y ago
That's not a good burn though.
Blikkentrekker 6y ago
It's incorrect, however. Heartbleed wasn't caused by reusing buffers; it was caused by not properly sanitizing the length of the buffer from entrusted input, and reading over it's allocated size, thus allowing the attacker to read into memory that wasn't meant for him.
- pornel 6y ago
  OpenSSL had its own memory-recycling allocator, which made the bug guarantee leaking OpenSSL's own data. Of course leaking random process memory wouldn't be safe either, but the custom allocator added that extra touch.
  - Blikkentrekker 6y ago
    OpenSSL, like many other software indeed used a custom allocator, but this hasn't much to do with this to anything at all, as the system allocator also strongly favors giving back memory that once belonged to the same process, as it has to zero memory that belonged to other processes first. This is of course a kernel feature when the lower level primitives are used that ask for blocks of memory from the kernel, which zeroes them if they had belonged to another process prior, and does not when they had not, and thus strongly favors giving back own memory. — allocators, such as the standard library's one or any custom ones, are built on top of these primitives.

sivizius 6y ago

This was actually a somewhat significant reason I shared this article. (^.^)

gridspy 6y ago

> in C I'd be tempted to reuse a buffer allocated for one purpose ... In rust I'd just declare an enum for this. Enums in Rust can store data. In this way they are like a safe union.

secondcoming 6y ago

It was quite funny but it's quite likely you'll be reusing memory anyway whether it's on the stack or the heap, no? The issue with this is that 'clever' compilers can optimise out any memset calls you do.

pornel 6y ago
Rust's safety rules also forbid access to uninitialized memory, even if it's just a basic array of bytes. This is an extra protection against accidentally disclosing data from a previous "recycled" allocation.