5 ms·

This is overly complicated, there is no need to bring Rust semantics to C to ensure memory safety. A good mempool implementation is all you need (i.e keeps tra

This is overly complicated, there is no need to bring Rust semantics to C to ensure memory safety. A good mempool implementation is all you need (i.e keeps track of every request, and zeros out the memory on release)

Voultapher 3y ago
Compiler optimizations and other forms of UB like integer overflow would like a word with you. If it were that simple, someone would have had success at scale by now https://alexgaynor.net/2020/may/27/science-on-memory-unsafety-and-security/ https://alexgaynor.net/2020/may/27/science-on-memory-unsafet....
- ActorNightly 3y ago
  >If it were that simple, someone would have had success at scale by now A lot of code in that article doesn't use mempools, and furthermore, just because a double free exists doesn't mean that its always exploitable. And if its exploitable, it doesn't mean that you can gain a shell or even exfil data, sometimes it means you can just crash the program. Fundamentally, if you write a wrapper around memory management that keeps track of allocated resources, much in the same way how rust includes some runtime code during compilation for memory safety, you gain the same functionality.
  - jart 3y ago
    malloc() already keeps track of every memory allocation. Just what kind of tracking are we talking about here?
    - ActorNightly 3y ago
      malloc doesn't keep track accurately in all cases, which is why double free is possible in the first place. with mempool implementation, you shouldn't be able to release a previously released chunk because the pointer to that chunk will be zeroed out. This requires one more level of indirection in accessing the memory, i.e pointer to a struct that contains the pointer to the memory, but is otherwise safe. As of note that may be causing some confusion, im not referencing the standard linux mempool implementation. I have written custom ones with a lot of helper functions for safe memory access.
      - jart 3y ago
        What is mempool? Is that this? https://en.wikipedia.org/wiki/Region-based_memory_management https://en.wikipedia.org/wiki/Region-based_memory_management
  - Voultapher 3y ago
    > Fundamentally, if you write a wrapper around memory management that keeps track of allocated resources, much in the same way how rust includes some runtime code during compilation for memory safety, you gain the same functionality. Can you substantiate that? There are commonly employed tracking allocators, such as ASAN that can catch certain kinds of UB, and UBSAN other, and with special interpreters you can catch even more. But even basic ASAN is more exhaustive than what you are suggesting, and it provably can't provide the same guarantees that safe and sound Rust gives you https://stackoverflow.com/a/48902567 https://stackoverflow.com/a/48902567: > And that is not accounting for the fact that sanitizers are incompatible with each others. That is, even if you were willing to accept the combined slow-down (15x-45x?) and memory overhead (15x-30x?), you would still NOT manage for a C++ program to be as safe as a Rust one. Also, I think you misunderstand the way Rust works, it does compile-time ownership checking, which allows it to avoid run-time checking, so this part "same way how rust includes some runtime code during compilation for memory safety" is factually wrong.
    - thradams 3y ago
      Rust needs to add some runtime checks when calling destructors in scenarios where some object may or may not be moved. In C++ for instance, for smart pointers, the destructor will have a "if p!= NULL". Then if the smart pointer was moved, it makes the pointer null and the destructor checks at runtime for it.
    - ActorNightly 3y ago
      >so this part "same way how rust includes some runtime code during compilation for memory safety" is factually wrong. RefCell includes runtime code. Fundamentally, because of Rice Theorem, the compiler cannot predict the state of memory at all points in time, so runtime checks are needed. >Can you substantiate that? I mean, double free relies on using free() twice. Mempool malloc()'s once, and free()'s once at exit. Use after free is mitigated by making sure that the pointer to the memory is set to zero (mempool either returns struct or a pointer to a pointer on allocation, and you access the requested memory through that). Furthermore, you can have multiple mempools, and keep critical data separate, so if the pointer doesn't get zeroed out in the implementation, use after free won't leak anything critical.
thradams 3y ago
mempool does not solve double free, use after free (at least at compile time) or fopen sample. But mempool and ownership can be complementary.
- ActorNightly 3y ago
  If you are talking about a very naive version of mempool, then you are correct, but thats why I said a good implementation. The whole point of a good mempool is that you malloc once, and only call free when you exit the program. The data structures for memory allocation will never get corrupted. And the memory pool will never release chunk twice cause it keeps tracks of allocated chunks. User after free is mitigated in the same way. When you allocate, you get a struct back that contains a pointer to the data. When you release, that pointer is zeroed out.
  - jart 3y ago
    > The whole point of a good mempool is that you malloc once, and only call free when you exit the program So you're describing fork() and _exit(). That's my favorite memory manager. For example, chibicc never calls free() and instead just forks a process for each item of work in the compile pipeline. It makes the codebase infinitely simpler. Rui literally solved memory leaks! No idea what you're talking about.
    - thradams 3y ago
      One issue I see with this approach (compiler leaking memory) is, for instance, if the requirements change and you need to utilize the compiler as a lib or service. For example, if the Cake source is used within a web browser compiled with Emscripten, leaking memory with each compilation would lead to a continuous increase in memory usage. Additionally, compilers often offer the option to compile multiple files. Therefore, we cannot afford to leak memory with each file compilation. Initially I was planning a global allocator for cake source. It had a lot of memory leaks that would be solved in the future. When ownership checks were added it was a perfect candidate for fixing leaks. (actually I also had this in mind)

Voultapher 3y ago

Compiler optimizations and other forms of UB like integer overflow would like a word with you. If it were that simple, someone would have had success at scale by now https://alexgaynor.net/2020/may/27/science-on-memory-unsafety-and-security/ https://alexgaynor.net/2020/may/27/science-on-memory-unsafet....

ActorNightly 3y ago
>If it were that simple, someone would have had success at scale by now A lot of code in that article doesn't use mempools, and furthermore, just because a double free exists doesn't mean that its always exploitable. And if its exploitable, it doesn't mean that you can gain a shell or even exfil data, sometimes it means you can just crash the program. Fundamentally, if you write a wrapper around memory management that keeps track of allocated resources, much in the same way how rust includes some runtime code during compilation for memory safety, you gain the same functionality.
- jart 3y ago
  malloc() already keeps track of every memory allocation. Just what kind of tracking are we talking about here?
  - ActorNightly 3y ago
    malloc doesn't keep track accurately in all cases, which is why double free is possible in the first place. with mempool implementation, you shouldn't be able to release a previously released chunk because the pointer to that chunk will be zeroed out. This requires one more level of indirection in accessing the memory, i.e pointer to a struct that contains the pointer to the memory, but is otherwise safe. As of note that may be causing some confusion, im not referencing the standard linux mempool implementation. I have written custom ones with a lot of helper functions for safe memory access.
    - jart 3y ago
      What is mempool? Is that this? https://en.wikipedia.org/wiki/Region-based_memory_management https://en.wikipedia.org/wiki/Region-based_memory_management
- Voultapher 3y ago
  > Fundamentally, if you write a wrapper around memory management that keeps track of allocated resources, much in the same way how rust includes some runtime code during compilation for memory safety, you gain the same functionality. Can you substantiate that? There are commonly employed tracking allocators, such as ASAN that can catch certain kinds of UB, and UBSAN other, and with special interpreters you can catch even more. But even basic ASAN is more exhaustive than what you are suggesting, and it provably can't provide the same guarantees that safe and sound Rust gives you https://stackoverflow.com/a/48902567 https://stackoverflow.com/a/48902567: > And that is not accounting for the fact that sanitizers are incompatible with each others. That is, even if you were willing to accept the combined slow-down (15x-45x?) and memory overhead (15x-30x?), you would still NOT manage for a C++ program to be as safe as a Rust one. Also, I think you misunderstand the way Rust works, it does compile-time ownership checking, which allows it to avoid run-time checking, so this part "same way how rust includes some runtime code during compilation for memory safety" is factually wrong.
  - thradams 3y ago
    Rust needs to add some runtime checks when calling destructors in scenarios where some object may or may not be moved. In C++ for instance, for smart pointers, the destructor will have a "if p!= NULL". Then if the smart pointer was moved, it makes the pointer null and the destructor checks at runtime for it.
  - ActorNightly 3y ago
    >so this part "same way how rust includes some runtime code during compilation for memory safety" is factually wrong. RefCell includes runtime code. Fundamentally, because of Rice Theorem, the compiler cannot predict the state of memory at all points in time, so runtime checks are needed. >Can you substantiate that? I mean, double free relies on using free() twice. Mempool malloc()'s once, and free()'s once at exit. Use after free is mitigated by making sure that the pointer to the memory is set to zero (mempool either returns struct or a pointer to a pointer on allocation, and you access the requested memory through that). Furthermore, you can have multiple mempools, and keep critical data separate, so if the pointer doesn't get zeroed out in the implementation, use after free won't leak anything critical.

thradams 3y ago

mempool does not solve double free, use after free (at least at compile time) or fopen sample. But mempool and ownership can be complementary.

ActorNightly 3y ago
If you are talking about a very naive version of mempool, then you are correct, but thats why I said a good implementation. The whole point of a good mempool is that you malloc once, and only call free when you exit the program. The data structures for memory allocation will never get corrupted. And the memory pool will never release chunk twice cause it keeps tracks of allocated chunks. User after free is mitigated in the same way. When you allocate, you get a struct back that contains a pointer to the data. When you release, that pointer is zeroed out.
- jart 3y ago
  > The whole point of a good mempool is that you malloc once, and only call free when you exit the program So you're describing fork() and _exit(). That's my favorite memory manager. For example, chibicc never calls free() and instead just forks a process for each item of work in the compile pipeline. It makes the codebase infinitely simpler. Rui literally solved memory leaks! No idea what you're talking about.
  - thradams 3y ago
    One issue I see with this approach (compiler leaking memory) is, for instance, if the requirements change and you need to utilize the compiler as a lib or service. For example, if the Cake source is used within a web browser compiled with Emscripten, leaking memory with each compilation would lead to a continuous increase in memory usage. Additionally, compilers often offer the option to compile multiple files. Therefore, we cannot afford to leak memory with each file compilation. Initially I was planning a global allocator for cake source. It had a lot of memory leaks that would be solved in the future. When ownership checks were added it was a perfect candidate for fixing leaks. (actually I also had this in mind)