4 ms·

How good are AIs at coding manual memory management? Is this a sea change in automatic memory management?

by rayiner 4mo ago

mtklein 4mo ago
I have never seen Codex or Claude get manual memory management wrong. I used to be pretty fastidious about using leak sanitizer or other such tools to catch my own memory management issues, and while not quite useless, that sort of testing has dropped way down my list of worries the more I lean on LLMs. I am constantly surprised by how many formerly tedious or error prone tasks stopped being either of those, and I expect to see practice shift away from middle-safe languages like C++ to not just much more safe languages like Rust but surprisingly also to much less safe ones like C and platform specific assembly.
- gf000 4mo ago
  The hard part was never getting it correct on a local scope, that's mostly solved by a linter, or even C++'s RAII will get it right. The hard part is doing it correctly on a global scope with non-trivial lifetimes, possibly influenced by multiple threads. And in my experience LLMs are still hit or miss on these kind of problems, they can find problems from time to time, but they can't really reason well about more complex global state reliably. They will come up with "hypotheses" that 'oh sure this is the root cause of the issue' only to say something completely wrong (which you may notice or not, only to fail later)
adrian_b 4mo ago
In my opinion, the "manual" memory management, introduced by the IBM PL/I programming language by the end of 1964, and inherited by C and other languages, i.e. where the programmer is responsible for invoking "free", was a serious mistake and it was an obsolete technique already at the date of its introduction. When the explicit "free" was invented, automatic memory reclamation while avoiding the non-determinism of garbage collectors had already been known for 4 years, since 1960, when another IBM employee had invented reference counting (as a reaction to the garbage collector of LISP I). When implemented naively, reference counting has some disadvantages, but those can be circumvented relatively easily in an optimized implementation. The book discussed in the parent article also has a chapter about reference counting. I have written C programs for many decades, but I have never invoked "free" directly, because I have always used reference counts. I have never encountered a circumstance when I would have wanted to invoke "free" directly. C has the disadvantage that the compiler will not do implicitly things like virtual function invocation, reference counts handling etc. but any such techniques that are provided by higher-level languages can still be used in a language like C, even if they require more boilerplate code. I do not like the "shared_ptr" implementation of reference counting in C++, because that data type is not directly usable in places where a plain reference or pointer is expected. Implementations that do not have this problem exist.
- dcuthbertson 4mo ago
  I too have written C programs for decades. I encountered reference counting when I learned how to write Windows kernel drivers. It was very liberating to see that there were so fewer opportunities for memory leaks when reference counts were applied liberally.
- SideQuark 4mo ago
  > where the programmer is responsible for invoking "free", was a serious mistake and it was an obsolete technique already at the date of its introduction. It wasn’t obsolete then nor now. Garbage collected languages, to this day, still use on average about 2x-5x the working memory of carefully written manual memory programs. A significant amount of devices then and even today cannot support such sloppy use of resources.
  - gf000 4mo ago
    I think they mean something more akin to RAII in C++/rust. Manually writing a free statement is very error-prone and there are better language-level features to avoid that.
    - SideQuark 4mo ago
      RAII has nothing to do with garbage collection. It only provides a way to get a resource somewhat atomically. If the creator doesn’t delete the object, or if the object has a malformed destructor, it fails. If the creator calls delete twice, it fails. There’s really no garbage collection at all here.
  - pmarreck 4mo ago
    Is there a scheme where memory is auto-freed if it goes out of scope? If so, are there benefits/drawbacks to that?
    - SideQuark 4mo ago
      If it's on the stack, it's almost free to do. If it's held by a unique_ptr, then it auto frees, but there is a lot of programmer work to ensure things behave well, and there are nuances to allocating something into the unique_ptr. If it's a std::shared_ptr, which is the closest to what people think of, it's then a reference counted object, which requires a little more storage, and also has the ability to be mishandled as a pointer, screwing it up, or making the wrong type of copy of the shared_ptr, so two there are two things, and also fails to release object in the case of circluar references. Then there's a lot of other subtle things you can do, and none of this is as lightweight as a pointer, and none is as automatic as what most people think of as garbage collection. And each has costs and tradeoffs, which is why C++ added them: making certain tradeoffs easier to use, but it requires solid understanding of how such things work.
a-french-anon 4mo ago
GC's strength is not only in the ease of writing but at also reading, since you don't need to interleave allocation and business logic everywhere (be it through types or imperative code). GC simply is the only way to approach the clarity of pseudo-code in real code. That's one of my later realizations concerning the subject (https://world-playground-deceit.net/blog/2024/11/how-i-learned-to-stop-worrying-and-love-gc.html https://world-playground-deceit.net/blog/2024/11/how-i-learn...)
- adrian_b 4mo ago
  You mean "interleave deallocation and business logic everywhere". For allocation there is no difference between automatic memory management with garbage collectors or reference counts and manual memory management, where the programmer is responsible for invoking "free". These alternative memory management methods differ only in how deallocation is handled. Allocation must always be done by defining a new object, regardless of how memory is managed. Moreover, allocation also does not depend on whether an object is allocated in static storage, in a stack or in a heap. You always must define the object, so that memory should be allocated for it at compile-time if in static storage, or at run-time if in a stack or in a heap.
  - a-french-anon 4mo ago
    Well, it's true that most of the noise comes from deallocation but I did mention types for good reasons: having your code littered with std::shared_ptr (or worse for Rust where encoding lifetime goes much further than Rc/Arc) is a direct consequence of wanting GC without a global one. The source of my second revelation: GC should be opt-out (e.g. SBCL's arena system) instead of opt-in via refcounted types.
    - adrian_b 4mo ago
      While C and C++ have ugly syntaxes for "malloc", "std::shared_ptr" and the like, it is quite easy to mask that ugliness by using macros and writing thus only cleanly-looking programs, without any noise words and symbols around dynamic allocations. In general, by using macros it is possible to transform so much a C or C++ program, that it becomes unrecognizable as C/C++ and it can mimic reasonably well any other programming language that you might fancy. The problem is when you work in a team, because even if everyone will agree that such programming languages have great deficiencies, it would be impossible to reach a consensus about how the ideal programming language should look like, so eventually the team remains stuck with writing programs in the ugly standard manner.
pmarreck 4mo ago
In Zig, pretty good, but not perfect yet

How good are AIs at coding manual memory management? Is this a sea change in automatic memory management?

by rayiner 4mo ago

How good are AIs at coding manual memory management? Is this a sea change in automatic memory management?

I have never seen Codex or Claude get manual memory management wrong. I used to be pretty fastidious about using leak sanitizer or other such tools to catch my own memory management issues, and while not quite useless, that sort of testing has dropped way down my list of worries the more I lean on LLMs. I am constantly surprised by how many formerly tedious or error prone tasks stopped being either of those, and I expect to see practice shift away from middle-safe languages like C++ to not just much more safe languages like Rust but surprisingly also to much less safe ones like C and platform specific assembly.

gf000 4mo ago
The hard part was never getting it correct on a local scope, that's mostly solved by a linter, or even C++'s RAII will get it right. The hard part is doing it correctly on a global scope with non-trivial lifetimes, possibly influenced by multiple threads. And in my experience LLMs are still hit or miss on these kind of problems, they can find problems from time to time, but they can't really reason well about more complex global state reliably. They will come up with "hypotheses" that 'oh sure this is the root cause of the issue' only to say something completely wrong (which you may notice or not, only to fail later)

adrian_b 4mo ago

In my opinion, the "manual" memory management, introduced by the IBM PL/I programming language by the end of 1964, and inherited by C and other languages, i.e. where the programmer is responsible for invoking "free", was a serious mistake and it was an obsolete technique already at the date of its introduction. When the explicit "free" was invented, automatic memory reclamation while avoiding the non-determinism of garbage collectors had already been known for 4 years, since 1960, when another IBM employee had invented reference counting (as a reaction to the garbage collector of LISP I). When implemented naively, reference counting has some disadvantages, but those can be circumvented relatively easily in an optimized implementation. The book discussed in the parent article also has a chapter about reference counting. I have written C programs for many decades, but I have never invoked "free" directly, because I have always used reference counts. I have never encountered a circumstance when I would have wanted to invoke "free" directly. C has the disadvantage that the compiler will not do implicitly things like virtual function invocation, reference counts handling etc. but any such techniques that are provided by higher-level languages can still be used in a language like C, even if they require more boilerplate code. I do not like the "shared_ptr" implementation of reference counting in C++, because that data type is not directly usable in places where a plain reference or pointer is expected. Implementations that do not have this problem exist.

dcuthbertson 4mo ago
I too have written C programs for decades. I encountered reference counting when I learned how to write Windows kernel drivers. It was very liberating to see that there were so fewer opportunities for memory leaks when reference counts were applied liberally.
SideQuark 4mo ago
> where the programmer is responsible for invoking "free", was a serious mistake and it was an obsolete technique already at the date of its introduction. It wasn’t obsolete then nor now. Garbage collected languages, to this day, still use on average about 2x-5x the working memory of carefully written manual memory programs. A significant amount of devices then and even today cannot support such sloppy use of resources.
- gf000 4mo ago
  I think they mean something more akin to RAII in C++/rust. Manually writing a free statement is very error-prone and there are better language-level features to avoid that.
  - SideQuark 4mo ago
    RAII has nothing to do with garbage collection. It only provides a way to get a resource somewhat atomically. If the creator doesn’t delete the object, or if the object has a malformed destructor, it fails. If the creator calls delete twice, it fails. There’s really no garbage collection at all here.
- pmarreck 4mo ago
  Is there a scheme where memory is auto-freed if it goes out of scope? If so, are there benefits/drawbacks to that?
  - SideQuark 4mo ago
    If it's on the stack, it's almost free to do. If it's held by a unique_ptr, then it auto frees, but there is a lot of programmer work to ensure things behave well, and there are nuances to allocating something into the unique_ptr. If it's a std::shared_ptr, which is the closest to what people think of, it's then a reference counted object, which requires a little more storage, and also has the ability to be mishandled as a pointer, screwing it up, or making the wrong type of copy of the shared_ptr, so two there are two things, and also fails to release object in the case of circluar references. Then there's a lot of other subtle things you can do, and none of this is as lightweight as a pointer, and none is as automatic as what most people think of as garbage collection. And each has costs and tradeoffs, which is why C++ added them: making certain tradeoffs easier to use, but it requires solid understanding of how such things work.

a-french-anon 4mo ago

GC's strength is not only in the ease of writing but at also reading, since you don't need to interleave allocation and business logic everywhere (be it through types or imperative code). GC simply is the only way to approach the clarity of pseudo-code in real code. That's one of my later realizations concerning the subject (https://world-playground-deceit.net/blog/2024/11/how-i-learned-to-stop-worrying-and-love-gc.html https://world-playground-deceit.net/blog/2024/11/how-i-learn...)

adrian_b 4mo ago
You mean "interleave deallocation and business logic everywhere". For allocation there is no difference between automatic memory management with garbage collectors or reference counts and manual memory management, where the programmer is responsible for invoking "free". These alternative memory management methods differ only in how deallocation is handled. Allocation must always be done by defining a new object, regardless of how memory is managed. Moreover, allocation also does not depend on whether an object is allocated in static storage, in a stack or in a heap. You always must define the object, so that memory should be allocated for it at compile-time if in static storage, or at run-time if in a stack or in a heap.
- a-french-anon 4mo ago
  Well, it's true that most of the noise comes from deallocation but I did mention types for good reasons: having your code littered with std::shared_ptr (or worse for Rust where encoding lifetime goes much further than Rc/Arc) is a direct consequence of wanting GC without a global one. The source of my second revelation: GC should be opt-out (e.g. SBCL's arena system) instead of opt-in via refcounted types.
  - adrian_b 4mo ago
    While C and C++ have ugly syntaxes for "malloc", "std::shared_ptr" and the like, it is quite easy to mask that ugliness by using macros and writing thus only cleanly-looking programs, without any noise words and symbols around dynamic allocations. In general, by using macros it is possible to transform so much a C or C++ program, that it becomes unrecognizable as C/C++ and it can mimic reasonably well any other programming language that you might fancy. The problem is when you work in a team, because even if everyone will agree that such programming languages have great deficiencies, it would be impossible to reach a consensus about how the ideal programming language should look like, so eventually the team remains stuck with writing programs in the ugly standard manner.

pmarreck 4mo ago

In Zig, pretty good, but not perfect yet