6 ms·

The most depressing thing about memory management is that there's been 0 innovation in the past 10+ years, and it seems like we're fundamentally at a dead end w

by tomp 4y ago

The most depressing thing about memory management is that there's been 0 innovation in the past 10+ years, and it seems like we're fundamentally at a dead end where only tradeoffs exist. There are basically 3 options going forward: - manual memory management (C++, Rust) "I can verify your code but you need to `unsafe` to do anything interesting" - reference counting (Swift) "I leak memory through cycles" - tracing GC (Java, Go) "I stop the world and trash your swap" The rest is engineering (e.g. Go reduced the "stop-the-world" phase to O(1) from O(n) where n is the size of memory)

kevin_thibedeau 4y ago
Nim has GC with controllable max time slices.
olliej 4y ago
By your definition of what is "engineering" I feel like all improvements or changes are just engineering. Because you've described three classes: * No explicit runtime knowledge of whether a given object is alive * Explicitly tracking how many references there are to an object * Searching the program memory to see if there are references to an object that make it live I'm not sure there is another class of memory management that isn't one of these three? It's also somewhat unfair to rust - the static and parametric lifetime management _is_ new, and doesn't "need unsafe" to do anything interesting; and to tracing GCs modern GCs have developed ever more techniques to not be "stop the world" including parallel, iterative, lock free tracing, etc.
- tomp 4y ago
  All of the ones you mentioned above still involve a stop-the-world phase, however brief. The keywords you’re looking for are “fullya-concurrent” or “on-the-fly” GC, but I’m not aware of any production runtime with those.
  - WJW 4y ago
    Doesn't the JVM ecosystem have fully concurrent GC with the Azul C4 collector? It's not my favorite set of languages but I don't think we can realistically claim the JVM is not a production-ready runtime.
    - tomp 4y ago
      Yes, "pauseless" is another keyword I forgot to mention. Azul uses a read barrier which I guess makes it so slow that noone uses it (and the same ideas haven't made their way into JDK collectors). The previous version required hardware-supported read barrier, now AFAIK they only require a Linux kernel extension.
oddity 4y ago
Depends on how you define innovation. The hard truth is that there is no free lunch. We like to pretend we're using a Turing machine, but the moment you start caring about performance or memory limits, the abstraction breaks down and you realize that physics dictates that we have a finite state machine instead. This was "solved" about as well as it could be many years ago with however many billions were poured into GC R&D for all the people who don't care deeply about the limits, but nothing is going to magic away that fundamental trade-off unless we discover new physics. Every innovation since then has been about developing workarounds to deal with that trade-off in more or less sophisticated ways.
Hirrolot 4y ago
There's also an interesting programming language called Neut [1]: > A dependently-typed programming language with compile-time malloc/free determination. Also related "ASAP: As Static As Possible memory management" [2]. I think most of the "innovation" you're talking about is happening in the programming language design field. [1] https://github.com/vekatze/neut https://github.com/vekatze/neut [2] https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf
WJW 4y ago
Well first of all there is tons of research being done right now into linear type systems, which have the potential to statically verify a whole bunch of things that Rusts borrow checking still struggles with. It's a very hot field of CS research atm. I also think that you discount the amount of advancement that has been done in the development of pauseless GC. It's a shame that many of these are only available on the JVM (and sometimes only at a steep price), but I have no doubt the main ideas will eventually get adopted by other languages. But really there are so many unexplored niches. AFAIK no current architecture has hardware support for read/write barriers, having that would make pauseless GC much more feasible. Zig has been experimenting with letting people use custom memory allocators, but there are many more avenues there that have not been explored. Why not have a language feature for "this value lives outside GC management" similar to Haskells compact regions but more powerful? Nim has some cool ideas about mixing reference counting for most values and a more general GC for cycle breaking. Mixing GC and non-GC values is also still a seriously underexplored field IMO.
- cb321 4y ago
  Nim has for a long time had automatically managed `ref` as well as manually managed `ptr` which is similar to (perhaps directly inspired?) your last comment, and has several options for Auto Mem Mgmt.
- whizzter 4y ago
  Actually _most_ hardware architectures has support for read/write barriers, it's just that the technique is patented. :( Also fully agreeing on the GC/non-GC mixing being underexplored, however outside of compiler analysis optimizations this would require language support and the intersection of people who would be interested in this mix doesn't seem that large.
verdagon 4y ago
(Disclaimer: Vale lead here!) You might be interested in what we're working on with Vale, its core innovations happen to be around memory management. Some off the top of my head: * Generational references, for single-ownership memory safety without garbage collection, ref counting, or borrow checking. [0] * Higher RAII, a form of "linear typing" that lets us add parameters and returns to destructors, to enforce that if you do X, you won't forget to do Y. [1] * Regions, which could make it so we can eliminate most generation checks in Vale. If it catches on, it could also help other paradigms and get reference counting back into the fight. (Note, we aren't done with this one yet, it's about 3-8 weeks out) [2] Those are just a few off the top of my head, there are a lot of crazy ideas floating around there. I'm hoping they'll combine to form something like an easier more flexible Rust, because that's the holy grail of language design, to me at least. A few years ago, I was also pretty down about the state of memory innovation... it felt like not much had been discovered since borrow checking. I knew there had to be something with its speed, but the flexibility of C++. Perhaps we found it! [0] https://verdagon.dev/blog/generational-references https://verdagon.dev/blog/generational-references [1] https://verdagon.dev/blog/higher-raii-7drl https://verdagon.dev/blog/higher-raii-7drl [2] https://verdagon.dev/blog/zero-cost-refs-regions https://verdagon.dev/blog/zero-cost-refs-regions
- avgcorrection 4y ago
  Don’t save the disclaimer for last.
  - verdagon 4y ago
    Happy to oblige, edited it upward. Hope your day is going well =)
    - avgcorrection 4y ago
      It is, and thanks!
- zozbot234 4y ago
  The real disclaimer should perhaps be that this stuff is something that's still being worked on wrt. production use, just like many other kinds of PL research. Once it's ready, Rust will probably be in a very good position to adopt these features.
vitiral 4y ago
I'm doing some pretty different things with civboot.org and http://github.com/vitiral/fngi http://github.com/vitiral/fngi, but most would consider them regressions instead of advancements. The goal is to have fragmentation-free memory managers that can work with minimal hardware support. This has limitations like block size (only 4k blocks allowed), but forces me to innovate. For instance, I'm planning on an "arena stack", where most functions simply use the arena on the top of the stack. Functions can reserve an arena on the arena-stack, call a function, then copy essential data and drop the arena, reducing fragmentation. Might not be the kind of ideas you were thinking, but I consider this simpler than the current abstractions which hide complexity.

by tomp 4y ago

kevin_thibedeau 4y ago

Nim has GC with controllable max time slices.

olliej 4y ago

By your definition of what is "engineering" I feel like all improvements or changes are just engineering. Because you've described three classes: * No explicit runtime knowledge of whether a given object is alive * Explicitly tracking how many references there are to an object * Searching the program memory to see if there are references to an object that make it live I'm not sure there is another class of memory management that isn't one of these three? It's also somewhat unfair to rust - the static and parametric lifetime management _is_ new, and doesn't "need unsafe" to do anything interesting; and to tracing GCs modern GCs have developed ever more techniques to not be "stop the world" including parallel, iterative, lock free tracing, etc.

tomp 4y ago
All of the ones you mentioned above still involve a stop-the-world phase, however brief. The keywords you’re looking for are “fullya-concurrent” or “on-the-fly” GC, but I’m not aware of any production runtime with those.
- WJW 4y ago
  Doesn't the JVM ecosystem have fully concurrent GC with the Azul C4 collector? It's not my favorite set of languages but I don't think we can realistically claim the JVM is not a production-ready runtime.
  - tomp 4y ago
    Yes, "pauseless" is another keyword I forgot to mention. Azul uses a read barrier which I guess makes it so slow that noone uses it (and the same ideas haven't made their way into JDK collectors). The previous version required hardware-supported read barrier, now AFAIK they only require a Linux kernel extension.

oddity 4y ago

Depends on how you define innovation. The hard truth is that there is no free lunch. We like to pretend we're using a Turing machine, but the moment you start caring about performance or memory limits, the abstraction breaks down and you realize that physics dictates that we have a finite state machine instead. This was "solved" about as well as it could be many years ago with however many billions were poured into GC R&D for all the people who don't care deeply about the limits, but nothing is going to magic away that fundamental trade-off unless we discover new physics. Every innovation since then has been about developing workarounds to deal with that trade-off in more or less sophisticated ways.

Hirrolot 4y ago

There's also an interesting programming language called Neut [1]: > A dependently-typed programming language with compile-time malloc/free determination. Also related "ASAP: As Static As Possible memory management" [2]. I think most of the "innovation" you're talking about is happening in the programming language design field. [1] https://github.com/vekatze/neut https://github.com/vekatze/neut [2] https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-908.pdf

WJW 4y ago

Well first of all there is tons of research being done right now into linear type systems, which have the potential to statically verify a whole bunch of things that Rusts borrow checking still struggles with. It's a very hot field of CS research atm. I also think that you discount the amount of advancement that has been done in the development of pauseless GC. It's a shame that many of these are only available on the JVM (and sometimes only at a steep price), but I have no doubt the main ideas will eventually get adopted by other languages. But really there are so many unexplored niches. AFAIK no current architecture has hardware support for read/write barriers, having that would make pauseless GC much more feasible. Zig has been experimenting with letting people use custom memory allocators, but there are many more avenues there that have not been explored. Why not have a language feature for "this value lives outside GC management" similar to Haskells compact regions but more powerful? Nim has some cool ideas about mixing reference counting for most values and a more general GC for cycle breaking. Mixing GC and non-GC values is also still a seriously underexplored field IMO.

cb321 4y ago
Nim has for a long time had automatically managed `ref` as well as manually managed `ptr` which is similar to (perhaps directly inspired?) your last comment, and has several options for Auto Mem Mgmt.
whizzter 4y ago
Actually _most_ hardware architectures has support for read/write barriers, it's just that the technique is patented. :( Also fully agreeing on the GC/non-GC mixing being underexplored, however outside of compiler analysis optimizations this would require language support and the intersection of people who would be interested in this mix doesn't seem that large.

verdagon 4y ago

(Disclaimer: Vale lead here!) You might be interested in what we're working on with Vale, its core innovations happen to be around memory management. Some off the top of my head: * Generational references, for single-ownership memory safety without garbage collection, ref counting, or borrow checking. [0] * Higher RAII, a form of "linear typing" that lets us add parameters and returns to destructors, to enforce that if you do X, you won't forget to do Y. [1] * Regions, which could make it so we can eliminate most generation checks in Vale. If it catches on, it could also help other paradigms and get reference counting back into the fight. (Note, we aren't done with this one yet, it's about 3-8 weeks out) [2] Those are just a few off the top of my head, there are a lot of crazy ideas floating around there. I'm hoping they'll combine to form something like an easier more flexible Rust, because that's the holy grail of language design, to me at least. A few years ago, I was also pretty down about the state of memory innovation... it felt like not much had been discovered since borrow checking. I knew there had to be something with its speed, but the flexibility of C++. Perhaps we found it! [0] https://verdagon.dev/blog/generational-references https://verdagon.dev/blog/generational-references [1] https://verdagon.dev/blog/higher-raii-7drl https://verdagon.dev/blog/higher-raii-7drl [2] https://verdagon.dev/blog/zero-cost-refs-regions https://verdagon.dev/blog/zero-cost-refs-regions

avgcorrection 4y ago
Don’t save the disclaimer for last.
- verdagon 4y ago
  Happy to oblige, edited it upward. Hope your day is going well =)
  - avgcorrection 4y ago
    It is, and thanks!
zozbot234 4y ago
The real disclaimer should perhaps be that this stuff is something that's still being worked on wrt. production use, just like many other kinds of PL research. Once it's ready, Rust will probably be in a very good position to adopt these features.

vitiral 4y ago

I'm doing some pretty different things with civboot.org and http://github.com/vitiral/fngi http://github.com/vitiral/fngi, but most would consider them regressions instead of advancements. The goal is to have fragmentation-free memory managers that can work with minimal hardware support. This has limitations like block size (only 4k blocks allowed), but forces me to innovate. For instance, I'm planning on an "arena stack", where most functions simply use the arena on the top of the stack. Functions can reserve an arena on the arena-stack, call a function, then copy essential data and drop the arena, reducing fragmentation. Might not be the kind of ideas you were thinking, but I consider this simpler than the current abstractions which hide complexity.