3 ms·

I think GC is a very cool set of algorithms that have their place for collecting detached elements of a graph. But in a programming language, the baseline memor

I think GC is a very cool set of algorithms that have their place for collecting detached elements of a graph. But in a programming language, the baseline memory model should not be a graph in the first place.

claytonwramsey 3y ago
That's an interesting idea. Can you clarify some more on what the baseline memory model should be, if not a graph?
- 3cats-in-a-coat 3y ago
  I don't want to impose what it should be, as I'm asking myself this question every day. But I can list a few thoughts. The actual memory model is (seemingly) a flat address space, in which processes allocate segments, and everything else arises from there. There's some remapping behind the scenes, some segments are shared, some not, but this is beside the point. And a pattern I see often is arena allocation, where you allocate a large segment and then "virtually" allocate smaller segments inside, down to individual scalar variables. This can happen several times with a lot of memory, but tends to be shallow. For this and many other reasons, I see mutable memory is best structured as a tree, a shallow tree, almost relational, but still permitting nesting, where every item has one owner, one caller, and anything else is routed through that tree. Of course once this semantic model is established, you can reduce and optimize many calls into more direct calls. But you get clear intercept points when you need to decorate, block, remap and so on behaviors in the system. No "action from distance" surprises. Everything should be pass by copy. For larger structures, we need copy-on-write optimization. Which means underneath the mutable tree there is a Directed Acyclic Graph of immutable segments supporting copy-on-write. This is also not new, it's how forking works on Unix for example. Every modern OS utilizes copy-on-write on the file system, in memory, it's also utilized, if we think about it, in network caching systems. But to the process it looks like normal flat address space, where everyone owns their own copy of the content. For collecting unused segments in COW you'd naively need refcounting. There are other approaches that borrow from GC algorithms, in a much simpler and more performant way, because, well, there are no cycles in a DAG. The best part is that once you decide to use copy semantics, you have a plethora of algorithms at your disposal to try and switch between adaptively (including... just copying the thing if it's small enough, which turns out is the fastest approach), while maintaining the same exact semantics for the programmer who doesn't have to care how it works under the hood. Another benefit of copy semantics is that things are copied between processes, or especially between machines in a network, and now your language locally has the same semantics as when it talks to the network, which removes barriers and unifies interfaces. "Write once, run at any scale" if you will. Anyway, this is not comprehensive. And just my opinion. But the problem with graphs is that it is the data structure with least discipline, and most complications. And in most cases those complications do not pay off. Not only they require complex GC, but they result in heavily entangled code, tons of shared mutable state, that gave the whole OOP space a bad name (that it doesn't deserve). This was never a problem with objects, because objects were never supposed to share mutable state. It's a problem of graphs and handles, a system that pretends everyone "has" a mutable piece of state... which it doesn't have, as ownership must be exclusive to be reliable. Discipline of dependency management (packages etc.) often removes cycles and results in a tree or a DAG. This is not coincidental. And there's no reason we can't replicate this at the lower levels to individual objects. Of course, we still need graphs. But not everywhere, and not all the time.
kaba0 3y ago
What else should it be? Besides the memory model being an implementation detail, that is not inherent to the problem, most problems (mathematically for sure, but imo also practically) require general graphs.
- 3cats-in-a-coat 3y ago
  Most problems don't require a general graph. They require subsets of a graph, because everything is a subset of a graph. A list is a subset of a graph. A map is. A tree is. A relational database is. A directed acyclic graph is. But we can stop before it's just any "graph". We can stop at DAG, like neural networks do, or even at tree, like programming language syntax does ("abstract syntax tree", "expression tree"). Code and data are two faces of the same thing. You can also easily reproduce a graph when you need it, by using simpler data with computation on top which builds the graph "on the fly", without it having to be your baseline memory model. You can for example easily describe a graph in two SQL tables: edges and nodes. But SQL state itself is not just any graph, and for a very good reason. You probably know that the first databases were hierarchical and graph. SQL won not because it gave more freedom, but because it removed freedom in a highly specific way, which enabled larger more reliable systems, contrary to naive expectations. Too much freedom is freedom to shoot yourself in the foot, whether you know better or not. Today graph databases have enjoyed a renaissance, but despite they have solid uses cases, they'll never be the baseline type of database. They won't replace relations. People have learned their lesson in databases. But in programming languages, they haven't. It's one of those things where we can't translate knowledge from one area to another, because the words we use for the same effects are different and therefore we can't make the connection. I wrote more here: https://news.ycombinator.com/item?id=37122934 https://news.ycombinator.com/item?id=37122934
  - kaba0 3y ago
    > Most problems don't require a general graph. They require subsets of a graph, because everything is a subset of a graph. A list is a subset of a graph. A map is. A tree is. A relational database is. A directed acyclic graph is. But we can stop before it's just any "graph". We can stop at DAG, like neural networks do, or even at tree, like programming language syntax does ("abstract syntax tree", "expression tree"). Code and data are two faces of the same thing. And if you have a single edge not fulfilling the stricter structure's properties, you are left with a general graph. A single one, hence my point. > but because it removed freedom in a highly specific way, which enabled larger more reliable systems, contrary to naive expectations. That I agree with, but I personally believe it is too restrictive in case of memory models. Especially that the price really is not high -- modern GCs have insanely good throughputs with bounded latency, the only price being a slightly larger memory footprint.

claytonwramsey 3y ago

That's an interesting idea. Can you clarify some more on what the baseline memory model should be, if not a graph?

3cats-in-a-coat 3y ago
I don't want to impose what it should be, as I'm asking myself this question every day. But I can list a few thoughts. The actual memory model is (seemingly) a flat address space, in which processes allocate segments, and everything else arises from there. There's some remapping behind the scenes, some segments are shared, some not, but this is beside the point. And a pattern I see often is arena allocation, where you allocate a large segment and then "virtually" allocate smaller segments inside, down to individual scalar variables. This can happen several times with a lot of memory, but tends to be shallow. For this and many other reasons, I see mutable memory is best structured as a tree, a shallow tree, almost relational, but still permitting nesting, where every item has one owner, one caller, and anything else is routed through that tree. Of course once this semantic model is established, you can reduce and optimize many calls into more direct calls. But you get clear intercept points when you need to decorate, block, remap and so on behaviors in the system. No "action from distance" surprises. Everything should be pass by copy. For larger structures, we need copy-on-write optimization. Which means underneath the mutable tree there is a Directed Acyclic Graph of immutable segments supporting copy-on-write. This is also not new, it's how forking works on Unix for example. Every modern OS utilizes copy-on-write on the file system, in memory, it's also utilized, if we think about it, in network caching systems. But to the process it looks like normal flat address space, where everyone owns their own copy of the content. For collecting unused segments in COW you'd naively need refcounting. There are other approaches that borrow from GC algorithms, in a much simpler and more performant way, because, well, there are no cycles in a DAG. The best part is that once you decide to use copy semantics, you have a plethora of algorithms at your disposal to try and switch between adaptively (including... just copying the thing if it's small enough, which turns out is the fastest approach), while maintaining the same exact semantics for the programmer who doesn't have to care how it works under the hood. Another benefit of copy semantics is that things are copied between processes, or especially between machines in a network, and now your language locally has the same semantics as when it talks to the network, which removes barriers and unifies interfaces. "Write once, run at any scale" if you will. Anyway, this is not comprehensive. And just my opinion. But the problem with graphs is that it is the data structure with least discipline, and most complications. And in most cases those complications do not pay off. Not only they require complex GC, but they result in heavily entangled code, tons of shared mutable state, that gave the whole OOP space a bad name (that it doesn't deserve). This was never a problem with objects, because objects were never supposed to share mutable state. It's a problem of graphs and handles, a system that pretends everyone "has" a mutable piece of state... which it doesn't have, as ownership must be exclusive to be reliable. Discipline of dependency management (packages etc.) often removes cycles and results in a tree or a DAG. This is not coincidental. And there's no reason we can't replicate this at the lower levels to individual objects. Of course, we still need graphs. But not everywhere, and not all the time.

kaba0 3y ago

What else should it be? Besides the memory model being an implementation detail, that is not inherent to the problem, most problems (mathematically for sure, but imo also practically) require general graphs.

3cats-in-a-coat 3y ago
Most problems don't require a general graph. They require subsets of a graph, because everything is a subset of a graph. A list is a subset of a graph. A map is. A tree is. A relational database is. A directed acyclic graph is. But we can stop before it's just any "graph". We can stop at DAG, like neural networks do, or even at tree, like programming language syntax does ("abstract syntax tree", "expression tree"). Code and data are two faces of the same thing. You can also easily reproduce a graph when you need it, by using simpler data with computation on top which builds the graph "on the fly", without it having to be your baseline memory model. You can for example easily describe a graph in two SQL tables: edges and nodes. But SQL state itself is not just any graph, and for a very good reason. You probably know that the first databases were hierarchical and graph. SQL won not because it gave more freedom, but because it removed freedom in a highly specific way, which enabled larger more reliable systems, contrary to naive expectations. Too much freedom is freedom to shoot yourself in the foot, whether you know better or not. Today graph databases have enjoyed a renaissance, but despite they have solid uses cases, they'll never be the baseline type of database. They won't replace relations. People have learned their lesson in databases. But in programming languages, they haven't. It's one of those things where we can't translate knowledge from one area to another, because the words we use for the same effects are different and therefore we can't make the connection. I wrote more here: https://news.ycombinator.com/item?id=37122934 https://news.ycombinator.com/item?id=37122934
- kaba0 3y ago
  > Most problems don't require a general graph. They require subsets of a graph, because everything is a subset of a graph. A list is a subset of a graph. A map is. A tree is. A relational database is. A directed acyclic graph is. But we can stop before it's just any "graph". We can stop at DAG, like neural networks do, or even at tree, like programming language syntax does ("abstract syntax tree", "expression tree"). Code and data are two faces of the same thing. And if you have a single edge not fulfilling the stricter structure's properties, you are left with a general graph. A single one, hence my point. > but because it removed freedom in a highly specific way, which enabled larger more reliable systems, contrary to naive expectations. That I agree with, but I personally believe it is too restrictive in case of memory models. Especially that the price really is not high -- modern GCs have insanely good throughputs with bounded latency, the only price being a slightly larger memory footprint.