6 ms·

ORC – Nim's new memory managment

240 points by mratsim 6y ago

treeform 6y ago
It's great that Nim is pushing boundaries in the memory management space. I remember reading how Java and Go were able to scan large heaps faster and faster. But what if you don't even scan the heap? Brilliant.
- disruptek 6y ago
  Also, we don't scan "live" memory, which matches your intuition -- why would we?
- int_19h 6y ago
  I may be missing something here, but reference counting + cycle detector is what Python has been doing for many years now? In my experience, it's the worst of both worlds. You no longer have the full determinism of reference counting - you only retain it for non-cyclical structures, and in a large enough program, it can be easy to introduce those without noticing (with callbacks etc). But you still have the overhead of RC on every reference assignment.
  - nikki93 6y ago
    An important thing is that with move semantics and good analysis for those at compile time, a lot of assignments can be moves and not need refcount changes. This fact actually comes up in Nim with ARC a lot. Refcounted refs are pretty explicit in Nim and you can use value types for the most part. That allows it to be pretty clear in your architecture what data has a refcount associated, and you can opt into that in a conscious way. The explicitness doesn't make the code look weird or unidiomatic either, it feels like a natural part of the language that it gives you the tools to do this and is clearly a core part of its design. The way I've gone about it in Nim so far, the refcounts are actually rare, especially copies (which is where increments occur). You can also look at the generated C and it's pretty clear where the overhead is happening, and I've found that in practice it happens rarely enough that this makes a lot of sense. It's pretty different from how things go in Python. In my project I also actually just use ARC and for sure don't have cycles, and the destructor calls are deterministic. Here's a talk that's somewhat related (refcounts in the "Lobster" programming language, and how it also elides refcount calls using compile-time logic (which ends up being equivalent-ish to a framing in terms of move semantics)): https://www.youtube.com/watch?v=WUkYIdv9B8c https://www.youtube.com/watch?v=WUkYIdv9B8c
- imtringued 6y ago
  Reference counting is an old hat.
skulk 6y ago
There's a lot of praise for ORC[1], but is there any discussion of the downsides of this approach? One potential downside is increased compile times, something that hasn't ever been a problem for Nim. However, I'm not sure what kind of load ORC puts on the compiler, and how that will scale to large projects. [1]: https://forum.nim-lang.org/t/6483 https://forum.nim-lang.org/t/6483
- planetis 6y ago
  I have switched to `--gc:arc`/`orc` for some time but I haven't noticed any difference in compilation speed.
scottlamb 6y ago
Couple questions about how this works. 1. Is "acyclic" checked by the compiler in any way? or is the programmer basically saying "I'm responsible for making sure this there are no cycles, and I accept that memory will leak if I'm wrong"? I imagine the latter because the compiler check sounds infeasible. (Similarly Rust's memory safety doesn't guarantee there are no leaks.) 2. How does it work with multiple threads? I see a Q/A here [1] which says > C: We do not do atomic reference counting, we have a shared heap where you move subgraphs completely to a different thread. A graph is always owned by a single thread, there is no need to make the reference counting atomic. We have mechanisms in development to ensure this at runtime or at compile-time, inspired by the Pony programming language. and would be curious to read the one-page rather than one-paragraph version of this... [1] https://forum.nim-lang.org/t/5734 https://forum.nim-lang.org/t/5734
- disruptek 6y ago
  You can read about the reasoning behind subgraph ownership here: https://github.com/nim-lang/RFCs/issues/244 https://github.com/nim-lang/RFCs/issues/244
  - scottlamb 6y ago
    Thanks. If I'm understanding correctly: with Isolated (which is still being built), any given object is reachable from only one thread. It can be used for message-passing but not shared memory. It looks like they've talked about shared memory as important for efficiency, eg https://nim-lang.org/araq/concurrency.html https://nim-lang.org/araq/concurrency.html (from back in 2013 apparently, and I don't know if it's a "canonical" project view or not) says: > For real CPU efficiency on commodity hardware shared memory is unavoidable. Message passing doesn't cut it and ultimately doesn't address the same problems. (http://www.yosefk.com/blog/parallelism-and-concurrency-need-different-tools.html http://www.yosefk.com/blog/parallelism-and-concurrency-need-...) Also shared mutable memory is unavoidable in a systems programming context. (I agree with that statement, fwiw.) It sounds like they haven't built support for shared memory yet; correct? and as this Isolated construct is also in progress, I think there's basically no threading support in Nim yet? Are they still intending to build shared memory as well as Isolated? Is it intended to be memory-safe (as in the Rust terminology) or not?
gok 6y ago
Extremely cool. Reference counting with cycle collection might be an idea whose time has finally come.

240 points by mratsim 6y ago

It's great that Nim is pushing boundaries in the memory management space. I remember reading how Java and Go were able to scan large heaps faster and faster. But what if you don't even scan the heap? Brilliant.

disruptek 6y ago
Also, we don't scan "live" memory, which matches your intuition -- why would we?
int_19h 6y ago
I may be missing something here, but reference counting + cycle detector is what Python has been doing for many years now? In my experience, it's the worst of both worlds. You no longer have the full determinism of reference counting - you only retain it for non-cyclical structures, and in a large enough program, it can be easy to introduce those without noticing (with callbacks etc). But you still have the overhead of RC on every reference assignment.
- nikki93 6y ago
  An important thing is that with move semantics and good analysis for those at compile time, a lot of assignments can be moves and not need refcount changes. This fact actually comes up in Nim with ARC a lot. Refcounted refs are pretty explicit in Nim and you can use value types for the most part. That allows it to be pretty clear in your architecture what data has a refcount associated, and you can opt into that in a conscious way. The explicitness doesn't make the code look weird or unidiomatic either, it feels like a natural part of the language that it gives you the tools to do this and is clearly a core part of its design. The way I've gone about it in Nim so far, the refcounts are actually rare, especially copies (which is where increments occur). You can also look at the generated C and it's pretty clear where the overhead is happening, and I've found that in practice it happens rarely enough that this makes a lot of sense. It's pretty different from how things go in Python. In my project I also actually just use ARC and for sure don't have cycles, and the destructor calls are deterministic. Here's a talk that's somewhat related (refcounts in the "Lobster" programming language, and how it also elides refcount calls using compile-time logic (which ends up being equivalent-ish to a framing in terms of move semantics)): https://www.youtube.com/watch?v=WUkYIdv9B8c https://www.youtube.com/watch?v=WUkYIdv9B8c
imtringued 6y ago
Reference counting is an old hat.

skulk 6y ago

There's a lot of praise for ORC[1], but is there any discussion of the downsides of this approach? One potential downside is increased compile times, something that hasn't ever been a problem for Nim. However, I'm not sure what kind of load ORC puts on the compiler, and how that will scale to large projects. [1]: https://forum.nim-lang.org/t/6483 https://forum.nim-lang.org/t/6483

planetis 6y ago
I have switched to `--gc:arc`/`orc` for some time but I haven't noticed any difference in compilation speed.

scottlamb 6y ago

Couple questions about how this works. 1. Is "acyclic" checked by the compiler in any way? or is the programmer basically saying "I'm responsible for making sure this there are no cycles, and I accept that memory will leak if I'm wrong"? I imagine the latter because the compiler check sounds infeasible. (Similarly Rust's memory safety doesn't guarantee there are no leaks.) 2. How does it work with multiple threads? I see a Q/A here [1] which says > C: We do not do atomic reference counting, we have a shared heap where you move subgraphs completely to a different thread. A graph is always owned by a single thread, there is no need to make the reference counting atomic. We have mechanisms in development to ensure this at runtime or at compile-time, inspired by the Pony programming language. and would be curious to read the one-page rather than one-paragraph version of this... [1] https://forum.nim-lang.org/t/5734 https://forum.nim-lang.org/t/5734

disruptek 6y ago
You can read about the reasoning behind subgraph ownership here: https://github.com/nim-lang/RFCs/issues/244 https://github.com/nim-lang/RFCs/issues/244
- scottlamb 6y ago
  Thanks. If I'm understanding correctly: with Isolated (which is still being built), any given object is reachable from only one thread. It can be used for message-passing but not shared memory. It looks like they've talked about shared memory as important for efficiency, eg https://nim-lang.org/araq/concurrency.html https://nim-lang.org/araq/concurrency.html (from back in 2013 apparently, and I don't know if it's a "canonical" project view or not) says: > For real CPU efficiency on commodity hardware shared memory is unavoidable. Message passing doesn't cut it and ultimately doesn't address the same problems. (http://www.yosefk.com/blog/parallelism-and-concurrency-need-different-tools.html http://www.yosefk.com/blog/parallelism-and-concurrency-need-...) Also shared mutable memory is unavoidable in a systems programming context. (I agree with that statement, fwiw.) It sounds like they haven't built support for shared memory yet; correct? and as this Isolated construct is also in progress, I think there's basically no threading support in Nim yet? Are they still intending to build shared memory as well as Isolated? Is it intended to be memory-safe (as in the Rust terminology) or not?

gok 6y ago

Extremely cool. Reference counting with cycle collection might be an idea whose time has finally come.