7 ms·

The part of GC which causes the most latency issues is compaction rather than merely collection. Using a language like rust won’t help if you have memory fragme

The part of GC which causes the most latency issues is compaction rather than merely collection. Using a language like rust won’t help if you have memory fragmentation and indeed allocation tends to be much faster with a GC than with malloc. I think the advantages of rust are more to do with often avoiding heap allocation entirely (and predictably) or value semantics leading to fewer small allocations or the language’s semantics not forcing you to choose between more reliable allocation-heavy immutable-everything code and faster harder-to-think-about mutation-heavy code.

gwbas1c 6y ago
IMO, the main advantage of Rust is that it doesn't require an extensive runtime in order to have memory safety. This allows you to write a library like an image processor or embedded database without using C / C++. Otherwise, if you wrote your image processor in C# or Java, it becomes hard to call your library from Python or Node because you have to require the entire VM. Likewise, you can ship an application binary that has no requirements for a runtime. (Your application binary doesn't require a JVM, CLR, Mono, Python, Node, or some other runtime.) I've been through the Rust book twice but I'm just getting to the point of trying to write something in it. The mental model is very different. Coming from C# / Java / Javascript / Objective C; I'm wondering how many hours I need before I can get my head into Rust?
- shpongled 6y ago
  Easiest way to get started if you're coming from a GC background is to just liberally `.clone()` everything in Rust. Once you're used to move semantics and the syntax, then you can start messing around with borrowing. It definitely has a learning curve, but I find it a breeze to write once you grok the ownership rules.
- pjmlp 6y ago
  On the other hand it requires jumping through hoops to make borrow checker friendly architecture designs, or fiddle your code base with Rc<> types everywhere. And when their count reaches 0, you have your stop the world, unless you move the destruction into a background thread, thus manually emulating a tracing GC.
  - pkolaczk 6y ago
    When RC drops to zero it is not STW. It is stop the current thread only. And even that can be trivially solved by background deallocation. Solving latency problem of GC is far from trivial.
    - pjmlp 6y ago
      As if stop the current thread isn't the world the world in single threaded programs. I explicitly mentioned how moving into a background thread is poor man's tracing GC. Languages like D and C# offer mechanisms for deterministic deallocation, while keep the productivity of a tracing GC.
      - orneryostrich 6y ago
        But if your program is single-threaded, you don’t need an `Rc` type. The whole point of `Rc` and `std::shared_ptr` is so an object can have one owning pointer per thread, in cases where you’re not sure which thread will finish using the object last.
        pjmlp 6y ago
        Sure you do, try to implement a Gtk-rs application without the Rc<RefCell<>> dance for widgets state.
      - pkolaczk 6y ago
        Finalizers (IDisposable) or try-with-resources are not equally strong as deterministic destruction in C++ or Rust. Or did you mean a different feature for deterministic destruction in C# that I don't know of? I'm quite curious.
        pjmlp 6y ago
        You can stack allocate objects, so those are alive just until the end of the stack. Then there native memory allocation and safe handles. IDisposable and Finalizers aren't the same thing, actually, although they happen to be used together as means to combine deterministic destruction alongside GC based destruction. You can also make use of lambdas or implicit IDisposable implementations via helper methods, that generate code similar to memory regions or arenas in C++, but in .NET. Finally, many tend to forget that .NET was designed to support C++ as well, so it is also possible to generate MSIL code that ensures deterministic destruction RAII style, naturally this falls into a bit more advanced programming, but it can be hidden away in helper classes.
- pizza234 6y ago
  I'm learning Rust as well; in my opinion, starting with small guided projects is the most stimulating and incremental approach, although unfortunately, I find that starting practicing Rust - differently from other languages - requires "having read the whole reference". The resources I've reviewed are: - Rustlings: I personally don't like the project; it's exercises for the sake of exercising, which may be good or not, depending on the person/approach. - Ferrous systems exercises (set of mini projects): very small and simple, but interesting to work on. I think they're a very good resource to try immediately after reading the book. - Rust Programming By Example (book): if one finds one or more projects interesting to work on, this is a very fun book. it's "not-perfectly-produced" though, if you know what I mean. - Hands-On Data Structures and Algorithms in Rust (udemy): even if one doesn't like algorithms, I think working with data structures implicity develops an understanding of "getting the head into Rust". - Build your own Jira with Rust (https://github.com/LukeMathWalker/build-your-own-jira-with-rust https://github.com/LukeMathWalker/build-your-own-jira-with-r...): An exercise-based approach to practice; probably a good alternative to the previous resource, for those who really don't want to work with A./D.S.. - Talent Plan (https://github.com/pingcap/talent-plan https://github.com/pingcap/talent-plan): like the previous, but more technical. Have fun :-)
  - gwbas1c 6y ago
    > I find that starting practicing Rust - differently from other languages - requires "having read the whole reference". And that's the problem that I have. In high school I was handheld into C / C++ with weekly lessons. By the time I started my career I abandoned C because the things I worked on professionally had no benefit from manual memory management. Now the thing that I want to write, an embedded database, requires manual memory management and no runtime. I could, in theory, go back and do it in C. It'd be slow working in a language that I haven't done anything in since 2002, but at least I'm familiar with all the conventions. Do I basically need to spend 40-80 hours doing silly exercises just to ease into the new conventions and mental model?
    - pizza234 6y ago
      It's not really clear what you mean with "silly exercises". One of the resources is a course for building "a [...] networked, parallel and asynchronous key/value store", which is far from being a "silly exercise". Even ignoring that, it's a matter of big picture. If learning Rust is only for this project, or only to be "quickly proficient in a new language", then I don't think it fits the specific case. The may be alternatives; I don't have experience (somebody else can surely advice better) something like C++ with smart pointers or memory safe D, I guess, could fit. In the big picture of a career, or even in the context of a single company, spending 40/80 hours to be proficient in a language is essentially an insignificant time.
darksaints 6y ago
Don't forget scanning. Yes, moving blocks of memory around is expensive, but it can also be done concurrently. Scanning, AFAIK, cannot be done concurrently, and thus remains the primary blocker to lower latency. And scanning is something that is entirely eliminated with static memory management.
- haxen 6y ago
  All the modern GCs scan the heap concurrently, the hardest problem is scanning the GC roots in the call stack. ZGC is currently implementing concurrent stack scanning.
- wokkel 6y ago
  Yes it can. Stop the world collection is a thing of the past. See for example: https://developers.redhat.com/blog/2019/06/27/shenandoah-gc-in-jdk-13-part-1-load-reference-barriers/ https://developers.redhat.com/blog/2019/06/27/shenandoah-gc-...
- pron 6y ago
  Scanning is most certainly done concurrently with ZGC. Even root scanning is on its way to become fully concurrent, which is why we're nearing the goal of <1ms latency.
  - jerrinot 6y ago
    I didn't know about fully concurrent root scanning, thank you! How does root scanning work wrt to Loom? Are stacks of virtual threads treated as roots? I guess there is no other option?
    - pron 6y ago
      No, virtual thread stacks are not roots! This is one of the main design highlights of the current Loom implementation. In fact, at least currently, the VM doesn't maintain any list of virtual threads at all. They are just Java objects, but the GC does treat them specially.
nestorD 6y ago
One thing I love about Rust is that allocation are very explicit and easily spotted which helps a lot when one wants to avoid them. I found C++ to be treacherous around corners cases on this subject.
- pjmlp 6y ago
  Unless you are going to review the whole code, written from scratch, there is no way to actually be aware of all allocations in Rust without help from a memory profiler.
  - cogman10 6y ago
    I think the point was more that allocations are fairly explicit. There are a few places in C++ where allocation can happen pretty much invisibly. A copy constructor is an example of that. You might see a new allocation simply by calling a method. With rust, you usually won't see an allocation unless it is explicitly called for. You can follow the call tree and very easily pick out when those allocations are happening.
    - pjmlp 6y ago
      Rust also has operator overloading and implicit conversations, both can cause memory allocations.
      - cogman10 6y ago
        Certainly. However, both of those things (from what I've seen) are fairly rare. The C++ copy constructor comes up fairly frequently.
        pjmlp 6y ago
        Like unsafe Rust code, there is the theory and then there are the code bases that one finds out in the wild on crates.io and in-house, not necessarily using best practices.
bananaface 6y ago
Can't you just allocate a huge block up-front and throw stuff into it with a custom allocator? I don't know if Rust allows you to do that kind of thing.
- steveklabnik 6y ago
  Yes. There are some trade offs, you can’t parameterize the standard library data structures over an allocator yet, for example.
AtlasBarfed 6y ago
So arena allocation and buffer reuse?

gwbas1c 6y ago

IMO, the main advantage of Rust is that it doesn't require an extensive runtime in order to have memory safety. This allows you to write a library like an image processor or embedded database without using C / C++. Otherwise, if you wrote your image processor in C# or Java, it becomes hard to call your library from Python or Node because you have to require the entire VM. Likewise, you can ship an application binary that has no requirements for a runtime. (Your application binary doesn't require a JVM, CLR, Mono, Python, Node, or some other runtime.) I've been through the Rust book twice but I'm just getting to the point of trying to write something in it. The mental model is very different. Coming from C# / Java / Javascript / Objective C; I'm wondering how many hours I need before I can get my head into Rust?

shpongled 6y ago
Easiest way to get started if you're coming from a GC background is to just liberally `.clone()` everything in Rust. Once you're used to move semantics and the syntax, then you can start messing around with borrowing. It definitely has a learning curve, but I find it a breeze to write once you grok the ownership rules.
pjmlp 6y ago
On the other hand it requires jumping through hoops to make borrow checker friendly architecture designs, or fiddle your code base with Rc<> types everywhere. And when their count reaches 0, you have your stop the world, unless you move the destruction into a background thread, thus manually emulating a tracing GC.
- pkolaczk 6y ago
  When RC drops to zero it is not STW. It is stop the current thread only. And even that can be trivially solved by background deallocation. Solving latency problem of GC is far from trivial.
  - pjmlp 6y ago
    As if stop the current thread isn't the world the world in single threaded programs. I explicitly mentioned how moving into a background thread is poor man's tracing GC. Languages like D and C# offer mechanisms for deterministic deallocation, while keep the productivity of a tracing GC.
    - orneryostrich 6y ago
      But if your program is single-threaded, you don’t need an `Rc` type. The whole point of `Rc` and `std::shared_ptr` is so an object can have one owning pointer per thread, in cases where you’re not sure which thread will finish using the object last.
      - pjmlp 6y ago
        Sure you do, try to implement a Gtk-rs application without the Rc<RefCell<>> dance for widgets state.
    - pkolaczk 6y ago
      Finalizers (IDisposable) or try-with-resources are not equally strong as deterministic destruction in C++ or Rust. Or did you mean a different feature for deterministic destruction in C# that I don't know of? I'm quite curious.
      - pjmlp 6y ago
        You can stack allocate objects, so those are alive just until the end of the stack. Then there native memory allocation and safe handles. IDisposable and Finalizers aren't the same thing, actually, although they happen to be used together as means to combine deterministic destruction alongside GC based destruction. You can also make use of lambdas or implicit IDisposable implementations via helper methods, that generate code similar to memory regions or arenas in C++, but in .NET. Finally, many tend to forget that .NET was designed to support C++ as well, so it is also possible to generate MSIL code that ensures deterministic destruction RAII style, naturally this falls into a bit more advanced programming, but it can be hidden away in helper classes.
pizza234 6y ago
I'm learning Rust as well; in my opinion, starting with small guided projects is the most stimulating and incremental approach, although unfortunately, I find that starting practicing Rust - differently from other languages - requires "having read the whole reference". The resources I've reviewed are: - Rustlings: I personally don't like the project; it's exercises for the sake of exercising, which may be good or not, depending on the person/approach. - Ferrous systems exercises (set of mini projects): very small and simple, but interesting to work on. I think they're a very good resource to try immediately after reading the book. - Rust Programming By Example (book): if one finds one or more projects interesting to work on, this is a very fun book. it's "not-perfectly-produced" though, if you know what I mean. - Hands-On Data Structures and Algorithms in Rust (udemy): even if one doesn't like algorithms, I think working with data structures implicity develops an understanding of "getting the head into Rust". - Build your own Jira with Rust (https://github.com/LukeMathWalker/build-your-own-jira-with-rust https://github.com/LukeMathWalker/build-your-own-jira-with-r...): An exercise-based approach to practice; probably a good alternative to the previous resource, for those who really don't want to work with A./D.S.. - Talent Plan (https://github.com/pingcap/talent-plan https://github.com/pingcap/talent-plan): like the previous, but more technical. Have fun :-)
- gwbas1c 6y ago
  > I find that starting practicing Rust - differently from other languages - requires "having read the whole reference". And that's the problem that I have. In high school I was handheld into C / C++ with weekly lessons. By the time I started my career I abandoned C because the things I worked on professionally had no benefit from manual memory management. Now the thing that I want to write, an embedded database, requires manual memory management and no runtime. I could, in theory, go back and do it in C. It'd be slow working in a language that I haven't done anything in since 2002, but at least I'm familiar with all the conventions. Do I basically need to spend 40-80 hours doing silly exercises just to ease into the new conventions and mental model?
  - pizza234 6y ago
    It's not really clear what you mean with "silly exercises". One of the resources is a course for building "a [...] networked, parallel and asynchronous key/value store", which is far from being a "silly exercise". Even ignoring that, it's a matter of big picture. If learning Rust is only for this project, or only to be "quickly proficient in a new language", then I don't think it fits the specific case. The may be alternatives; I don't have experience (somebody else can surely advice better) something like C++ with smart pointers or memory safe D, I guess, could fit. In the big picture of a career, or even in the context of a single company, spending 40/80 hours to be proficient in a language is essentially an insignificant time.

darksaints 6y ago

Don't forget scanning. Yes, moving blocks of memory around is expensive, but it can also be done concurrently. Scanning, AFAIK, cannot be done concurrently, and thus remains the primary blocker to lower latency. And scanning is something that is entirely eliminated with static memory management.

haxen 6y ago
All the modern GCs scan the heap concurrently, the hardest problem is scanning the GC roots in the call stack. ZGC is currently implementing concurrent stack scanning.
wokkel 6y ago
Yes it can. Stop the world collection is a thing of the past. See for example: https://developers.redhat.com/blog/2019/06/27/shenandoah-gc-in-jdk-13-part-1-load-reference-barriers/ https://developers.redhat.com/blog/2019/06/27/shenandoah-gc-...
pron 6y ago
Scanning is most certainly done concurrently with ZGC. Even root scanning is on its way to become fully concurrent, which is why we're nearing the goal of <1ms latency.
- jerrinot 6y ago
  I didn't know about fully concurrent root scanning, thank you! How does root scanning work wrt to Loom? Are stacks of virtual threads treated as roots? I guess there is no other option?
  - pron 6y ago
    No, virtual thread stacks are not roots! This is one of the main design highlights of the current Loom implementation. In fact, at least currently, the VM doesn't maintain any list of virtual threads at all. They are just Java objects, but the GC does treat them specially.

nestorD 6y ago

One thing I love about Rust is that allocation are very explicit and easily spotted which helps a lot when one wants to avoid them. I found C++ to be treacherous around corners cases on this subject.

pjmlp 6y ago
Unless you are going to review the whole code, written from scratch, there is no way to actually be aware of all allocations in Rust without help from a memory profiler.
- cogman10 6y ago
  I think the point was more that allocations are fairly explicit. There are a few places in C++ where allocation can happen pretty much invisibly. A copy constructor is an example of that. You might see a new allocation simply by calling a method. With rust, you usually won't see an allocation unless it is explicitly called for. You can follow the call tree and very easily pick out when those allocations are happening.
  - pjmlp 6y ago
    Rust also has operator overloading and implicit conversations, both can cause memory allocations.
    - cogman10 6y ago
      Certainly. However, both of those things (from what I've seen) are fairly rare. The C++ copy constructor comes up fairly frequently.
      - pjmlp 6y ago
        Like unsafe Rust code, there is the theory and then there are the code bases that one finds out in the wild on crates.io and in-house, not necessarily using best practices.

bananaface 6y ago

Can't you just allocate a huge block up-front and throw stuff into it with a custom allocator? I don't know if Rust allows you to do that kind of thing.

steveklabnik 6y ago
Yes. There are some trade offs, you can’t parameterize the standard library data structures over an allocator yet, for example.

AtlasBarfed 6y ago

So arena allocation and buffer reuse?