6 ms·

For backend web development I was somewhat disappointed to see that many of the new web frameworks (all async) allocate extensively on the heap (for example lot

For backend web development I was somewhat disappointed to see that many of the new web frameworks (all async) allocate extensively on the heap (for example lots of Strings in their http request types.) I mean it works but I don't understand why I would go to the bother of thinking about lifetimes when it seems performance would be similar to Kotlin / Swift / F#.

staticassertion 6y ago
I often write my code to allocate at first just to get a feel for the API and business logic and then go through it later to remove them. Given that all of those frameworks are still fairly new it wouldn't shock me to hear that that's the case.
drewm1980 6y ago
Rust never has to run a garbage collector, because it already figured out the lifetimes of all the values in your program statically, with relatively minor help from the programmer. This may make your code faster or more likely to be correct than code written in some of those languages.
- laurencerowe 6y ago
  That's a fair point, I should distinguish heap allocation from reference counting. It still seems somewhat tricky to replicate the well proven arena allocation approach of Apache and Nginx which avoids memory fragmentation in current stable Rust though.
  - kd5bjo 6y ago
    For general areana allocation, you can use something like the Bumpalo crate: https://docs.rs/bumpalo/3.4.0/bumpalo/ https://docs.rs/bumpalo/3.4.0/bumpalo/
- blub 6y ago
  It didn't figure them out at all, the programmer had to figure them out the hard way and then code them in so that the compiler understands them. GC is figuring out things mostly correctly and the programmer can focus on other things, at least until they hit a performance problem :)
  - Lorkki 6y ago
    For the most part, lifetime is inferred from scope, so there isn't anything special to figure out; objects live exactly for as long as you can see and use them. This also includes such things as lock handles, or references to refcounted objects.
    - jashmatthews 6y ago
      Only in the most trivial cases. Tons of real Rust uses copying or RC/ARC or even unsafe to avoid dealing with complex lifetimes because it's... complex.
      - Lorkki 6y ago
        Copying, refcounting and locking work within the exact same semantics. They're among the most typical tools that are used to transmit or share data, so I'm not sure why it should be strange to see them used in most real programs?
- Karrot_Kream 6y ago
  That doesn't change the reality of allocation though. Programmers in languages like C++ and Rust choose when and where to allocate memory, and if there's one thing we know about performant code, it's that we shouldn't be allocating and deallocating repeatedly. Nothing about lifetimes helps or hurts about this programming model. Lifetimes (Rust) and RAII (C++) are just semantic models to help reign in the cognitive load of memory allocation.
- adrianN 6y ago
  I doubt that you'll write a large Rust program without relying on refcounting at least for some of your objects. Refcounting is not fundamentally different than a GC.
  - zozbot234 6y ago
    Reference counting is a kind of garbage collection (taken broadly), but it's quite different from tracing GC.
    - jashmatthews 6y ago
      Naive use of RC doesn't look like tracing much but as soon as you need cycle-aware RC like the CactusRef the algo looks a lot like tracing GC https://github.com/artichoke/cactusref#cycle-detection https://github.com/artichoke/cactusref#cycle-detection
- rcxdude 6y ago
  I don't think you've understood the nature of the borrow checker in Rust. It doesn't add any extra runtime behaviour, in fact you could compile Rust code without running these checks (though I don't think rustc provides you with a way to do so). Rust's memory management tools are no different from C++: you have stack allocated structures with destructors, which the standard library uses to implement heap-allocated smart pointers, shared pointers, mutex guards, etc, and you can hand out pointers to any of these. The only difference in rust is it checks how these pointers get handed out so you don't get use-after-free, data races, etc.
codemac 6y ago
As a complete noob to rust, I found attempting to manage a memory pool that is handed out per request to cause all kinds of borrow checking headaches. My bet is people choose to allocate heavily rather than figure out how to share memory correctly with Rust. I need to learn more rust to get a hang of it so I don't allocate so much. Are there any rust folks who have a great "list of memory management models for rust"?
- zozbot234 6y ago
  Rust does not yet have local custom allocators ala C++. (You can change the global allocator, but this doesn't help the "memory arena" use case.) The feature is very much on the roadmap, though.
  - steveklabnik 6y ago
    While this is true, this doesn't inherently mean you can't do this pattern. https://crates.io/crates/typed_arena https://crates.io/crates/typed_arena for example. It depends on exactly how allocations and your data structure are tied together.
    - laurencerowe 6y ago
      It would be amazingly handy if someday it was possible to direct Rust to make all heap allocations under a certain block within an arena. At the moment it looks like you have to rewrite any third party code that makes heap allocations to use the arena explicitly at each allocation.
      - steveklabnik 6y ago
        Yep, it would. There’s certainly a bunch of work to do here still.
cies 6y ago
I'd be hard pressed if a backend in Rust gives the same perf as Kotlin/F#. Especially if you write the Rust with an async stack you will likely come out significantly faster than the other options: small binaries, short startup time, less memory usage, more throughput, slightly shorter request cycles, ... The only place that the langs you mention probably win is: learning curve (F# maybe not so much) a.k.a. getting language illiterate people up to speed, and compile times.
- laurencerowe 6y ago
  Whether to use an async stack or not is very application dependent. For something closer to an application server than a proxy server I’m unconvinced async would be superior to threads. I feel that Rust should be competing with C/C++ here though. I want a modern language with the same speed and control of memory usage. What’s A little frustrating (and don’t get me wrong, I’m rooting for Rust to get there) is that it feels like it should be possible to offer this, at least for threaded web servers.
  - Karrot_Kream 6y ago
    There are other interesting languages in this space that I really enjoy playing with. Both Nim and Zig generate native code and are really fun, albeit very different, languages to work with.
- Karrot_Kream 6y ago
  I'm not sure. These reasons sound reasonable, but in webapps most of the time is spent IO handling anyway. As long as you're using async implementations in any language with performant "threads", I'm not sure of there would be a large difference in response times unless computation was involved in the request path.
- entha_saava 6y ago
  You forgot development times.
  - cies 6y ago
deleted 6y ago
[deleted]
pas 6y ago
The short answer is that because async is not done yet in Rust. The long answer probably includes GATs (generic associated types), and the even longer answer starts with the amazing work that Nico et al. does to reinvent the internals of the Rust type checker. (Basically - if I remember correctly - the compiler team is currently refactoring big parts of rustc, to librarify the type checker, and replace it with a PROLOG-ish library called "chalk", which is able to prove more things, so it allows better handling of GATs, so it allows better handling of async, where you get "impl Future<Output = T>"-s everywhere. But also somewhere there is that the ergonomics of async/.await are still not very much in progress too. With better error messages people will be able to forego Box<>-ing and figure out what to use instead, and only heap allocate where they must. It's possible to write low-level async code with hand rolled-polled Futures, and push/manage as much stuff on the stack as possible. But ... that takes time, and performance is already "good enough". (And/or probably there are bigger gains in performance in other areas, such as scheduling.) And, finally, boxing helps with compile times. (Because it's basically trivial for the compile to prove that a heap allocated simple trait object behaves well compared to a stack allocated concrete, but usually highly complex (plus unnameable) type.)
- dljsjr 6y ago
  Can I ask what you mean when you say that async isn't done? async/await and Futures are on stable now. There is no runtime in the core/std but that's by design, there is no short term intent from the Rust team to do that. Tokio and async-std are both fairly usable as well.
  - steveklabnik 6y ago
    The team itself described the stabilization of async/await as being in an "MVP" state. Stuff that's been fixed since then: * error messages (I think there may be more to do, but they're better and better all the time) * size of tasks (Still more to do, but they've shrunk a lot since it was first implemented * async/await on no_std (initial implementation used TLS to manage some state, that's since been fixed) Stuff still to do: * Async functions in traits (this has a library that lets you work around this with one allocation, having this requires GATs) * More improvements to the stuff above * other stuff I'm sure I'm forgetting > There is no runtime in the core/std but that's by design, there is no short term intent from the Rust team to do that. This is true but with an asterick; https://github.com/rust-lang/rust/pull/65875 https://github.com/rust-lang/rust/pull/65875 ended up being closed with "please write an RFC," rather than "no."

staticassertion 6y ago

I often write my code to allocate at first just to get a feel for the API and business logic and then go through it later to remove them. Given that all of those frameworks are still fairly new it wouldn't shock me to hear that that's the case.

drewm1980 6y ago

Rust never has to run a garbage collector, because it already figured out the lifetimes of all the values in your program statically, with relatively minor help from the programmer. This may make your code faster or more likely to be correct than code written in some of those languages.

laurencerowe 6y ago
That's a fair point, I should distinguish heap allocation from reference counting. It still seems somewhat tricky to replicate the well proven arena allocation approach of Apache and Nginx which avoids memory fragmentation in current stable Rust though.
- kd5bjo 6y ago
  For general areana allocation, you can use something like the Bumpalo crate: https://docs.rs/bumpalo/3.4.0/bumpalo/ https://docs.rs/bumpalo/3.4.0/bumpalo/
blub 6y ago
It didn't figure them out at all, the programmer had to figure them out the hard way and then code them in so that the compiler understands them. GC is figuring out things mostly correctly and the programmer can focus on other things, at least until they hit a performance problem :)
- Lorkki 6y ago
  For the most part, lifetime is inferred from scope, so there isn't anything special to figure out; objects live exactly for as long as you can see and use them. This also includes such things as lock handles, or references to refcounted objects.
  - jashmatthews 6y ago
    Only in the most trivial cases. Tons of real Rust uses copying or RC/ARC or even unsafe to avoid dealing with complex lifetimes because it's... complex.
    - Lorkki 6y ago
      Copying, refcounting and locking work within the exact same semantics. They're among the most typical tools that are used to transmit or share data, so I'm not sure why it should be strange to see them used in most real programs?
Karrot_Kream 6y ago
That doesn't change the reality of allocation though. Programmers in languages like C++ and Rust choose when and where to allocate memory, and if there's one thing we know about performant code, it's that we shouldn't be allocating and deallocating repeatedly. Nothing about lifetimes helps or hurts about this programming model. Lifetimes (Rust) and RAII (C++) are just semantic models to help reign in the cognitive load of memory allocation.
adrianN 6y ago
I doubt that you'll write a large Rust program without relying on refcounting at least for some of your objects. Refcounting is not fundamentally different than a GC.
- zozbot234 6y ago
  Reference counting is a kind of garbage collection (taken broadly), but it's quite different from tracing GC.
  - jashmatthews 6y ago
    Naive use of RC doesn't look like tracing much but as soon as you need cycle-aware RC like the CactusRef the algo looks a lot like tracing GC https://github.com/artichoke/cactusref#cycle-detection https://github.com/artichoke/cactusref#cycle-detection
rcxdude 6y ago
I don't think you've understood the nature of the borrow checker in Rust. It doesn't add any extra runtime behaviour, in fact you could compile Rust code without running these checks (though I don't think rustc provides you with a way to do so). Rust's memory management tools are no different from C++: you have stack allocated structures with destructors, which the standard library uses to implement heap-allocated smart pointers, shared pointers, mutex guards, etc, and you can hand out pointers to any of these. The only difference in rust is it checks how these pointers get handed out so you don't get use-after-free, data races, etc.

codemac 6y ago

As a complete noob to rust, I found attempting to manage a memory pool that is handed out per request to cause all kinds of borrow checking headaches. My bet is people choose to allocate heavily rather than figure out how to share memory correctly with Rust. I need to learn more rust to get a hang of it so I don't allocate so much. Are there any rust folks who have a great "list of memory management models for rust"?

zozbot234 6y ago
Rust does not yet have local custom allocators ala C++. (You can change the global allocator, but this doesn't help the "memory arena" use case.) The feature is very much on the roadmap, though.
- steveklabnik 6y ago
  While this is true, this doesn't inherently mean you can't do this pattern. https://crates.io/crates/typed_arena https://crates.io/crates/typed_arena for example. It depends on exactly how allocations and your data structure are tied together.
  - laurencerowe 6y ago
    It would be amazingly handy if someday it was possible to direct Rust to make all heap allocations under a certain block within an arena. At the moment it looks like you have to rewrite any third party code that makes heap allocations to use the arena explicitly at each allocation.
    - steveklabnik 6y ago
      Yep, it would. There’s certainly a bunch of work to do here still.

cies 6y ago

I'd be hard pressed if a backend in Rust gives the same perf as Kotlin/F#. Especially if you write the Rust with an async stack you will likely come out significantly faster than the other options: small binaries, short startup time, less memory usage, more throughput, slightly shorter request cycles, ... The only place that the langs you mention probably win is: learning curve (F# maybe not so much) a.k.a. getting language illiterate people up to speed, and compile times.

laurencerowe 6y ago
Whether to use an async stack or not is very application dependent. For something closer to an application server than a proxy server I’m unconvinced async would be superior to threads. I feel that Rust should be competing with C/C++ here though. I want a modern language with the same speed and control of memory usage. What’s A little frustrating (and don’t get me wrong, I’m rooting for Rust to get there) is that it feels like it should be possible to offer this, at least for threaded web servers.
- Karrot_Kream 6y ago
  There are other interesting languages in this space that I really enjoy playing with. Both Nim and Zig generate native code and are really fun, albeit very different, languages to work with.
Karrot_Kream 6y ago
I'm not sure. These reasons sound reasonable, but in webapps most of the time is spent IO handling anyway. As long as you're using async implementations in any language with performant "threads", I'm not sure of there would be a large difference in response times unless computation was involved in the request path.
entha_saava 6y ago
You forgot development times.
- cies 6y ago

deleted 6y ago

[deleted]

pas 6y ago

The short answer is that because async is not done yet in Rust. The long answer probably includes GATs (generic associated types), and the even longer answer starts with the amazing work that Nico et al. does to reinvent the internals of the Rust type checker. (Basically - if I remember correctly - the compiler team is currently refactoring big parts of rustc, to librarify the type checker, and replace it with a PROLOG-ish library called "chalk", which is able to prove more things, so it allows better handling of GATs, so it allows better handling of async, where you get "impl Future<Output = T>"-s everywhere. But also somewhere there is that the ergonomics of async/.await are still not very much in progress too. With better error messages people will be able to forego Box<>-ing and figure out what to use instead, and only heap allocate where they must. It's possible to write low-level async code with hand rolled-polled Futures, and push/manage as much stuff on the stack as possible. But ... that takes time, and performance is already "good enough". (And/or probably there are bigger gains in performance in other areas, such as scheduling.) And, finally, boxing helps with compile times. (Because it's basically trivial for the compile to prove that a heap allocated simple trait object behaves well compared to a stack allocated concrete, but usually highly complex (plus unnameable) type.)

dljsjr 6y ago
Can I ask what you mean when you say that async isn't done? async/await and Futures are on stable now. There is no runtime in the core/std but that's by design, there is no short term intent from the Rust team to do that. Tokio and async-std are both fairly usable as well.
- steveklabnik 6y ago
  The team itself described the stabilization of async/await as being in an "MVP" state. Stuff that's been fixed since then: * error messages (I think there may be more to do, but they're better and better all the time) * size of tasks (Still more to do, but they've shrunk a lot since it was first implemented * async/await on no_std (initial implementation used TLS to manage some state, that's since been fixed) Stuff still to do: * Async functions in traits (this has a library that lets you work around this with one allocation, having this requires GATs) * More improvements to the stuff above * other stuff I'm sure I'm forgetting > There is no runtime in the core/std but that's by design, there is no short term intent from the Rust team to do that. This is true but with an asterick; https://github.com/rust-lang/rust/pull/65875 https://github.com/rust-lang/rust/pull/65875 ended up being closed with "please write an RFC," rather than "no."