3 ms·

This is not about a heap-vs-stack tradeoff, this is about the compiler routinely generating very inefficient code that copies data around on stack for no good r

This is not about a heap-vs-stack tradeoff, this is about the compiler routinely generating very inefficient code that copies data around on stack for no good reason. For an example I've seen myself: using a custom GC pointer library, calling `Gc::allocate(SomeBigStruct{...})` constructed the SomeBigStruct on stack and copied it around using memcpy 4 times before it actually landed in the allocated heap memory. The equivalent code compiled with a C++ compiler would have probably optimized the program enough to fill the struct in-place on heap without any issues. (this example is from over a year ago; it's not as bad anymore, but it still generates suboptimal assembly with too much copying)

CapmCrackaWaka 4y ago
If you don't mind me asking - how do you witness these low level memory allocations? Specific program, plugin to vs code?
- spacechild1 4y ago
  You look at the generated assembly.
- adrian17 4y ago
  The simplest way to make quick experiments for me is with https://godbolt.org/ https://godbolt.org/ . For my particular example: https://godbolt.org/z/8GvYzYj5h https://godbolt.org/z/8GvYzYj5h You can see we're trying to put a 1kB object on heap, but the compiler generates two `memcpy` calls - first to copy it on stack to build the wrapper struct, second to actually copy the entire struct onto allocated memory. In "real programs", you just need to look at the program's assembly. Or even more generally, I originally noticed this when observing the unusually high amount of time spent in some functions when profiling.
- hmfrh 4y ago
  Other than Godbolt Rust also has cargo-show-asm[1] that directly shows the actual assembly. [1]: https://crates.io/crates/cargo-show-asm https://crates.io/crates/cargo-show-asm
pkolaczk 4y ago
The optimizing backend for Rust and C++ is common. So if it didn't get optimized in Rust, it is very likely it wouldn't be in C++ as well. However, the code style of those two codebases might be different. IMHO C++ code is traditionally a lot more pointer and heap allocation heavy than Rust code. Rust makes moving stuff very convenient and using pointers/references quite inconvenient. Therefore showing heap allocation profiles would help us understand if those differences are due to actual compiler inefficiencies or different memory management patterns used in the source code. Also rustc code has been written in different times than the majority of clang code. That might as well affect the copying patterns. Move semantics is actually a quite modern thing in C++ (and not default like in Rust).
- edflsafoiewq 4y ago
  C++'s constructors are basically custom built for this situation. It's really the only thing they're good for.
- bananapub 4y ago
  > The optimizing backend for Rust and C++ is common. So if it didn't get optimized in Rust, it is very likely it wouldn't be in C++ as well. However, the code style of those two codebases eh? Rust doesn't have placement-new or specify copy elision. nothing at all to do with backends or "code style".
  - pkolaczk 4y ago
    Sure it doesn't do placement new yet, but I think you're exaggerating the effect on lack of copy elision. Rust doesnt really need copy elision so heavily because it defaults to moving stuff, and move is just a very shallow copy taking typically one or two cycles. I've never seen it become visible in a profile. In C++, copies are much more heavy, because they need to preserve the original, so copy elision matters a lot more. Also a developer is free to put arbitrarily complex stuff into a copy constructor. In Rust, those heavy copies are explicit so the developer can fully control when they happen. Nevertheless - it could be all those reasons together. It's good someone is looking into it.
- 4y ago
zamadatix 4y ago
Most likely these kinds of things are the cause but this data does not actually show that assumption because it lacks the other information. It’s also possible rust has more copies on each which would be useful to track anyways.

CapmCrackaWaka 4y ago

If you don't mind me asking - how do you witness these low level memory allocations? Specific program, plugin to vs code?

spacechild1 4y ago
You look at the generated assembly.
adrian17 4y ago
The simplest way to make quick experiments for me is with https://godbolt.org/ https://godbolt.org/ . For my particular example: https://godbolt.org/z/8GvYzYj5h https://godbolt.org/z/8GvYzYj5h You can see we're trying to put a 1kB object on heap, but the compiler generates two `memcpy` calls - first to copy it on stack to build the wrapper struct, second to actually copy the entire struct onto allocated memory. In "real programs", you just need to look at the program's assembly. Or even more generally, I originally noticed this when observing the unusually high amount of time spent in some functions when profiling.
hmfrh 4y ago
Other than Godbolt Rust also has cargo-show-asm[1] that directly shows the actual assembly. [1]: https://crates.io/crates/cargo-show-asm https://crates.io/crates/cargo-show-asm

pkolaczk 4y ago

The optimizing backend for Rust and C++ is common. So if it didn't get optimized in Rust, it is very likely it wouldn't be in C++ as well. However, the code style of those two codebases might be different. IMHO C++ code is traditionally a lot more pointer and heap allocation heavy than Rust code. Rust makes moving stuff very convenient and using pointers/references quite inconvenient. Therefore showing heap allocation profiles would help us understand if those differences are due to actual compiler inefficiencies or different memory management patterns used in the source code. Also rustc code has been written in different times than the majority of clang code. That might as well affect the copying patterns. Move semantics is actually a quite modern thing in C++ (and not default like in Rust).

edflsafoiewq 4y ago
C++'s constructors are basically custom built for this situation. It's really the only thing they're good for.
bananapub 4y ago
> The optimizing backend for Rust and C++ is common. So if it didn't get optimized in Rust, it is very likely it wouldn't be in C++ as well. However, the code style of those two codebases eh? Rust doesn't have placement-new or specify copy elision. nothing at all to do with backends or "code style".
- pkolaczk 4y ago
  Sure it doesn't do placement new yet, but I think you're exaggerating the effect on lack of copy elision. Rust doesnt really need copy elision so heavily because it defaults to moving stuff, and move is just a very shallow copy taking typically one or two cycles. I've never seen it become visible in a profile. In C++, copies are much more heavy, because they need to preserve the original, so copy elision matters a lot more. Also a developer is free to put arbitrarily complex stuff into a copy constructor. In Rust, those heavy copies are explicit so the developer can fully control when they happen. Nevertheless - it could be all those reasons together. It's good someone is looking into it.
4y ago

zamadatix 4y ago

Most likely these kinds of things are the cause but this data does not actually show that assumption because it lacks the other information. It’s also possible rust has more copies on each which would be useful to track anyways.