5 ms·

Over allocating maps and arrays in standard libs is not really a memory "leak" . Other langs do it.

by teknopaul 3y ago

gipp 3y ago
Yeah, came here to say this. Still an interesting post, but the title implies a much more serious problem
dylan604 3y ago
[flagged]
- GrumpySloth 3y ago
  There is nothing unsafe about allocating more memory than necessary.
- hyperpape 3y ago
  Please go read Wikipedia: https://en.wikipedia.org/wiki/Memory_safety https://en.wikipedia.org/wiki/Memory_safety.
- phanimahesh 3y ago
  Memory safety has not been violated here.
- unshavedyak 3y ago
  Being so quick to jump this sarcastic gun on something so clearly unrelated to memory safety (even if it was a leak, which it isn't!) is not a great look.
hyperpape 3y ago
Agreed. In the Haskell world, "space leak" is used to refer to a lazy computation that overallocates because of laziness (classic example being the difference between foldl and foldr or maybe that's foldl' and foldr'--I forget some details/cannot be bothered to think about folds right now). I've personally stolen the terminology for a computation that temporarily holds on to a bunch of unneeded memory.
beltsazar 3y ago
After skimming the first few paragraphs, I almost commented something similar to yours. But after reading it until the end, the problem is more nuanced than that. The issue is more about reusing memory that has been allocated when you "convert" (in Rust term: into_iter()) a Vec to another Vec. More surprisingly, the reuse still happens even when the new Vec has a different type than the original Vec's type. You should read the whole post—it's more interesting than I (and you) thought.
wredue 3y ago
You’re right, but it’s (I agree; incorrectly) been called a memory leak for a long time, to inappropriately over allocate/not free resources. Coming from my java days, they called this a memory leak there too. It’s definitely a more inappropriate term in rust, where memory leak has a definite meaning already.
hulium 3y ago
Conceptually, collect seems to create a new vector. In that case you would expect that the memory of the old vector is freed. The library tries to be smart and reuses the old vector as an (undocumented?) optimization. So the old memory that is no longer needed is not released. Whether you call that a leak or not, Rust is known for its predictability. Allocating 200x the memory which a programmer would expect if he doesn't know the implementation details is bad.
wavemode 3y ago
The semantics are kind of besides the point, I think. This allocation reuse behavior was meant as an optimization, but the real-world programs this is meant to be "optimizing" run significantly slower in beta than they did in stable. So there is a bug here no matter how you look at it.
deleted 3y ago
[deleted]

Over allocating maps and arrays in standard libs is not really a memory "leak" . Other langs do it.

gipp 3y ago

Yeah, came here to say this. Still an interesting post, but the title implies a much more serious problem

dylan604 3y ago

[flagged]

GrumpySloth 3y ago
There is nothing unsafe about allocating more memory than necessary.
hyperpape 3y ago
Please go read Wikipedia: https://en.wikipedia.org/wiki/Memory_safety https://en.wikipedia.org/wiki/Memory_safety.
phanimahesh 3y ago
Memory safety has not been violated here.
unshavedyak 3y ago
Being so quick to jump this sarcastic gun on something so clearly unrelated to memory safety (even if it was a leak, which it isn't!) is not a great look.

hyperpape 3y ago

Agreed. In the Haskell world, "space leak" is used to refer to a lazy computation that overallocates because of laziness (classic example being the difference between foldl and foldr or maybe that's foldl' and foldr'--I forget some details/cannot be bothered to think about folds right now). I've personally stolen the terminology for a computation that temporarily holds on to a bunch of unneeded memory.

beltsazar 3y ago

After skimming the first few paragraphs, I almost commented something similar to yours. But after reading it until the end, the problem is more nuanced than that. The issue is more about reusing memory that has been allocated when you "convert" (in Rust term: into_iter()) a Vec to another Vec. More surprisingly, the reuse still happens even when the new Vec has a different type than the original Vec's type. You should read the whole post—it's more interesting than I (and you) thought.

wredue 3y ago

You’re right, but it’s (I agree; incorrectly) been called a memory leak for a long time, to inappropriately over allocate/not free resources. Coming from my java days, they called this a memory leak there too. It’s definitely a more inappropriate term in rust, where memory leak has a definite meaning already.

hulium 3y ago

Conceptually, collect seems to create a new vector. In that case you would expect that the memory of the old vector is freed. The library tries to be smart and reuses the old vector as an (undocumented?) optimization. So the old memory that is no longer needed is not released. Whether you call that a leak or not, Rust is known for its predictability. Allocating 200x the memory which a programmer would expect if he doesn't know the implementation details is bad.

wavemode 3y ago

The semantics are kind of besides the point, I think. This allocation reuse behavior was meant as an optimization, but the real-world programs this is meant to be "optimizing" run significantly slower in beta than they did in stable. So there is a bug here no matter how you look at it.

deleted 3y ago

[deleted]