7 ms·

I hope that the Rust devs end up doing the "right thing", which from this article seems like going back and fixing Rc, instead of just marking mem::forget as sa

I hope that the Rust devs end up doing the "right thing", which from this article seems like going back and fixing Rc, instead of just marking mem::forget as safe. They're still pre-1.0, and anything they do now they will be stuck with for a long, long time, especially if Rust succeeds as much as I hope it does. Delaying 1.0 by a few weeks may seem like a big deal, but ultimately it is a self-imposed deadline. It's great to have those, but following them dogmatically might not be the best strategy. In cases like this, I generally lean towards slowing down and doing things right: otherwise you will pay the price ten times over later. (That said, while I understand this issue, I don't know very much about the context in the Rust community, so I'm not actually sure that mem::forget should be unsafe. It was just the impression from the article and from previous Rust code I've read/written.)

Jweb_Guru 11y ago
There isn't an acceptable way to fix Rc. It wouldn't be hard to require it not to have cycles, but cyclic data is actually a major usecase for Rc. A cycle collector wouldn't fix any of the unsafety here unless it was run on literally every drop, which would be far too slow to be acceptable in a systems language. An opt-in trait to prevent leaking specific types would be practical, but the implementation details aren't settled and it can be done backwards-compatibly, so there's a strong feeling that it shouldn't block the release. Any other attempts to determine whether cyclic data structures always eventually drop their contents enter very-active-research territory and would be more like a ten year than a few week thing.
- deleted 11y ago
  [deleted]
- pcwalton 11y ago
  Precisely. Rc is not easy to "fix": the fact that it allows cycles is part of its design.
  - mcguire 11y ago
    There's a reason why finalizers aren't a widely used technique in Java and why Python's cycle detector simply stacks things with destructors in an out-of-the-way location.
    - Jweb_Guru 11y ago
      Those languages have bigger problems than Rust here. Rust can still recover correct behavior for APIs like scoped because it can ensure (using lifetimes) that the relevant value never escapes the stack frame. In the languages you mention (and most other languages with a "with" or "IDisposable" equivalent), the "close" equivalent, or even the finalizer, can be called on objects that still have valid references to them hidden in another object or thread.
      - mcguire 11y ago
        I was actually speaking of the original finalize method it Java[1]. There's no guarantee that it will be called, there's no guarantee when it will be called if it is, and as a result it shouldn't be used to reclaim resources or much of anything, really. Where "should" is so strong in this case that I've never seen it in the wild, even given the N-monkeys code that I have seen. [1] http://www.informit.com/articles/article.aspx?p=1216151&seqNum=7 http://www.informit.com/articles/article.aspx?p=1216151&seqN...
  - haberman 11y ago
    I don't know enough about how Rc is used to know if this is an acceptable possibility, I just wanted to mention it in case it is possibly helpful. I developed a reference-counting system in my library "upb" that handles cycles and never leaks objects. It is precise for no-longer-mutable objects (objects which may no longer change the set of other refcounted objects pointed to) and less precise for mutable objects. The basic idea is to reference-count groups of objects instead of single objects, and define the groups such that no reference cycle spans groups. If you introduce a link between two refcounted objects A -> B, then A and B's groups are merged. Now refs/unrefs of A or B ref/unref the (merged) group. Nothing in the group is freed until both A and B's refcounts both fall to zero. This conservative group-merging whenever you create a link ensures that no reference cycle spans groups. This is imprecise and relatively wasteful if the group grows really large. But you can make it totally precise for any subgraph of refcounted objects that you're willing to freeze. At freeze time, compute strongly-connected components, and each SCC becomes a refcounted group. For any frozen subgraphs, the refcounting is totally precise. And unfrozen objects can reference frozen ones (but not the other way around). If your application ends up freezing most of the graph, this works great. Or if you're ok with the collection being pretty coarse for your groups of objects, it also works great. If you have an application that can't freeze the graph and wants pretty precise collection, this doesn't work so well. More info about my scheme is in comments in these headers: https://github.com/haberman/upb/blob/master/upb/refcounted.h https://github.com/haberman/upb/blob/master/upb/refcounted.h https://github.com/haberman/upb/blob/master/upb/refcounted.c https://github.com/haberman/upb/blob/master/upb/refcounted.c I always was curious if the semantics of this scheme would play nicely with the ownership system in Rust.
nosefrog 11y ago
You'd have to delay 1.0 by more than a few weeks -- making mem::forget unsafe means marking all memory leaks as unsafe, which is unsolvable.
Manishearth 11y ago
I'd prefer the `Leak` based solution too, but it's going to cause a _lot_ of churn. 1.0 has already been delayed often, every time it gets planned and then not followed through on because the date is fuzzy and ignorable. This time they've set a concrete no-nonsense date which we should follow through on IMO. We already had a second alpha.
- vegedor 11y ago
  >concrete, no-nonsense date As the OP said, the date is set voluntarily, before upcoming issues. How much time is left before release is immaterial, as that isn't a factor in the severity of the issue.
- krick 11y ago
  I guess it's better that way than saying something broken is "ok" only because "I promised myself that I'll finish until next Monday". It's more like Ubisoft-style, or something, except deadlines really matter for them, because money depend on it, and they cannot just "move the deadline" because of all the advertising must be timed, people tend to buy games and go to cinema more on specific dates, etc. For project like Rust deadline doesn't actually matter that much, grasping for it is almost stupid. The only thing that actually can suffer from breaking it is self-esteem, which should't worry a reasonable man much. But magic "1.0" number does matter a bit more than just deadline, because after it there's no "breaking changes". So I'd be much happier if Rust wouldn't reach 1.0 for the next 2 years, but would actually become satisfying instead. Somehow "non-stable but working" is better suited for making software than "broken and stable". We have plenty of "broken and stable" out there already.
  - kibwen 11y ago
    Rust is not seeking pure programming perfection, it is seeking to be useful and to fulfill its goals of safe systems programming, and it is fulfilling these goals with flying colors. There's absolutely no reason for this to push the release date. There was a bug in a stdlib API, and that issue was fixed weeks ago. Trying to pivot the language by taking a fundamental feature back to the drawing board for nebulous benefit would be utter foolishness at this point.
theseoafs 11y ago
It's very disappointing that the Rust community has opted to go back and change the definition of "safety" instead of actually tackling this problem head-on, all due to a self-imposed deadline there is no need to actually meet.
- steveklabnik 11y ago
  The definition of 'unsafe' has not changed.
- aturon 11y ago
  Rust's basic guarantee has been, and continues to be: unless code uses `unsafe`, it is guaranteeed to be memory safe. The issue under discussion doesn't change that -- it's largely about what such `unsafe` code is allowed to do and assume. I've laid this out in a bit more detail in a comment below (https://news.ycombinator.com/item?id=9447938 https://news.ycombinator.com/item?id=9447938)
aturon 11y ago
Please see my comment below (https://news.ycombinator.com/item?id=9447938 https://news.ycombinator.com/item?id=9447938). There is no risk to Rust's basic safety guarantees here, and there are several ways forward (including introducing things like `Leak` backwards-compatibly later on).

Jweb_Guru 11y ago

There isn't an acceptable way to fix Rc. It wouldn't be hard to require it not to have cycles, but cyclic data is actually a major usecase for Rc. A cycle collector wouldn't fix any of the unsafety here unless it was run on literally every drop, which would be far too slow to be acceptable in a systems language. An opt-in trait to prevent leaking specific types would be practical, but the implementation details aren't settled and it can be done backwards-compatibly, so there's a strong feeling that it shouldn't block the release. Any other attempts to determine whether cyclic data structures always eventually drop their contents enter very-active-research territory and would be more like a ten year than a few week thing.

deleted 11y ago
[deleted]
pcwalton 11y ago
Precisely. Rc is not easy to "fix": the fact that it allows cycles is part of its design.
- mcguire 11y ago
  There's a reason why finalizers aren't a widely used technique in Java and why Python's cycle detector simply stacks things with destructors in an out-of-the-way location.
  - Jweb_Guru 11y ago
    Those languages have bigger problems than Rust here. Rust can still recover correct behavior for APIs like scoped because it can ensure (using lifetimes) that the relevant value never escapes the stack frame. In the languages you mention (and most other languages with a "with" or "IDisposable" equivalent), the "close" equivalent, or even the finalizer, can be called on objects that still have valid references to them hidden in another object or thread.
    - mcguire 11y ago
      I was actually speaking of the original finalize method it Java[1]. There's no guarantee that it will be called, there's no guarantee when it will be called if it is, and as a result it shouldn't be used to reclaim resources or much of anything, really. Where "should" is so strong in this case that I've never seen it in the wild, even given the N-monkeys code that I have seen. [1] http://www.informit.com/articles/article.aspx?p=1216151&seqNum=7 http://www.informit.com/articles/article.aspx?p=1216151&seqN...
- haberman 11y ago
  I don't know enough about how Rc is used to know if this is an acceptable possibility, I just wanted to mention it in case it is possibly helpful. I developed a reference-counting system in my library "upb" that handles cycles and never leaks objects. It is precise for no-longer-mutable objects (objects which may no longer change the set of other refcounted objects pointed to) and less precise for mutable objects. The basic idea is to reference-count groups of objects instead of single objects, and define the groups such that no reference cycle spans groups. If you introduce a link between two refcounted objects A -> B, then A and B's groups are merged. Now refs/unrefs of A or B ref/unref the (merged) group. Nothing in the group is freed until both A and B's refcounts both fall to zero. This conservative group-merging whenever you create a link ensures that no reference cycle spans groups. This is imprecise and relatively wasteful if the group grows really large. But you can make it totally precise for any subgraph of refcounted objects that you're willing to freeze. At freeze time, compute strongly-connected components, and each SCC becomes a refcounted group. For any frozen subgraphs, the refcounting is totally precise. And unfrozen objects can reference frozen ones (but not the other way around). If your application ends up freezing most of the graph, this works great. Or if you're ok with the collection being pretty coarse for your groups of objects, it also works great. If you have an application that can't freeze the graph and wants pretty precise collection, this doesn't work so well. More info about my scheme is in comments in these headers: https://github.com/haberman/upb/blob/master/upb/refcounted.h https://github.com/haberman/upb/blob/master/upb/refcounted.h https://github.com/haberman/upb/blob/master/upb/refcounted.c https://github.com/haberman/upb/blob/master/upb/refcounted.c I always was curious if the semantics of this scheme would play nicely with the ownership system in Rust.

nosefrog 11y ago

You'd have to delay 1.0 by more than a few weeks -- making mem::forget unsafe means marking all memory leaks as unsafe, which is unsolvable.

Manishearth 11y ago

I'd prefer the `Leak` based solution too, but it's going to cause a _lot_ of churn. 1.0 has already been delayed often, every time it gets planned and then not followed through on because the date is fuzzy and ignorable. This time they've set a concrete no-nonsense date which we should follow through on IMO. We already had a second alpha.

vegedor 11y ago
>concrete, no-nonsense date As the OP said, the date is set voluntarily, before upcoming issues. How much time is left before release is immaterial, as that isn't a factor in the severity of the issue.
krick 11y ago
I guess it's better that way than saying something broken is "ok" only because "I promised myself that I'll finish until next Monday". It's more like Ubisoft-style, or something, except deadlines really matter for them, because money depend on it, and they cannot just "move the deadline" because of all the advertising must be timed, people tend to buy games and go to cinema more on specific dates, etc. For project like Rust deadline doesn't actually matter that much, grasping for it is almost stupid. The only thing that actually can suffer from breaking it is self-esteem, which should't worry a reasonable man much. But magic "1.0" number does matter a bit more than just deadline, because after it there's no "breaking changes". So I'd be much happier if Rust wouldn't reach 1.0 for the next 2 years, but would actually become satisfying instead. Somehow "non-stable but working" is better suited for making software than "broken and stable". We have plenty of "broken and stable" out there already.
- kibwen 11y ago
  Rust is not seeking pure programming perfection, it is seeking to be useful and to fulfill its goals of safe systems programming, and it is fulfilling these goals with flying colors. There's absolutely no reason for this to push the release date. There was a bug in a stdlib API, and that issue was fixed weeks ago. Trying to pivot the language by taking a fundamental feature back to the drawing board for nebulous benefit would be utter foolishness at this point.

theseoafs 11y ago

It's very disappointing that the Rust community has opted to go back and change the definition of "safety" instead of actually tackling this problem head-on, all due to a self-imposed deadline there is no need to actually meet.

steveklabnik 11y ago
The definition of 'unsafe' has not changed.
aturon 11y ago
Rust's basic guarantee has been, and continues to be: unless code uses `unsafe`, it is guaranteeed to be memory safe. The issue under discussion doesn't change that -- it's largely about what such `unsafe` code is allowed to do and assume. I've laid this out in a bit more detail in a comment below (https://news.ycombinator.com/item?id=9447938 https://news.ycombinator.com/item?id=9447938)

aturon 11y ago

Please see my comment below (https://news.ycombinator.com/item?id=9447938 https://news.ycombinator.com/item?id=9447938). There is no risk to Rust's basic safety guarantees here, and there are several ways forward (including introducing things like `Leak` backwards-compatibly later on).