3 ms·

Author here. In scientific computing, there isn't much of a security implication of UB, and the most dangerous kind of error are when your program silently comp

Author here. In scientific computing, there isn't much of a security implication of UB, and the most dangerous kind of error are when your program silently computes the wrong thing. And that is especially likely when you use indices manually. You could say that UB enables all behaviour, including silently wrong answers, and you'd be right. But it's more likely to crash your program and therefore be caught. Most importantly, the comparison to a raw pointer is not relevant. My blog post states that integers come with zero safety (as in: preventing bugs, not risk of UB) and zero language support and that is true. My blog post compares Rust with GC languages, not with raw pointer arithmetic. And it's clear that, when you compare to using GC references, manual indices are horribly unsafe.

zozbot234 1y ago
For the record, I completely agree that using GC is appropriate for problems that deal with possibly-cyclic general graphs, and maybe even wrt. DAG structures whenever one wishes to maximize throughput and the use of simpler refcounting would involve heavy overhead. (This is a well-known pitfall wrt. languages like Swift, that enforce pervasive use of atomic reference counting.) Since the use of "pluggable" GC in C-like or Rust-like languages is still uncommon, this generally means resorting to a language which happens to inherently rely on GC, such as Golang.
burntsushi 1y ago
You're engaging in a motte-and-bailey fallacy. Your motte is your much more narrow claim about Rust's advantages in a context where you claim not to care about undefined behavior as a class of bugs more severe than logic errors. And you specifically make this claim in an additional context where a GC language is appropriate. That's an overall pretty easy position to defend, and if that were clearly the extent of it, I probably wouldn't have responded at all. But, that context has been dropped in this thread, and instead folks are making very general claims outside of that very restricted context. Moreover, your bailey is that you don't do a good job outlining that context either. Your blog's opening paragraphs mention nothing about that more constrained context and instead seem to imply a very general context. This is much harder to defend. You go on to mention scientific computing, but instead of it being a centerpiece of the context of your claim, it's just mentioned as aside. Instead, your blog appears to be making very broad claims. But you've jumped in here to narrow them significantly, to the point that it materially changes your point IMO. Let's just look at what you said here: > The first time someone gave be this advice, I had to do a double take. The Rust community's whole thing is commitment to compiler-enforced correctness, and they built the borrowchecker on the premise that humans can't be trusted to handle references manually. When the same borrowchecker makes references unworkable, their solution is to... recommend that I manually manage them, with zero safety and zero language support?!? The irony is unreal. Asking people to manually manage references is so hilariously unsafe and unergonomic, the suggestion would be funny if it wasn't mostly sad. There's no circumspection about the context. You're just generally and broadly dismissing this entirely as if it weren't a valid thing ever. But it absolutely is a valid technique and it has real practical differences with an approach that uses raw pointers. If the comparison is with a GC and that context is made clear, then yes, absolutely, the comparison point changes entirely! If you can abide a GC, then a whole bunch of things get easier... at some cost. For example, I don't think it's possible to write a tool like ripgrep with its performance profile in a GC language. At least, I've never seen it done.
- estebank 1y ago
  > I don't think it's possible to write a tool like ripgrep with its performance profile in a GC language. I think it is possible to make a language that has both a GC and a borrow checker, treating the GC types as a third level next to the stack and the heap, where complex referencial cycles can bé promoted to the GC, but the defaults push you towards fast execution patterns. Don't know if such a language would be successful in finding its niche. The only way I could see that, is of a non-gc mode could be enforced so that libraries can be written in the more restrictive, faster by default mode, while being consumed by application developers that have less stringent restrictions. This is no different in concept than Python libraries implemented in native languages. Making it the mode be part of the same language could help with prototyping pains: write with the GC and then refactor once at the end after the general design is mostly found.
  - burntsushi 1y ago
    My standard of evidence here is existence. There are many greps written in a "GC language," and none of them have a broad performance profile in the same category as ripgrep (or other greps written in C and C++). > This is no different in concept than Python libraries implemented in native languages. If that's true, then why isn't there a fast grep written in Python? IMO, it's for the same reason that tools like csvkit are slow. Once you get to the boundary between GC and non-GC, some kind of cost gets paid. Now maybe Python is paying higher costs here than your hypothetical language, but who knows. Without actual existence, I find this sort of hypothetical to be very uncompelling. It's too broad of a stroke.
  - zozbot234 1y ago
    > I think it is possible to make a language that has both a GC and a borrow checker, treating the GC types as a third level next to the stack and the heap, where complex referencial cycles can be 'promoted to the GC' It's doable but there are a few issues with that whole idea. You need the ability to safely promote objects to GC-roots whenever they're being referenced by GC-unaware code, and demote them again afterwards. And if any GC object happens to control the lifecycle of any heap-allocated objects, it must have a finalizer that cleans them up RAII style to avoid resource leaks.
estebank 1y ago
> And that is especially likely when you use indices manually. But that is never the recommendation Rust practitioners would give for graph data structures. One would instead recommend using a generational arena, where each index holds their "generation". The arena can be growable or not. When an element is removed from the arena it gets tombstoned, marked as no longer valid. If a new value reuses a tombstoned position, its generation changes. This shifts the cost of verifying the handle is correct at the read point: if the handle corresponds to an index with a tombstone sentinel or has a different generation, the result of the read operation is None, meaning the handle is no longer valid. This is much better than the behavior of pointers.

zozbot234 1y ago

For the record, I completely agree that using GC is appropriate for problems that deal with possibly-cyclic general graphs, and maybe even wrt. DAG structures whenever one wishes to maximize throughput and the use of simpler refcounting would involve heavy overhead. (This is a well-known pitfall wrt. languages like Swift, that enforce pervasive use of atomic reference counting.) Since the use of "pluggable" GC in C-like or Rust-like languages is still uncommon, this generally means resorting to a language which happens to inherently rely on GC, such as Golang.

burntsushi 1y ago

You're engaging in a motte-and-bailey fallacy. Your motte is your much more narrow claim about Rust's advantages in a context where you claim not to care about undefined behavior as a class of bugs more severe than logic errors. And you specifically make this claim in an additional context where a GC language is appropriate. That's an overall pretty easy position to defend, and if that were clearly the extent of it, I probably wouldn't have responded at all. But, that context has been dropped in this thread, and instead folks are making very general claims outside of that very restricted context. Moreover, your bailey is that you don't do a good job outlining that context either. Your blog's opening paragraphs mention nothing about that more constrained context and instead seem to imply a very general context. This is much harder to defend. You go on to mention scientific computing, but instead of it being a centerpiece of the context of your claim, it's just mentioned as aside. Instead, your blog appears to be making very broad claims. But you've jumped in here to narrow them significantly, to the point that it materially changes your point IMO. Let's just look at what you said here: > The first time someone gave be this advice, I had to do a double take. The Rust community's whole thing is commitment to compiler-enforced correctness, and they built the borrowchecker on the premise that humans can't be trusted to handle references manually. When the same borrowchecker makes references unworkable, their solution is to... recommend that I manually manage them, with zero safety and zero language support?!? The irony is unreal. Asking people to manually manage references is so hilariously unsafe and unergonomic, the suggestion would be funny if it wasn't mostly sad. There's no circumspection about the context. You're just generally and broadly dismissing this entirely as if it weren't a valid thing ever. But it absolutely is a valid technique and it has real practical differences with an approach that uses raw pointers. If the comparison is with a GC and that context is made clear, then yes, absolutely, the comparison point changes entirely! If you can abide a GC, then a whole bunch of things get easier... at some cost. For example, I don't think it's possible to write a tool like ripgrep with its performance profile in a GC language. At least, I've never seen it done.

estebank 1y ago
> I don't think it's possible to write a tool like ripgrep with its performance profile in a GC language. I think it is possible to make a language that has both a GC and a borrow checker, treating the GC types as a third level next to the stack and the heap, where complex referencial cycles can bé promoted to the GC, but the defaults push you towards fast execution patterns. Don't know if such a language would be successful in finding its niche. The only way I could see that, is of a non-gc mode could be enforced so that libraries can be written in the more restrictive, faster by default mode, while being consumed by application developers that have less stringent restrictions. This is no different in concept than Python libraries implemented in native languages. Making it the mode be part of the same language could help with prototyping pains: write with the GC and then refactor once at the end after the general design is mostly found.
- burntsushi 1y ago
  My standard of evidence here is existence. There are many greps written in a "GC language," and none of them have a broad performance profile in the same category as ripgrep (or other greps written in C and C++). > This is no different in concept than Python libraries implemented in native languages. If that's true, then why isn't there a fast grep written in Python? IMO, it's for the same reason that tools like csvkit are slow. Once you get to the boundary between GC and non-GC, some kind of cost gets paid. Now maybe Python is paying higher costs here than your hypothetical language, but who knows. Without actual existence, I find this sort of hypothetical to be very uncompelling. It's too broad of a stroke.
- zozbot234 1y ago
  > I think it is possible to make a language that has both a GC and a borrow checker, treating the GC types as a third level next to the stack and the heap, where complex referencial cycles can be 'promoted to the GC' It's doable but there are a few issues with that whole idea. You need the ability to safely promote objects to GC-roots whenever they're being referenced by GC-unaware code, and demote them again afterwards. And if any GC object happens to control the lifecycle of any heap-allocated objects, it must have a finalizer that cleans them up RAII style to avoid resource leaks.

estebank 1y ago

> And that is especially likely when you use indices manually. But that is never the recommendation Rust practitioners would give for graph data structures. One would instead recommend using a generational arena, where each index holds their "generation". The arena can be growable or not. When an element is removed from the arena it gets tombstoned, marked as no longer valid. If a new value reuses a tombstoned position, its generation changes. This shifts the cost of verifying the handle is correct at the read point: if the handle corresponds to an index with a tombstone sentinel or has a different generation, the result of the read operation is None, meaning the handle is no longer valid. This is much better than the behavior of pointers.