3 ms·

> "ndarray" has unsafe indexing We've had this discussion before. ndarray exposes both a safe and unsafe API for indexing, just like the stdlib vector (or core

> "ndarray" has unsafe indexing We've had this discussion before. ndarray exposes both a safe and unsafe API for indexing, just like the stdlib vector (or core arrays). That is not problematic. I agree that the lack of standardization is a problem, though. I think that mostly folks use ndarray or nalgebra (and num if they need bigints). Anyone can upload a crate, the question is if the crate is the main one used by the community. There are some efforts to make it easier to choose crates amongst alternatives on crates.io, however. I suspect most of the issue here is just that no group (only individuals) is using Rust for scientific computing yet, so there's no large driving effort behind getting good libraries here. When I needed this in Rust I just picked the library that I thought would work best with very little thought, with no thoughts on writing my own or improving the lib because I didn't have time. When larger groups work on things generally attention gets paid to stuff like this and better solutions come out on top. Rust isn't quite there yet in adoption for this to happen, maybe soon :)

Animats 10y ago
Is it really necessary to expose an unsafe API that bypasses subscript checking? Another Rust advocate was arguing, the last time this came up, that LLVM could optimize out most of the subscript checks. As I showed, it optimizes out about half of them for multidimensional arrays implemented with an explicit multiply. That may improve, especially if there's some standard idiom for declaring multidimensional arrays and the compiler handles that idiom well. For 1D, though, the optimization of checking is pretty good. "get_unchecked" for Vec may be obsolete. There's an amusing Stack Overflow question [1] from someone who complains that he changed an access from "[]" to "get_unchecked()" and his program didn't get faster. Maybe it's time to deprecate some of the legacy "unsafe" stuff. Preferably before the first CERT advisory involving a buffer overflow in a Rust program. [1] http://stackoverflow.com/questions/39196594/why-dont-i-get-performance-improvement-by-using-get-unchecked http://stackoverflow.com/questions/39196594/why-dont-i-get-p...
- Manishearth 10y ago
  > Is it really necessary to expose an unsafe API that bypasses subscript checking? Sometimes you don't want to rely on LLVM, and sometimes the size invariants aren't as easily available to LLVM. I'm not even sure if this would be that necessary outside of the library itself; unchecked indexing would be useful to implement ops like matrix multiplication within the library, once, and probably never used after that. The API exists mostly to mirror the stdlib one and its use cases. I don't think it's supposed to be the API you reach for usually. Just because the API exists doesn't mean it's the one you're supposed to reach for. > That may improve, especially if there's some standard idiom for declaring multidimensional arrays and the compiler handles that idiom well. I mean, multidimensional arrays do exist in the language. I'm not sure what you're getting at here. These libraries provide support for treating multidimensional arrays as matrices, with the semantics of matrices (multiplication, addition, etc). You seem to be arguing for making this a first-class compiler feature? I'm not sure how that's very different from just implementing it with some unsafe code -- the bug-prone-ness of the unsafe code is about the same as the bug-prone-ness of a compiler optimization that replicates it. There is a case to be made for moving it into the stdlib itself to avoid the "10 ways to do it" issue, but Rust wants to keep the stdlib small so that's not going to happen. This is the point of unsafe code. You use it to implement a few safe abstractions like matrix multiplication and iteration in libraries, verify that, and use them. There's a stigma attached to unsafe code; rightly so; because there be nasal demons. But we shouldn't take it to its extreme and conclude that all unsafe code is bad. > "get_unchecked" for Vec may be obsolete. Wouldn't count on it. A single datapoint doesn't really mean much here. I mostly see get_unchecked being used in the context of other unsafe code. Rust's stdlib uses it for the internals of CString and for implementing iterators. It also uses it in an optimization for the rand crate; which does some trickyness with bitmask indexing that LLVM might optimize but I suspect the authors didn't want to rely on LLVM optimizing. Servo doesn't use it at all, though. Going through my cargo cache dir the crates that use it are those which implement abstractions. Some, but not all, of the use cases there probably get optimized out anyway. The regex stuff also uses it but the invariants there are complicated enough that llvm won't optimize it (there are some decent comments listing out these invariants and explaining exactly how to use it safely, though).

Animats 10y ago

Is it really necessary to expose an unsafe API that bypasses subscript checking? Another Rust advocate was arguing, the last time this came up, that LLVM could optimize out most of the subscript checks. As I showed, it optimizes out about half of them for multidimensional arrays implemented with an explicit multiply. That may improve, especially if there's some standard idiom for declaring multidimensional arrays and the compiler handles that idiom well. For 1D, though, the optimization of checking is pretty good. "get_unchecked" for Vec may be obsolete. There's an amusing Stack Overflow question [1] from someone who complains that he changed an access from "[]" to "get_unchecked()" and his program didn't get faster. Maybe it's time to deprecate some of the legacy "unsafe" stuff. Preferably before the first CERT advisory involving a buffer overflow in a Rust program. [1] http://stackoverflow.com/questions/39196594/why-dont-i-get-performance-improvement-by-using-get-unchecked http://stackoverflow.com/questions/39196594/why-dont-i-get-p...

Manishearth 10y ago
> Is it really necessary to expose an unsafe API that bypasses subscript checking? Sometimes you don't want to rely on LLVM, and sometimes the size invariants aren't as easily available to LLVM. I'm not even sure if this would be that necessary outside of the library itself; unchecked indexing would be useful to implement ops like matrix multiplication within the library, once, and probably never used after that. The API exists mostly to mirror the stdlib one and its use cases. I don't think it's supposed to be the API you reach for usually. Just because the API exists doesn't mean it's the one you're supposed to reach for. > That may improve, especially if there's some standard idiom for declaring multidimensional arrays and the compiler handles that idiom well. I mean, multidimensional arrays do exist in the language. I'm not sure what you're getting at here. These libraries provide support for treating multidimensional arrays as matrices, with the semantics of matrices (multiplication, addition, etc). You seem to be arguing for making this a first-class compiler feature? I'm not sure how that's very different from just implementing it with some unsafe code -- the bug-prone-ness of the unsafe code is about the same as the bug-prone-ness of a compiler optimization that replicates it. There is a case to be made for moving it into the stdlib itself to avoid the "10 ways to do it" issue, but Rust wants to keep the stdlib small so that's not going to happen. This is the point of unsafe code. You use it to implement a few safe abstractions like matrix multiplication and iteration in libraries, verify that, and use them. There's a stigma attached to unsafe code; rightly so; because there be nasal demons. But we shouldn't take it to its extreme and conclude that all unsafe code is bad. > "get_unchecked" for Vec may be obsolete. Wouldn't count on it. A single datapoint doesn't really mean much here. I mostly see get_unchecked being used in the context of other unsafe code. Rust's stdlib uses it for the internals of CString and for implementing iterators. It also uses it in an optimization for the rand crate; which does some trickyness with bitmask indexing that LLVM might optimize but I suspect the authors didn't want to rely on LLVM optimizing. Servo doesn't use it at all, though. Going through my cargo cache dir the crates that use it are those which implement abstractions. Some, but not all, of the use cases there probably get optimized out anyway. The regex stuff also uses it but the invariants there are complicated enough that llvm won't optimize it (there are some decent comments listing out these invariants and explaining exactly how to use it safely, though).