4 ms·

It's always interesting to see the experiences of new people to Rust. There's a few curious misconceptions in here that I hadn't seen before: > Iterators are a

by eridius 12y ago

It's always interesting to see the experiences of new people to Rust. There's a few curious misconceptions in here that I hadn't seen before: > Iterators are a great and reusable way to encapsulate your blah-blah-blah-blah-blah, but of course they’re impossible to optimize. I'm very curious to know where you got this idea from. Iterators actually optimize very well, in most cases being indistinguishable from a manual imperative for-loop. If you forget to turn on compiler optimizations then you'll see a significant performance difference, but that's true for a lot of different things you might want to do. Compiler optimizations are important whenever you're measuring performance. > pragma It's not a pragma. It uses the same # sigil that C compilers use to introduce pragmas, but in Rust, it actually denotes an attribute which modifies the next item (or in the enclosing item with the #! syntax). This is used for a number of things. As you've seen, it can be used to automatically derive implementations of traits, and it can be used to mark a function as being a candidate for inlining, among other things. > Rust rather inelegantly overloads the assignment operator to mean either binding or copying. This is indeed a very curious misconception. Assignment actually isn't overloaded like that at all. In your code example, when you say let y = x; You're not binding y to the value of x, you're actually moving the value of x into y. There is no implicit bind-by-reference in Rust. If you want a reference, you need to use the & sigil. The confusion here stems from the fact that some values can be copied and some values cannot. When you move a value in Rust, if the value can be copied (if it conforms to the Copy trait), then the original value is still usable after the move. This means you can say let x = 1; let y = x; let z = x; The line `let y = x` moves the value of x into y, but since the value is copyable, it's really just moving a copy of the value. In this context, "copyable" basically means memcpy() can be used to produce a valid copy. There's a different trait called Clone which has an explicit .clone() method that is used for values that require additional work beyond memcpy() to copy. In your code example, your struct is not Copy, so moving its value makes the original value inaccessible. Basically, it's considered garbage memory and cannot be read from again. This is why your code let x = MyValue::Digit(10); let y = x; let z = x; throws an error. It isn't because `y` and `z` would be referring to the same value, it's because after the line `let y = x;` the original value `x` is garbage. So really, any time you do assignment like that (or any time you pass a value as an argument to a function) without taking an explicit reference (using &), you're doing a move. Values that confirm to Copy will move a copy of the value, and all other values will leave the original value inaccessible. This should actually be familiar to people coming from C++, where values that aren't Copy are basically like std::unique_ptr, except that instead of leaving the original value with a known-"zero" state, the compiler prevents you from accessing the original value at all.

eridius 12y ago
Further commentary: > When Rust was originally announced, the team had ambitions to pursue the multi-core MacGuffin with a green-threaded actor model, but they found out that it’s very hard to do green threads with native code compilation. It's easy to do if you're willing to define certain synchronization points (such as I/O). IIRC, Rust abandoned Green threads because of a few reasons: 1. It doesn't play well with FFI. This is particularly true with segmented stacks, but Rust abandoned those before it abandoned green threading. With regular OS-provided stacks, FFI is a problem because the FFI call can't yield to the green scheduler. 2. It has an unavoidable performance problem even for code that doesn't use it. The existence of green threading means the entire runtime needs to be abstracted over a threading and I/O interface. This is a lot of complexity, and requires the use of dynamic dispatch for a lot of things that would otherwise be static dispatch. Rust originally switched to native as the default threading model but kept green threading as an option for a while, but this violates the concept of you-don't-pay-for-what-you-don't-use, especially since most people weren't even using green threading anymore. 3. It also made implementing some things very awkward. For example, because of the abstract notion of a runtime, you couldn't really implement selection over multiple file descriptors very well. Things like that must be handled by the runtime library or not at all, because for native threading you'd want to use select() (or kqueue or epoll) but for green threading the green threading library needs to deal with that. In Go, the standard solution there was to spawn multiple green threads that each block on one fd and communicate using channels (because the runtime can then collect all the blocked fds into a single select/kqueue/epoll), but that's unwanted overhead, and is especially not very good when you're not using the green threading. 4. Green threads encourage the use of lots of short-lived threads that have very shallow stacks. Rust abandoned segmented stacks with the understanding that the OS-provided virtual memory support was good enough that it didn't matter if every thread got an 8MB stack as it would allocate pages on-demand. But a 4k page for every green thread can be a lot of overhead if you're using thousands of green threads. You can of course explicitly request a smaller stack, but that's tricky to get right (if you request too little you explode). There was a lot of thought given to trying to statically determine how much stack space a given function call required (which is not always possible, and is especially tricky if there's any recursion) to deal with this, but that's a very hard problem. Ultimately, it was determined that green threads simply weren't worth all the costs. The OS is pretty good about dealing with lots of threads, and arguably it's actually better than the green threading library is about scheduling them all. Green threading makes some sense if you want to spawn 10,000 simultaneous threads, but in nearly all real-world workloads it's just not really worth it. > Rust has taken up a different challenge: eliminating crashes from otherwise traditional, “boring” multi-threaded programs. In the sense that Rust has abandoned its original vision in favor of pursuing more modest and achievable goals* I find this an odd thing to say. Statically eliminating crashes (and data races) without a garbage collector or any performance penalty is actually significantly more interesting than actors. Rust did not decide to pursue the more modest "boring" goal, it redoubled its focus on pursuing the very hard (but achievable) and very important goal of solving the problem of memory/data safety in multithreaded programs. This aspect of Rust is what most excited me when I first got involved 2 years ago and what still excites me to this day and makes me wish I could use Rust in my day job. > For the record, the Nim language manages to get this right, and the Rust folks might look to it for inspiration. I've heard of Nim before but haven't actually looked at it. Does it attempt to solve memory/data race safety at compile-time like Rust does? > It is also worth noting that processing non-overlapping slices in parallel is destined to come into mortal conflict with The Iterator, which is by its nature sequential I don't think this is true. You can build an Iterator that yields non-overlapping mutable slices, and then you can process them in parallel. The iterator is sequential, yes, but you can call it multiple times and hand the resulting references to your parallel computations (as opposed to handing the iterator itself over, which doesn't work because you'd be sharing mutable state without a lock and Rust does not allow you to write that code). Although it's worth noting that at the moment you can't do fork-join concurrency (so you can't share mutable slices with parallel computations like that unless you cheat with `unsafe`). There's `thread::scoped()` but it's marked as unstable (so you can't use it from Beta or from tomorrow's Stable 1.0) because it has a serious safety issue, and the proposed replacements are still being worked on.

by eridius 12y ago

eridius 12y ago

Further commentary: > When Rust was originally announced, the team had ambitions to pursue the multi-core MacGuffin with a green-threaded actor model, but they found out that it’s very hard to do green threads with native code compilation. It's easy to do if you're willing to define certain synchronization points (such as I/O). IIRC, Rust abandoned Green threads because of a few reasons: 1. It doesn't play well with FFI. This is particularly true with segmented stacks, but Rust abandoned those before it abandoned green threading. With regular OS-provided stacks, FFI is a problem because the FFI call can't yield to the green scheduler. 2. It has an unavoidable performance problem even for code that doesn't use it. The existence of green threading means the entire runtime needs to be abstracted over a threading and I/O interface. This is a lot of complexity, and requires the use of dynamic dispatch for a lot of things that would otherwise be static dispatch. Rust originally switched to native as the default threading model but kept green threading as an option for a while, but this violates the concept of you-don't-pay-for-what-you-don't-use, especially since most people weren't even using green threading anymore. 3. It also made implementing some things very awkward. For example, because of the abstract notion of a runtime, you couldn't really implement selection over multiple file descriptors very well. Things like that must be handled by the runtime library or not at all, because for native threading you'd want to use select() (or kqueue or epoll) but for green threading the green threading library needs to deal with that. In Go, the standard solution there was to spawn multiple green threads that each block on one fd and communicate using channels (because the runtime can then collect all the blocked fds into a single select/kqueue/epoll), but that's unwanted overhead, and is especially not very good when you're not using the green threading. 4. Green threads encourage the use of lots of short-lived threads that have very shallow stacks. Rust abandoned segmented stacks with the understanding that the OS-provided virtual memory support was good enough that it didn't matter if every thread got an 8MB stack as it would allocate pages on-demand. But a 4k page for every green thread can be a lot of overhead if you're using thousands of green threads. You can of course explicitly request a smaller stack, but that's tricky to get right (if you request too little you explode). There was a lot of thought given to trying to statically determine how much stack space a given function call required (which is not always possible, and is especially tricky if there's any recursion) to deal with this, but that's a very hard problem. Ultimately, it was determined that green threads simply weren't worth all the costs. The OS is pretty good about dealing with lots of threads, and arguably it's actually better than the green threading library is about scheduling them all. Green threading makes some sense if you want to spawn 10,000 simultaneous threads, but in nearly all real-world workloads it's just not really worth it. > Rust has taken up a different challenge: eliminating crashes from otherwise traditional, “boring” multi-threaded programs. In the sense that Rust has abandoned its original vision in favor of pursuing more modest and achievable goals* I find this an odd thing to say. Statically eliminating crashes (and data races) without a garbage collector or any performance penalty is actually significantly more interesting than actors. Rust did not decide to pursue the more modest "boring" goal, it redoubled its focus on pursuing the very hard (but achievable) and very important goal of solving the problem of memory/data safety in multithreaded programs. This aspect of Rust is what most excited me when I first got involved 2 years ago and what still excites me to this day and makes me wish I could use Rust in my day job. > For the record, the Nim language manages to get this right, and the Rust folks might look to it for inspiration. I've heard of Nim before but haven't actually looked at it. Does it attempt to solve memory/data race safety at compile-time like Rust does? > It is also worth noting that processing non-overlapping slices in parallel is destined to come into mortal conflict with The Iterator, which is by its nature sequential I don't think this is true. You can build an Iterator that yields non-overlapping mutable slices, and then you can process them in parallel. The iterator is sequential, yes, but you can call it multiple times and hand the resulting references to your parallel computations (as opposed to handing the iterator itself over, which doesn't work because you'd be sharing mutable state without a lock and Rust does not allow you to write that code). Although it's worth noting that at the moment you can't do fork-join concurrency (so you can't share mutable slices with parallel computations like that unless you cheat with `unsafe`). There's `thread::scoped()` but it's marked as unstable (so you can't use it from Beta or from tomorrow's Stable 1.0) because it has a serious safety issue, and the proposed replacements are still being worked on.