4 ms·

>It would be very valuable to have something like OpenMP (parallel loops, etc.) and something for concurrent IO (either asynchronous or coroutine-based). https

>It would be very valuable to have something like OpenMP (parallel loops, etc.) and something for concurrent IO (either asynchronous or coroutine-based). https://crates.io/crates/mio https://crates.io/crates/mio for the async IO. There was another library out there that provided tons of concurrency utilities, but I can't find it now. Parallel loop-like syntax should be easy with scoped_threadpool and a macro though. Rust tries to avoid stuffing everything into the standard library. So the lack of utilities in the stdlib isn't an artifact of some ignorance of concurrency, it's because Rust doesn't want to keep everything in the stdlib. Rust keeps the basic framework for thread safety in the stdlib (though it doesn't need to, not exactly), but the rest is built upon by the community.

acconsta 11y ago
>Rust tries to avoid stuffing everything into the standard library That's a valid philosophy, but also one that leads to problems with fragmentation, quality, portability, dependency management, and compiler support. Concurrent IO is surely important enough to standardize.
- burntsushi 11y ago
  > That's a valid philosophy, but also one that leads to problems with fragmentation, quality, portability, dependency management, and compiler support. No, not necessarily. We would like for the standard library to remain minimal, but one of its important functions is to collect common interfaces to maximize interoperability between crates. This has worked well in practice so far. Similarly, the standard library provides portable facades on top of platform specific APIs, for example, for performing IO. Crates can take advantage of this so that they can be portable themselves. Moreover, crates themselves can also provide portable facades over platform specific APIs, so I'm not convinced that this will be a problem in practice. Dependency management is handled quite well by Cargo. It has been a wonderful tool to have at our disposal and is really the crux of what makes a small standard library possible. Compiler support is a good argument, but one that I hope becomes weaker in time as we stabilize more functionality. To be clear, I agree that a small standard library has its own downsides. In particular, quality is IMO on of the best arguments against a small standard library. Fortunately, we trying to mitigate this by adopting officially blessed libraries into the `rust-lang` organization: https://github.com/rust-lang/rfcs/blob/master/text/1242-rust-lang-crates.md https://github.com/rust-lang/rfcs/blob/master/text/1242-rust... --- This allows us to avoid the problems with a big standard library (too much stuff that is hard to evolve because of stability) while still providing quality with crates that we promise to maintain.
  - 15155 11y ago
    > so I'm not convinced that this will be a problem in practice. Historically, it's been a massive problem. See my other post. Rust is already seeing IO-related fragmentation. C, C++, Ruby, Python, etc. are all massively-fragmented ecosystems regarding IO, concurrency, (safe) parallelism. I don't think we need to soil a great language (Rust) with these same mistakes.
    - burntsushi 11y ago
      I was specifically speaking about the size of Rust's standard library. Python, at least, has a massive standard library, so I'm not sure how that's applicable here. (Or, at the very least, demonstrates that a big standard library isn't sufficient. What matters is what is in the standard library, which is not fundamentally incompatible with a small standard library.) Also, none of those languages started out with a tool like Cargo. I made a few other comments about mitigating this as well that should be considered.
      - 15155 11y ago
        I don't agree with the idea that Cargo is just going to magically make all fragmentation disappear, simply because it's convenient, good, and available. If this were true, I would suspect we'd have seen some of this fragmentation stop in these other ecosystems: it hasn't, despite excellent tooling. I'm not suggesting a Python-esque stdlib. I'm suggesting a multi-threaded, cross-platform, (ideally, edge-triggered) event system, above which higher level primitives can be introduced and safely interoperate. If Rust already has std/net, this is not that far of a gap to close. Granted, implementing a reactor or (higher level) green threads greatly affects the way your programs execute, in my opinion, the benefits of a "blessed way" would outweigh the problems. Also, as with all of those languages with fragmented ecosystems: nothing is preventing a developer from implementing their own solutions, they're just heavily encouraged to be compatible.
        burntsushi 11y ago
        I never meant to imply that any one tool is a panacea. If this is an argument you think I'm making, then let's just squash that right now: I'm not. What I'm suggesting is that we have thought through this very problem and come up with a number of ways to mitigate it. We don't want to see the ecosystem fragmented. Cargo is one strategy. Blessed crates are another. In particular: > To be clear, I agree that a small standard library has its own downsides. In particular, quality is IMO on of the best arguments against a small standard library. Fortunately, we trying to mitigate this by adopting officially blessed libraries into the `rust-lang` organization: https://github.com/rust-lang/rfcs/blob/master/text/1242-rust-lang-crates.md https://github.com/rust-lang/rfcs/blob/master/text/1242-rust... --- This allows us to avoid the problems with a big standard library (too much stuff that is hard to evolve because of stability) while still providing quality with crates that we promise to maintain. I could absolutely see one of these crates providing async IO. I am less sure of seeing it wind up in std. Some examples of crates that are currently on track to being blessed---but maybe never end up in std---are regex and rand. I disagree with adding good package tooling after-the-fact and using its failure in preventing fragmentation as a reason for why Cargo will be ineffective. Overcoming inertia is hard. Having Cargo at the outset is a nice advantage we have working in our favor. We should acknowledge that. > If Rust already has std/net, this is not that far of a gap to close. It's a pretty big gap IMO, especially if you want to provide a common high level interface. std::net is a portable interface around platform specific APIs and not much else. Async IO is quite a bit more involved. > I'm not suggesting a Python-esque stdlib. I'm suggesting a multi-threaded, cross-platform, (ideally, edge-triggered) event system, above which higher level primitives can be introduced and safely interoperate. Note that my comment was specifically about arguing against a Python-esque stdlib, or rather, in favor of a small standard library. It was not meant to target omission of any one particular feature, which is what you seem to be focused on. (The criticism I responded to was not specific to async IO.)
15155 11y ago
What happens when one has 10 different ways of handling async IO? Massive ecosystem fragmentation. Ruby has EventMachine, the standard lib, and Celluloid. None of these are interoperable. Python has the standard lib, Twisted, and a number of other projects. All have the same problems. C++ has Boost, Asio, a few others. This is more along the lines of where Rust is headed. C has /countless/ options. None of them are standard, and it's totally understood in such an old language. --- Go has goroutines: it's a harmonious, unified ecosystem. I greatly dislike go, but this is one thing they do absolutely correctly. Erlang has fully-preemptive multitasking (which the entire ecosystem is built upon). Haskell has incredible parallelization and concurrency primitives built right into the stdlib. Certain things belong in the stdlib. Async IO is definitely one of them. If not an implementation, a defined, common interface of some nature, to help prevent some of the fragmentation. async/await, if I am not mistaken, will require compiler and borrow-checker support. This would be a good start.

acconsta 11y ago

>Rust tries to avoid stuffing everything into the standard library That's a valid philosophy, but also one that leads to problems with fragmentation, quality, portability, dependency management, and compiler support. Concurrent IO is surely important enough to standardize.

burntsushi 11y ago
> That's a valid philosophy, but also one that leads to problems with fragmentation, quality, portability, dependency management, and compiler support. No, not necessarily. We would like for the standard library to remain minimal, but one of its important functions is to collect common interfaces to maximize interoperability between crates. This has worked well in practice so far. Similarly, the standard library provides portable facades on top of platform specific APIs, for example, for performing IO. Crates can take advantage of this so that they can be portable themselves. Moreover, crates themselves can also provide portable facades over platform specific APIs, so I'm not convinced that this will be a problem in practice. Dependency management is handled quite well by Cargo. It has been a wonderful tool to have at our disposal and is really the crux of what makes a small standard library possible. Compiler support is a good argument, but one that I hope becomes weaker in time as we stabilize more functionality. To be clear, I agree that a small standard library has its own downsides. In particular, quality is IMO on of the best arguments against a small standard library. Fortunately, we trying to mitigate this by adopting officially blessed libraries into the `rust-lang` organization: https://github.com/rust-lang/rfcs/blob/master/text/1242-rust-lang-crates.md https://github.com/rust-lang/rfcs/blob/master/text/1242-rust... --- This allows us to avoid the problems with a big standard library (too much stuff that is hard to evolve because of stability) while still providing quality with crates that we promise to maintain.
- 15155 11y ago
  > so I'm not convinced that this will be a problem in practice. Historically, it's been a massive problem. See my other post. Rust is already seeing IO-related fragmentation. C, C++, Ruby, Python, etc. are all massively-fragmented ecosystems regarding IO, concurrency, (safe) parallelism. I don't think we need to soil a great language (Rust) with these same mistakes.
  - burntsushi 11y ago
    I was specifically speaking about the size of Rust's standard library. Python, at least, has a massive standard library, so I'm not sure how that's applicable here. (Or, at the very least, demonstrates that a big standard library isn't sufficient. What matters is what is in the standard library, which is not fundamentally incompatible with a small standard library.) Also, none of those languages started out with a tool like Cargo. I made a few other comments about mitigating this as well that should be considered.
    - 15155 11y ago
      I don't agree with the idea that Cargo is just going to magically make all fragmentation disappear, simply because it's convenient, good, and available. If this were true, I would suspect we'd have seen some of this fragmentation stop in these other ecosystems: it hasn't, despite excellent tooling. I'm not suggesting a Python-esque stdlib. I'm suggesting a multi-threaded, cross-platform, (ideally, edge-triggered) event system, above which higher level primitives can be introduced and safely interoperate. If Rust already has std/net, this is not that far of a gap to close. Granted, implementing a reactor or (higher level) green threads greatly affects the way your programs execute, in my opinion, the benefits of a "blessed way" would outweigh the problems. Also, as with all of those languages with fragmented ecosystems: nothing is preventing a developer from implementing their own solutions, they're just heavily encouraged to be compatible.
      - burntsushi 11y ago
        I never meant to imply that any one tool is a panacea. If this is an argument you think I'm making, then let's just squash that right now: I'm not. What I'm suggesting is that we have thought through this very problem and come up with a number of ways to mitigate it. We don't want to see the ecosystem fragmented. Cargo is one strategy. Blessed crates are another. In particular: > To be clear, I agree that a small standard library has its own downsides. In particular, quality is IMO on of the best arguments against a small standard library. Fortunately, we trying to mitigate this by adopting officially blessed libraries into the `rust-lang` organization: https://github.com/rust-lang/rfcs/blob/master/text/1242-rust-lang-crates.md https://github.com/rust-lang/rfcs/blob/master/text/1242-rust... --- This allows us to avoid the problems with a big standard library (too much stuff that is hard to evolve because of stability) while still providing quality with crates that we promise to maintain. I could absolutely see one of these crates providing async IO. I am less sure of seeing it wind up in std. Some examples of crates that are currently on track to being blessed---but maybe never end up in std---are regex and rand. I disagree with adding good package tooling after-the-fact and using its failure in preventing fragmentation as a reason for why Cargo will be ineffective. Overcoming inertia is hard. Having Cargo at the outset is a nice advantage we have working in our favor. We should acknowledge that. > If Rust already has std/net, this is not that far of a gap to close. It's a pretty big gap IMO, especially if you want to provide a common high level interface. std::net is a portable interface around platform specific APIs and not much else. Async IO is quite a bit more involved. > I'm not suggesting a Python-esque stdlib. I'm suggesting a multi-threaded, cross-platform, (ideally, edge-triggered) event system, above which higher level primitives can be introduced and safely interoperate. Note that my comment was specifically about arguing against a Python-esque stdlib, or rather, in favor of a small standard library. It was not meant to target omission of any one particular feature, which is what you seem to be focused on. (The criticism I responded to was not specific to async IO.)

15155 11y ago

What happens when one has 10 different ways of handling async IO? Massive ecosystem fragmentation. Ruby has EventMachine, the standard lib, and Celluloid. None of these are interoperable. Python has the standard lib, Twisted, and a number of other projects. All have the same problems. C++ has Boost, Asio, a few others. This is more along the lines of where Rust is headed. C has /countless/ options. None of them are standard, and it's totally understood in such an old language. --- Go has goroutines: it's a harmonious, unified ecosystem. I greatly dislike go, but this is one thing they do absolutely correctly. Erlang has fully-preemptive multitasking (which the entire ecosystem is built upon). Haskell has incredible parallelization and concurrency primitives built right into the stdlib. Certain things belong in the stdlib. Async IO is definitely one of them. If not an implementation, a defined, common interface of some nature, to help prevent some of the fragmentation. async/await, if I am not mistaken, will require compiler and borrow-checker support. This would be a good start.