4 ms·

> has no good epoll/select abstraction, which is wrong I can't reiterate this enough. Rust has one of the best abstractions I've ever used over epoll/kqueue an

> has no good epoll/select abstraction, which is wrong I can't reiterate this enough. Rust has one of the best abstractions I've ever used over epoll/kqueue and MS' variant, MIO. MIO has picked not the crappy POSIX select, which every major OS has a better alternative to now, but instead the OS' preferred event system. The 10 year stability requirement is awkward, as I don't think anyone can predict the future, and in 10 years, who knows what will change. As you mention, tokio is an awesome futures based abstraction on top of that, making it a breeze to write complex, fast, event driven logic. I'm sad that after four days he didn't like Rust, I had a 100% different experience. After one day I fell in love with nearly every aspect of the language, and it was specifically because of state machines (which they mention as one of their core desires). My biggest complaint against the language hasn't changed, which is dealing with different Error types, but even that after a lot more experience with Rust has become easier. Anyway, for NTPSec it's sad that this didn't suit their desires, but that doesn't mean that we can't write our own.

zbobet2012 10y ago
Go's epoll abstraction is baked into the runtime. They are called goroutines. Goroutines blocked of FD's automatically use epoll and schedule based on its results: https://github.com/golang/go/blob/master/src/runtime/netpoll_epoll.go https://github.com/golang/go/blob/master/src/runtime/netpoll...
- bluejekyll 10y ago
  I wasn't commenting on go, just pointing out the library that most people in Rust are using for async-io.
AsyncAwait 10y ago
> My biggest complaint against the language hasn't changed, which is dealing with different Error types Do you mean syntactic sugar? Because there's try! and ? Or do you mean dealing with Option and Result in a more uniform way?
- bluejekyll 10y ago
  So error_chain! is actually the answer to most of my issues with it, but it was the largest hurdle for me (coming mainly from Java) where you need to convert from one type to the next up the return stack. It took me a while to get used to this, but I'd say it's a large hurdle when learning the language. And, while error-chain is great, it means introducing you to macros and such if you try to adopt it early on. Mind you, this is my biggest complaint, and it's trivial for me now, but after you grok the other major features of the language, properly dealing with errors/results is a hurdle for people new to the language. link: https://docs.rs/error-chain/0.7.2/error_chain/ https://docs.rs/error-chain/0.7.2/error_chain/
soulbadguy 10y ago
Best abstraction over epoll/kqueue ? As compared to what ? From all the things that rust does right, MIO event loop seems lacking as compared to boost::asio for example.
- bluejekyll 10y ago
  It strikes a good balance of portability and features, and where MIO leaves off, tokio picks up. When comparing to other libraries in other languages, MIO probably looks like it's devoid of features, but that's because it's trying to be an abstraction layer only, and tokio is probably where most people should turn for higher level semantics...
mixedCase 10y ago
It might be that MIO is a good epoll abstraction, but in Go you don't have to care wether your code blocks on IO or if it's computationally intensive and requires a separate, costlier thread. You just spawn a goroutine. Go's runtime does the rest. And that for me that is the way I want to do concurrency and parallelism for pretty much everything I'll ever write.
- pcwalton 10y ago
  > And that for me that is the way I want to do concurrency and parallelism for pretty much everything I'll ever write. 1. You can get exactly this with concurrency by spawning one thread per request. The only difference between Go and the pthread model is that Go has a userland scheduler. 2. If I tried to do my parallelism work using this model, the amount of multicore speedup I would get would be zero (actually, it would be negative). Sometimes you're compiling multiple .o files--you get lucky and the overhead of spawning a thread per work item is not restrictive. Sometimes you're shading individual pixels--spawn a goroutine per pixel and you're going to be laughably slow.
  - wahern 10y ago
    The only difference between Go and the pthread model is that Go has a userland scheduler. That's a _huge_ difference in the context of a network service, and also obscures some other details. If I have several thousand long-polling clients, one kernel thread per client is simply not realistic. It uses up alot of memory, and the context switching can be costly. Throwing more hardware at the problem is just throwing money down the drain. And if throwing money down the drain is fine, one might as well use one process per connection with blocking I/O. In order to avoid that in Rust, Node, or most other languages one needs to use a callback or future. But both a callback and future require a dynamic allocation and deallocation per invocation of each function that might block. Especially under heavy load (lots of broadcasts), that's alot when you consider chaining invocations, as you must when composing non-blocking I/O interfaces. Chaining futures hinders compiler optimizations. And in general it hinders function composition. (I commonly see claims that the react pattern is better for composition, yet I never see such people writing _all_ their code that way, e.g. for things like string manipulation. In reality most concurrency, even in network services, is effectively sequential within a wider context beyond the immediate operation. Have you ever tried to implement, e.g., a non-blocking MySQL protocol parser with callbacks/futures? It's ugly.) People underestimate how efficient and useful storing function invocation state on a stack is. Goroutines (or contiguous, dynamically-sizeable stacks like in Lua) give you the best of all worlds--efficiency of a contiguous stack without the high fixed costs. That means performance _and_ scalability for massively concurrent network services, but also for other patterns--like being able to convert a callback-based push interface into a pull interface, without having to refactor the intermediate code, which can be amazingly powerful for things like parsers. But for a language like Rust it's understandably difficult to implement stackful coroutines, let alone goroutines. I just wish people wouldn't whitewash the issue.

zbobet2012 10y ago

Go's epoll abstraction is baked into the runtime. They are called goroutines. Goroutines blocked of FD's automatically use epoll and schedule based on its results: https://github.com/golang/go/blob/master/src/runtime/netpoll_epoll.go https://github.com/golang/go/blob/master/src/runtime/netpoll...

bluejekyll 10y ago
I wasn't commenting on go, just pointing out the library that most people in Rust are using for async-io.

AsyncAwait 10y ago

> My biggest complaint against the language hasn't changed, which is dealing with different Error types Do you mean syntactic sugar? Because there's try! and ? Or do you mean dealing with Option and Result in a more uniform way?

bluejekyll 10y ago
So error_chain! is actually the answer to most of my issues with it, but it was the largest hurdle for me (coming mainly from Java) where you need to convert from one type to the next up the return stack. It took me a while to get used to this, but I'd say it's a large hurdle when learning the language. And, while error-chain is great, it means introducing you to macros and such if you try to adopt it early on. Mind you, this is my biggest complaint, and it's trivial for me now, but after you grok the other major features of the language, properly dealing with errors/results is a hurdle for people new to the language. link: https://docs.rs/error-chain/0.7.2/error_chain/ https://docs.rs/error-chain/0.7.2/error_chain/

soulbadguy 10y ago

Best abstraction over epoll/kqueue ? As compared to what ? From all the things that rust does right, MIO event loop seems lacking as compared to boost::asio for example.

bluejekyll 10y ago
It strikes a good balance of portability and features, and where MIO leaves off, tokio picks up. When comparing to other libraries in other languages, MIO probably looks like it's devoid of features, but that's because it's trying to be an abstraction layer only, and tokio is probably where most people should turn for higher level semantics...

mixedCase 10y ago

It might be that MIO is a good epoll abstraction, but in Go you don't have to care wether your code blocks on IO or if it's computationally intensive and requires a separate, costlier thread. You just spawn a goroutine. Go's runtime does the rest. And that for me that is the way I want to do concurrency and parallelism for pretty much everything I'll ever write.

pcwalton 10y ago
> And that for me that is the way I want to do concurrency and parallelism for pretty much everything I'll ever write. 1. You can get exactly this with concurrency by spawning one thread per request. The only difference between Go and the pthread model is that Go has a userland scheduler. 2. If I tried to do my parallelism work using this model, the amount of multicore speedup I would get would be zero (actually, it would be negative). Sometimes you're compiling multiple .o files--you get lucky and the overhead of spawning a thread per work item is not restrictive. Sometimes you're shading individual pixels--spawn a goroutine per pixel and you're going to be laughably slow.
- wahern 10y ago
  The only difference between Go and the pthread model is that Go has a userland scheduler. That's a _huge_ difference in the context of a network service, and also obscures some other details. If I have several thousand long-polling clients, one kernel thread per client is simply not realistic. It uses up alot of memory, and the context switching can be costly. Throwing more hardware at the problem is just throwing money down the drain. And if throwing money down the drain is fine, one might as well use one process per connection with blocking I/O. In order to avoid that in Rust, Node, or most other languages one needs to use a callback or future. But both a callback and future require a dynamic allocation and deallocation per invocation of each function that might block. Especially under heavy load (lots of broadcasts), that's alot when you consider chaining invocations, as you must when composing non-blocking I/O interfaces. Chaining futures hinders compiler optimizations. And in general it hinders function composition. (I commonly see claims that the react pattern is better for composition, yet I never see such people writing _all_ their code that way, e.g. for things like string manipulation. In reality most concurrency, even in network services, is effectively sequential within a wider context beyond the immediate operation. Have you ever tried to implement, e.g., a non-blocking MySQL protocol parser with callbacks/futures? It's ugly.) People underestimate how efficient and useful storing function invocation state on a stack is. Goroutines (or contiguous, dynamically-sizeable stacks like in Lua) give you the best of all worlds--efficiency of a contiguous stack without the high fixed costs. That means performance _and_ scalability for massively concurrent network services, but also for other patterns--like being able to convert a callback-based push interface into a pull interface, without having to refactor the intermediate code, which can be amazingly powerful for things like parsers. But for a language like Rust it's understandably difficult to implement stackful coroutines, let alone goroutines. I just wish people wouldn't whitewash the issue.