6 ms·
Am I missing something? It seems to propose to ditch all non-blocking APIs under the assumption that "runtimes" will provide lightweight concurrency. But AFAIK
by cafxx 10y ago
Am I missing something? It seems to propose to ditch all non-blocking APIs under the assumption that "runtimes" will provide lightweight concurrency. But AFAIK most (all?) runtimes that provide lightweight concurrency make extensive use of the non-blocking network APIs to work - including Golang, that is even cited as an example:
During the decades since BSD sockets were first introduced the way
they are used have changed significantly. While in the beginning the
user was supposed to fork a new process for each connection and do
all the work using simple blocking calls nowadays they are expected
to keep a pool of connections, check them via functions like poll()
or kqueue() and dispatch any work to be done to one of the worker
threads in a thread pool. In other words, user is supposed to do
both network and CPU scheduling.
[...]
To address this problem, this memo assumes that there already exists
an efficient concurrency implementation where forking a new
lightweight process takes at most hundreds of nanoseconds and context
switch takes tens of nanoseconds. Note that there are already such
concurrency systems deployed in the wild. One well-known example are
Golang's goroutines but there are others available as well.
- tptacek 10y agoApart from the layering, which really has nothing to do with the sockets interface and can be (and has been, repeatedly, as this author knows) built today on top of the sockets interface, I don't really see what's interesting at all about this interface; it seems like a simple restatement of the existing blocking socket API.
- rumcajz 10y agoIt tries to describe in detail what are the minimal semantic preconditions for protocol implementations to be composable (both vertically and horizontally) while at the same time being efficient (so that a stack with 20 microprotocols won't grind to a halt). The most interesting parts are handshakes, how they are conflated among multiple protocols as well as semantics of error handling. But the real proposition here is to have an ecosystem of microprotocols (e.g. a CRLF protocol that splits bytestream into messages using CR+LF as a delimiter) which, by the virtue of having common, well understood semantics, could by composed into ad-hoc networking stacks according to users' needs.
- JoshTriplett 10y agoAgreed; if anything, the blocking API seems like the more artificial one, as the lowest-level implementation needs to handle the asynchronous receipt of data. A network card receives data when it receives data, sends asynchronous notifications to the OS in the form of interrupts, and keeps receiving data while you pull data out of its buffers. You can build a synchronous interface on top of that for convenience, but it fundamentally works asynchronously.
- zzzcpan 10y agoYeah, one would expect for a new networking API proposal to embrace our asynchronous world, look into what people are already doing and why, not blocking APIs and multithreading. Personally, I was hoping for something in a spirit of netmap with the whole stack living in userspace for performance and security reasons and APIs being completely non-blocking with zero-copying and batching in mind.
- dboreham 10y agoDitching nonblocking mode would be ok if threads worked (by some definition of worked : for example 100K threads waiting on reads on open sockets performs well). But I think you are right that in present-day reality all the runtimes that provide "workingness" for concurrency (Go, perhaps the JVM, Erlang, homebrew libraries) do so by working around system threading in one way or another, and to do so they use non-blocking interfaces.
- rumcajz 10y agoThe implementation in the project uses its own lightweight threads (ctx switch ~20ns, possibility of arbitrarily small stacks, even stacks located on parent thread's stack). Also, it's written in C, so other languages can wrap it.
- jstimpfle 10y agoI think you've read that wrong. It's the other way around. Runtimes use non-blocking kernel interfaces to provide the convenient blocking interfaces to lightweight threads. So, no blocking kernel interfaces are needed anymore.
- tmyklebu 10y agoI'm not sure we have a sufficiently efficient concurrency mechanism that doesn't simply give the user control over scheduling decisions. Depending on how many tens we're talking about, those tens of nanoseconds for a context switch might themselves blow your time budget even on ordinary hardware. 10 gigabits per second means one byte is about 800ps long, so a 64-byte ethernet frame (plus the gunk before and after) is about 7 tens of nanoseconds. To keep up with this traffic pattern, we need to be able to process packets at least this quickly. A goroutine context switch plus one unbuffered channel read and one unbuffered channel write on my machine seems to take about 18 tens of nanoseconds on my machine when Go's runtime is made to use the same thread for everything. (I'm not sure how to better isolate context switch overhead in Go, but a context switch, channel read, and channel write per packet does seem to be the anticipated use case.)
- jstimpfle 10y agoIs 180ns that bad? Assuming that at each turn you process a packet of 180 bytes...
- tmyklebu 10y ago180ns overhead per packet means the time budget per packet for your business logic goes down by 180ns. That said, I suspect you can live a long life without ever encountering a situation where this matters.
- rumcajz 10y agoThe question to ask is: Can you do better? Say you use a state machine instead of using coroutines. Will one state transition of the machine be faster than one context switch? 180ns can definitely be improved on (with the C implementation in the project it's more likely to be 60ns or so) there's no fundamental reason why one approach should be faster than the other, given that both are doing basically the same thing, albeit on the different layers of the stack.
- tmyklebu 10y agoI'm not clear on why coroutines or an explicit state machine are necessary to implement network protocols. We have been doing it for decades by writing straightforward imperative C code and for years by writing C++ template classes that are templated on the downstream type (CRTP). Both of these approaches result in near-zero overhead (computers are good at fetching and decoding consecutive instructions) or even negative apparent overhead (if the compiler sees caller and callee, inlining and dead code elimination can potentially eliminate lower-level protocol handling irrelevant to the higher-level protocol). I would guess that transforming coroutine-based code into state machines hampers performance over the long term by scaring the programmer away from wanting to touch it. However, I would also guess that the direct cost of a coroutine context switch (save and restore all callee-save registers and stack pointer then indirect-jump somewhere unpredictable) outweighs the direct cost of a state machine transition (update some variable). I do not want to pass judgment on the approach beyond saying that its attendant overhead is too high for some workloads.