4 ms·

RethinkDB internals: making coroutines fast

79 points by coffeemug 16y ago

jhrobert 16y ago
So, coroutine are as fast as callbacks and easier to program with... so, why aren't available is all language? I understand that it is difficult to change JavaScript, yet, when you create a whole new framework, say, nodejs, why not providing coroutines? Does it need a language construct to be efficient? Maybe, then have a look the Icon programming language, where coroutines rule,
- sharkbot 16y ago
  Lua's primary multitasking primitive is coroutines, and they work quite well. Preemptive threading support can be bolted onto the language, but the coroutine support is well thought-out and elegant, and doesn't suffer from the limitations of Python's generators (i.e., yielding from within nested function calls).
  - epall 16y ago
    Monocle (http://saucelabs.github.com/monocle/ http://saucelabs.github.com/monocle/) solves a lot of the issues with using Python generators as coroutines.
- silentbicycle 16y ago
  Languages that implement efficient coroutines (e.g. Scheme, Lua) usually manage the C stack themselves (like "Stackless" Python). First-class coroutines are very similar to first-class continuations, and many of the same fundamental issues manifest. Lua has one of the best coroutine implementations I've seen. Scheme has continuations which are convertible to coroutines, while Lua has coroutines which are convertible to (one-shot) continuations, with community insight on their use.
  - seiji 16y ago
    Silent explained it all very succinctly. If you get confused about coroutines versus continuations versus generators, check out everybody's favorite coroutine manual: http://www.inf.puc-rio.br/~roberto/docs/MCC15-04.pdf http://www.inf.puc-rio.br/~roberto/docs/MCC15-04.pdf
- strlen 16y ago
  Personally, I view node.JS as a step backwards: it's great to provide the idea of a single thread holding multiple connections, with high-performance non-blocking I/O within each thread; problem is, in their case, there's also a single thread per UNIX process with no primitives for communications between processes (unlike e.g., Erlang/OTP): if you'd want good performance, you want an event loop per logical core.; completely independent processes may be acceptable for simple, stateless web apps, but with most anything else you risk losing a great deal of efficiency without correct primitives. For example, if you have on-disk state, unless the threads are able to share memory (either by running within the same process or using UNIX shm) you're risking a situation where threads are competing for resources like OS page cache (provided you're not doing your own in-process caching and direct I/O, but even in this case, you've now lost the ability share a resource between any two connections without copying). Some e.g., Asana have added fibers to Javascript (in their case V8): http://asana.com/blog/?p=49 http://asana.com/blog/?p=49 The previous post from RethinkDB is quite interesting in terms of motivation for coroutines: to me, this superficially seems like SEDA. Here, instead of stages in the pipeline (thread pool per stage, first stage being processing events from epoll/kqueue, thread per core in each pool, each thread holding state machines, communication between threads via a queue) you are using coroutines for clearer code.
  - ericflo 16y ago
    I'm getting an "Error establishing a database connection" on that post. Kind of ironic. I agree about your point regarding IPC, but that's something they can always add later. It's unfortunate that they've gone the callback route instead of innovating like Asana has done with coroutines.
    - strlen 16y ago
      > I agree about your point regarding IPC, but that's something they can always add later. Add some form of IPC as an afterthought? Sure. Create powerful primitives for IPC as done in Erlang/OTP? No.
      - ericflo 16y ago
- bkudria 16y ago
  "Adding fibers to v8: efficiency + clarity in SSJS": http://asana.com/blog/?p=49 http://asana.com/blog/?p=49
warrenwilkinson 16y ago
I've blogged about co-routines before, and this post inspired me to write a little bit that contrasts RethinkDB with my own special-purpose DB that my startup uses. http://formlis.wordpress.com/2010/12/24/co-routines/ http://formlis.wordpress.com/2010/12/24/co-routines/
pmjordan 16y ago
Great stuff, good catch on the syscall! I've been tinkering with C coroutines as well recently but since our critical code runs in kernel space they ended up being more hassle than they're worth. I'm still using them in our (user space) testing sandbox. As you mention malloc()ing stacks: you might want to allocate them using mmap() and MAP_ANONYMOUS instead. You can map the adjacent memory pages with appropriate protection to prevent stack overflows and as a result, memory corruption. I'm not aware of any drawbacks (malloc itself typically uses mmap above a certain size) but it certainly beats hoping your stacks are big enough. Less of an issue on 64-bit environments of course.
- littledanehren 16y ago
  Thanks for the suggestion! I'm actually looking into doing something like that right now.
finiteloop 16y ago
This looks really cool. I have been messing around with coroutines a lot lately as well, testing libtask (http://swtch.com/libtask/ http://swtch.com/libtask/) as well as libcoroutine. Is the source for the changes to libcoroutine available? I didn't see a link from the article.
- coffeemug 16y ago
  We'd like to. The difficulty is that in order to get the performance improvements we had to integrate three different components: a few patches to libcoroutine, a patch to glibc, and pooling code that's tightly integrated with our asynchronous IO layer. A compelling patch would span three different licenses, involve patches to two libraries and re-engineering of the integrated piece to make it usable by the outside world. We'd like to do it, but for now this would be too much of a distraction.
mscarborough 16y ago
Sounds great, and I love the idea that RethinkDB is following. I don't know a lot about them though, and having an extra benchmark against a similar SSD installation of MySQL or Postgres with the same select query that achieved 1.5 million QPS would be pretty interesting.
modeless 16y ago
I prefer Grand Central Dispatch, which fixes the callback model by providing closures and anonymous functions as an extension to the C language. http://en.wikipedia.org/wiki/Grand_Central_Dispatch#Examples http://en.wikipedia.org/wiki/Grand_Central_Dispatch#Examples

79 points by coffeemug 16y ago

So, coroutine are as fast as callbacks and easier to program with... so, why aren't available is all language? I understand that it is difficult to change JavaScript, yet, when you create a whole new framework, say, nodejs, why not providing coroutines? Does it need a language construct to be efficient? Maybe, then have a look the Icon programming language, where coroutines rule,

sharkbot 16y ago
Lua's primary multitasking primitive is coroutines, and they work quite well. Preemptive threading support can be bolted onto the language, but the coroutine support is well thought-out and elegant, and doesn't suffer from the limitations of Python's generators (i.e., yielding from within nested function calls).
- epall 16y ago
  Monocle (http://saucelabs.github.com/monocle/ http://saucelabs.github.com/monocle/) solves a lot of the issues with using Python generators as coroutines.
silentbicycle 16y ago
Languages that implement efficient coroutines (e.g. Scheme, Lua) usually manage the C stack themselves (like "Stackless" Python). First-class coroutines are very similar to first-class continuations, and many of the same fundamental issues manifest. Lua has one of the best coroutine implementations I've seen. Scheme has continuations which are convertible to coroutines, while Lua has coroutines which are convertible to (one-shot) continuations, with community insight on their use.
- seiji 16y ago
  Silent explained it all very succinctly. If you get confused about coroutines versus continuations versus generators, check out everybody's favorite coroutine manual: http://www.inf.puc-rio.br/~roberto/docs/MCC15-04.pdf http://www.inf.puc-rio.br/~roberto/docs/MCC15-04.pdf
strlen 16y ago
Personally, I view node.JS as a step backwards: it's great to provide the idea of a single thread holding multiple connections, with high-performance non-blocking I/O within each thread; problem is, in their case, there's also a single thread per UNIX process with no primitives for communications between processes (unlike e.g., Erlang/OTP): if you'd want good performance, you want an event loop per logical core.; completely independent processes may be acceptable for simple, stateless web apps, but with most anything else you risk losing a great deal of efficiency without correct primitives. For example, if you have on-disk state, unless the threads are able to share memory (either by running within the same process or using UNIX shm) you're risking a situation where threads are competing for resources like OS page cache (provided you're not doing your own in-process caching and direct I/O, but even in this case, you've now lost the ability share a resource between any two connections without copying). Some e.g., Asana have added fibers to Javascript (in their case V8): http://asana.com/blog/?p=49 http://asana.com/blog/?p=49 The previous post from RethinkDB is quite interesting in terms of motivation for coroutines: to me, this superficially seems like SEDA. Here, instead of stages in the pipeline (thread pool per stage, first stage being processing events from epoll/kqueue, thread per core in each pool, each thread holding state machines, communication between threads via a queue) you are using coroutines for clearer code.
- ericflo 16y ago
  I'm getting an "Error establishing a database connection" on that post. Kind of ironic. I agree about your point regarding IPC, but that's something they can always add later. It's unfortunate that they've gone the callback route instead of innovating like Asana has done with coroutines.
  - strlen 16y ago
    > I agree about your point regarding IPC, but that's something they can always add later. Add some form of IPC as an afterthought? Sure. Create powerful primitives for IPC as done in Erlang/OTP? No.
    - ericflo 16y ago
bkudria 16y ago
"Adding fibers to v8: efficiency + clarity in SSJS": http://asana.com/blog/?p=49 http://asana.com/blog/?p=49

warrenwilkinson 16y ago

I've blogged about co-routines before, and this post inspired me to write a little bit that contrasts RethinkDB with my own special-purpose DB that my startup uses. http://formlis.wordpress.com/2010/12/24/co-routines/ http://formlis.wordpress.com/2010/12/24/co-routines/

pmjordan 16y ago

Great stuff, good catch on the syscall! I've been tinkering with C coroutines as well recently but since our critical code runs in kernel space they ended up being more hassle than they're worth. I'm still using them in our (user space) testing sandbox. As you mention malloc()ing stacks: you might want to allocate them using mmap() and MAP_ANONYMOUS instead. You can map the adjacent memory pages with appropriate protection to prevent stack overflows and as a result, memory corruption. I'm not aware of any drawbacks (malloc itself typically uses mmap above a certain size) but it certainly beats hoping your stacks are big enough. Less of an issue on 64-bit environments of course.

littledanehren 16y ago
Thanks for the suggestion! I'm actually looking into doing something like that right now.

finiteloop 16y ago

This looks really cool. I have been messing around with coroutines a lot lately as well, testing libtask (http://swtch.com/libtask/ http://swtch.com/libtask/) as well as libcoroutine. Is the source for the changes to libcoroutine available? I didn't see a link from the article.

coffeemug 16y ago
We'd like to. The difficulty is that in order to get the performance improvements we had to integrate three different components: a few patches to libcoroutine, a patch to glibc, and pooling code that's tightly integrated with our asynchronous IO layer. A compelling patch would span three different licenses, involve patches to two libraries and re-engineering of the integrated piece to make it usable by the outside world. We'd like to do it, but for now this would be too much of a distraction.

mscarborough 16y ago

Sounds great, and I love the idea that RethinkDB is following. I don't know a lot about them though, and having an extra benchmark against a similar SSD installation of MySQL or Postgres with the same select query that achieved 1.5 million QPS would be pretty interesting.

modeless 16y ago

I prefer Grand Central Dispatch, which fixes the callback model by providing closures and anonymous functions as an extension to the C language. http://en.wikipedia.org/wiki/Grand_Central_Dispatch#Examples http://en.wikipedia.org/wiki/Grand_Central_Dispatch#Examples