4 ms·

I like the redis protocol compatibility and the HTTP compatibility, but from the initial skim through I guess you are using abseil-cpp and the home-grown helio

I like the redis protocol compatibility and the HTTP compatibility, but from the initial skim through I guess you are using abseil-cpp and the home-grown helio (https://github.com/romange/helio https://github.com/romange/helio) library. Could you get me a one liner on the helio library is it used as a fiber wrapper around the io_uring facility in the kernel? Can it be used as a standalone library for implementing fibers in application code? Also it seems that spinlock has become a defacto standard in the DB world today, thanks for not falling into the trap (because 90% of the users of any DB do not need spinlocks). Another curious question would be - why not implement with seastar (since you're not speaking to disk often enough)?

romange 4y ago
Yes, helio is the library that allows you to build c++ backends easily similar to Seastar. Unlike Seastar that is designed as futures and continuations library, helio uses fibers which I think simpler to use and reason about. I've wrote a few blog posts a while ago about fibers and Seastar: https://www.romange.com/2018/07/12/seastar-asynchronous-c-framework/ https://www.romange.com/2018/07/12/seastar-asynchronous-c-fr... one of them. You will see there a typical Seastar flow with continuations. I just do not like this style and I think C++ is not a good fit for it. Having said that, I do think Seastar is 5-star framework and the team behind it are all superstars. I learned about shared-nothing architecture from Seastar. Re helio: You will find examples folder inside the projects with sample backends: echo_server and pingpong_server. Both are similar but the latter speaks RESP. I also implemented a toy midi-redis project https://github.com/romange/midi-redis https://github.com/romange/midi-redis which is also based on helio. In fact dragonfly evolved from it. Another interesting moment about Seastarr - I decided to adopt io_uring as my only polling API and Seastar did not use io_uring at that time.
- sudarshnachakra 4y ago
  Thanks for taking the time to reply - yes in fact seastar does not use io_uring but it's rust equivalent glommio does use it (IIRC it is based on io_uring). Any reasons for using c++ instead of Rust (are u more familiar with it? or just the learning curve hinders the time to market? or is it the Rc/Arc fatigue with rust async? I guess Rust should be a fairly easy language to pick up for good c++ programmers like you)
  - romange 4y ago
    If I would choose another language it would be Rust. Why I did not choose Rust? 1. I speak fluently C++ and learning Rust would take me years. 2. Foodchain of libraries that I am intimately fimiliar with in C++ and I am not familiar with in Rust. Take Rust Tokyo, for example. This is the de facto standard for how to build I/O backends. However if you benchmark Tokyo's min-redis with memtier_benchmark you will see it has much lower throughput than helio and much higher latency. (At least this is what I observed a year ago). Tokyo is a combination of myriad design decisions that authors of the framework had to do to serve tha mainstream of use-cases. helio is opinionated. DF is opinionated. Shared-nothing architecture is not for everyone. But if you master it - it's invincible.
- alecco 4y ago
  That blog post is a bit dated. That's the old way. Seastar supports C++20 coroutines. http://docs.seastar.io/master/tutorial.html#coroutines http://docs.seastar.io/master/tutorial.html#coroutines

romange 4y ago

Yes, helio is the library that allows you to build c++ backends easily similar to Seastar. Unlike Seastar that is designed as futures and continuations library, helio uses fibers which I think simpler to use and reason about. I've wrote a few blog posts a while ago about fibers and Seastar: https://www.romange.com/2018/07/12/seastar-asynchronous-c-framework/ https://www.romange.com/2018/07/12/seastar-asynchronous-c-fr... one of them. You will see there a typical Seastar flow with continuations. I just do not like this style and I think C++ is not a good fit for it. Having said that, I do think Seastar is 5-star framework and the team behind it are all superstars. I learned about shared-nothing architecture from Seastar. Re helio: You will find examples folder inside the projects with sample backends: echo_server and pingpong_server. Both are similar but the latter speaks RESP. I also implemented a toy midi-redis project https://github.com/romange/midi-redis https://github.com/romange/midi-redis which is also based on helio. In fact dragonfly evolved from it. Another interesting moment about Seastarr - I decided to adopt io_uring as my only polling API and Seastar did not use io_uring at that time.

sudarshnachakra 4y ago
Thanks for taking the time to reply - yes in fact seastar does not use io_uring but it's rust equivalent glommio does use it (IIRC it is based on io_uring). Any reasons for using c++ instead of Rust (are u more familiar with it? or just the learning curve hinders the time to market? or is it the Rc/Arc fatigue with rust async? I guess Rust should be a fairly easy language to pick up for good c++ programmers like you)
- romange 4y ago
  If I would choose another language it would be Rust. Why I did not choose Rust? 1. I speak fluently C++ and learning Rust would take me years. 2. Foodchain of libraries that I am intimately fimiliar with in C++ and I am not familiar with in Rust. Take Rust Tokyo, for example. This is the de facto standard for how to build I/O backends. However if you benchmark Tokyo's min-redis with memtier_benchmark you will see it has much lower throughput than helio and much higher latency. (At least this is what I observed a year ago). Tokyo is a combination of myriad design decisions that authors of the framework had to do to serve tha mainstream of use-cases. helio is opinionated. DF is opinionated. Shared-nothing architecture is not for everyone. But if you master it - it's invincible.
alecco 4y ago
That blog post is a bit dated. That's the old way. Seastar supports C++20 coroutines. http://docs.seastar.io/master/tutorial.html#coroutines http://docs.seastar.io/master/tutorial.html#coroutines