4 ms·

Building a database buffer pool in Zig using io_uring's new fixed-buffer mode

158 points by gavinray 4y ago

cmrdporcupine 4y ago
Aha! I was thinking about trying this exact same approach recently, so I'm glad to see I wasn't insane! Very clever. I have been "in my spare time" (that is, never finishing) building out a page buffer mgmt system based on the one in the Umbra paper (see https://db.in.tum.de/~freitag/papers/p29-neumann-cidr20.pdf https://db.in.tum.de/~freitag/papers/p29-neumann-cidr20.pdf) which has a kind of novel approach where they essentially allocate the same physical address at multiple virtual addresses but with different block/slab sizes so that they can reduce fragmentation and still support dynamic page sizes. Anyways, it seems this author is also on the same "page" (sorry) as me but focusing more on exploiting `io_uring`. Nifty. I'm working in Rust, not Zig, but a lot of nice stuff to read through here. I will have to read through this in more detail later when I have more brain-space available.
- gavinray 4y ago
  Jinx, I'm also working on this, as a tutorial series! I had the pleasure of hearing Neumann present the Umbra architecture on the CMU DB series webinar recently. I'm slowly working through re-implementing the Coroutines async buffer pool paper. Blocked on trying to properly implement and understand "Morsel-Driven Parallelism": https://db.in.tum.de/~fent/papers/coroutines.pdf https://db.in.tum.de/~fent/papers/coroutines.pdf I have the async, io_uring buffer pool written in Kotlin using it's native Coroutines but the scheduling and morsel stuff I'm still trying to understand
  - cmrdporcupine 4y ago
    I watched the same CMU lecture, very cool. Are you getting paid to work on this kind of thing, anywhere? Do you want to be?
    - gavinray 4y ago
      > Are you getting paid to work on this kind of thing, anywhere? My dayjob is a fair bit higher level. I work on a sort of query engine on the JVM that translates an internal query IR/AST into dialect-specific SQL + other languages so you can query anything with the same frontend language: https://techcrunch.com/2022/06/28/hasura-now-lets-developers-turn-any-data-source-into-a-graphql-api/ https://techcrunch.com/2022/06/28/hasura-now-lets-developers... I think it'd be neato to work on a database, but my dayjob is also pretty interesting to me.
- gavinray 4y ago
  > I'm working in Rust, not Zig, but a lot of nice stuff to read through here. FYI, Datadog has a Rust library for scheduling things to run thread-per-core with io_uring It'd be really useful for DB use cases: https://github.com/DataDog/glommio https://github.com/DataDog/glommio
  - cmrdporcupine 4y ago
    Nifty. I started from the part of the Umbra presentation that initially struck me as most interesting -- the allocator portion (where he talks about repeatedly MMAP_ANONYMOUS on the same address range to reduce fragmentation, etc.) I stopped working when I started pondering the bits about pointer swizzling and how I'd make that work nicely in Rust, and never got as far as the pieces that would actually do the syncs to disk. I also came across another fellow who was inspired by the optimistic locking btree presented in LeanStore Umbra: https://github.com/NeowayLabs/bplustree https://github.com/NeowayLabs/bplustree
gavinray 4y ago
One note: I mistakenly thought that fixed buffers were part of the 5.19 changes for provided buffers Jens Axboe corrected me on this, fixed buffers have existed for a while apparently! https://twitter.com/axboe/status/1581392910687567873?t=gx_tQCEwL0Kh_OfO1QEBow&s=19 https://twitter.com/axboe/status/1581392910687567873?t=gx_tQ...
andrewedstrom 4y ago
- e12e 4y ago
  > I’ve never heard of a company actually using it Depending on your definition of "company" and "actually using it", https://oven.sh https://oven.sh might qualify?
- ksec 4y ago
  People say that, yet it is not anywhere near Rust. But yes, I do wish people could now keep Zig submission under control. I dont want Zig community becomes another Rust community. Zig https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=Zig&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu... Rust https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=Rust&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
  - unwind 4y ago
    I'm in neither community, so what does that even mean? Why?
  - kristoff_it 4y ago
    > I do wish people could now keep Zig submission under control. I dont want Zig community becomes another Rust community. I don't think anybody has control over what gets posted on HN or not. People act independently and to whatever fancies them. Should the author of this blog post have avoided using Zig to keep the frontpage of HN more balanced? Obviously no. Maybe the curators of HN could prevent Zig posts from reaching the front page but they too have their own motives (eg having the frontpage reflect the topics of interest of the collective HN readership) and it might just be that these are Zig's 15mins of fame. So I don't think your request can be fulfilled in practice and I don't think that Zig's popularity (or lack thereof) on HN will impact our community. I would recommend judging both Zig's and Rust's respective communities for how they are in their own social spaces (chatrooms, meetups, conferences) and for what they achieve on a concrete level (ie quality of the software they make), more than for what reaches the frontpage of HN with the brand name in the title.
tobyhinloopen 4y ago
Where did Zig come from? Never did I hear about it and suddenly people talk about it. What happened
- unwind 4y ago
  It was first released in 2016 so it's not ultra new but it's getting more attention at least here.
- kristoff_it 4y ago
  If you want an overview of the language and the Zig Software Foundation: https://www.youtube.com/watch?v=Gv2I7qTux7g https://www.youtube.com/watch?v=Gv2I7qTux7g https://ziglang.org/zsf/ https://ziglang.org/zsf/
- fulafel 4y ago
  Portland, USA apparently (from info at https://twitter.com/andy_kelley https://twitter.com/andy_kelley).
- thejosh 4y ago
  Popular language of the moment. Rust, Go, Elixir. :) Next few months will be "built X with Zig" with hundreds of upvotes.
  - SoraNoTenshi 4y ago
    tbh, i don't even think that promotion a product based on their language is a selling point (unless it's a lib of course!) But this whole `built with X` is in my opinion just some 'i try to appear cool by using x language' and i don't really like it. Don't get me wrong, i absolutely love rust, zig and go, but i believe, that you shouldn't pick a technology based on such subjective matter. A language is just a tool to solve problems, and you don't really want to screw in a screw with a hammer, do you?
  - tobyhinloopen 4y ago
    Zig does look great
grogers 4y ago
Aren't pages in a buffer pool usually in a different representation than the on-disk format?

158 points by gavinray 4y ago

Aha! I was thinking about trying this exact same approach recently, so I'm glad to see I wasn't insane! Very clever. I have been "in my spare time" (that is, never finishing) building out a page buffer mgmt system based on the one in the Umbra paper (see https://db.in.tum.de/~freitag/papers/p29-neumann-cidr20.pdf https://db.in.tum.de/~freitag/papers/p29-neumann-cidr20.pdf) which has a kind of novel approach where they essentially allocate the same physical address at multiple virtual addresses but with different block/slab sizes so that they can reduce fragmentation and still support dynamic page sizes. Anyways, it seems this author is also on the same "page" (sorry) as me but focusing more on exploiting `io_uring`. Nifty. I'm working in Rust, not Zig, but a lot of nice stuff to read through here. I will have to read through this in more detail later when I have more brain-space available.

gavinray 4y ago
Jinx, I'm also working on this, as a tutorial series! I had the pleasure of hearing Neumann present the Umbra architecture on the CMU DB series webinar recently. I'm slowly working through re-implementing the Coroutines async buffer pool paper. Blocked on trying to properly implement and understand "Morsel-Driven Parallelism": https://db.in.tum.de/~fent/papers/coroutines.pdf https://db.in.tum.de/~fent/papers/coroutines.pdf I have the async, io_uring buffer pool written in Kotlin using it's native Coroutines but the scheduling and morsel stuff I'm still trying to understand
- cmrdporcupine 4y ago
  I watched the same CMU lecture, very cool. Are you getting paid to work on this kind of thing, anywhere? Do you want to be?
  - gavinray 4y ago
    > Are you getting paid to work on this kind of thing, anywhere? My dayjob is a fair bit higher level. I work on a sort of query engine on the JVM that translates an internal query IR/AST into dialect-specific SQL + other languages so you can query anything with the same frontend language: https://techcrunch.com/2022/06/28/hasura-now-lets-developers-turn-any-data-source-into-a-graphql-api/ https://techcrunch.com/2022/06/28/hasura-now-lets-developers... I think it'd be neato to work on a database, but my dayjob is also pretty interesting to me.
gavinray 4y ago
> I'm working in Rust, not Zig, but a lot of nice stuff to read through here. FYI, Datadog has a Rust library for scheduling things to run thread-per-core with io_uring It'd be really useful for DB use cases: https://github.com/DataDog/glommio https://github.com/DataDog/glommio
- cmrdporcupine 4y ago
  Nifty. I started from the part of the Umbra presentation that initially struck me as most interesting -- the allocator portion (where he talks about repeatedly MMAP_ANONYMOUS on the same address range to reduce fragmentation, etc.) I stopped working when I started pondering the bits about pointer swizzling and how I'd make that work nicely in Rust, and never got as far as the pieces that would actually do the syncs to disk. I also came across another fellow who was inspired by the optimistic locking btree presented in LeanStore Umbra: https://github.com/NeowayLabs/bplustree https://github.com/NeowayLabs/bplustree

gavinray 4y ago

One note: I mistakenly thought that fixed buffers were part of the 5.19 changes for provided buffers Jens Axboe corrected me on this, fixed buffers have existed for a while apparently! https://twitter.com/axboe/status/1581392910687567873?t=gx_tQCEwL0Kh_OfO1QEBow&s=19 https://twitter.com/axboe/status/1581392910687567873?t=gx_tQ...

andrewedstrom 4y ago

e12e 4y ago
> I’ve never heard of a company actually using it Depending on your definition of "company" and "actually using it", https://oven.sh https://oven.sh might qualify?
ksec 4y ago
People say that, yet it is not anywhere near Rust. But yes, I do wish people could now keep Zig submission under control. I dont want Zig community becomes another Rust community. Zig https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=Zig&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu... Rust https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=Rust&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
- unwind 4y ago
  I'm in neither community, so what does that even mean? Why?
- kristoff_it 4y ago
  > I do wish people could now keep Zig submission under control. I dont want Zig community becomes another Rust community. I don't think anybody has control over what gets posted on HN or not. People act independently and to whatever fancies them. Should the author of this blog post have avoided using Zig to keep the frontpage of HN more balanced? Obviously no. Maybe the curators of HN could prevent Zig posts from reaching the front page but they too have their own motives (eg having the frontpage reflect the topics of interest of the collective HN readership) and it might just be that these are Zig's 15mins of fame. So I don't think your request can be fulfilled in practice and I don't think that Zig's popularity (or lack thereof) on HN will impact our community. I would recommend judging both Zig's and Rust's respective communities for how they are in their own social spaces (chatrooms, meetups, conferences) and for what they achieve on a concrete level (ie quality of the software they make), more than for what reaches the frontpage of HN with the brand name in the title.

tobyhinloopen 4y ago

Where did Zig come from? Never did I hear about it and suddenly people talk about it. What happened

unwind 4y ago
It was first released in 2016 so it's not ultra new but it's getting more attention at least here.
kristoff_it 4y ago
If you want an overview of the language and the Zig Software Foundation: https://www.youtube.com/watch?v=Gv2I7qTux7g https://www.youtube.com/watch?v=Gv2I7qTux7g https://ziglang.org/zsf/ https://ziglang.org/zsf/
fulafel 4y ago
Portland, USA apparently (from info at https://twitter.com/andy_kelley https://twitter.com/andy_kelley).
thejosh 4y ago
Popular language of the moment. Rust, Go, Elixir. :) Next few months will be "built X with Zig" with hundreds of upvotes.
- SoraNoTenshi 4y ago
  tbh, i don't even think that promotion a product based on their language is a selling point (unless it's a lib of course!) But this whole `built with X` is in my opinion just some 'i try to appear cool by using x language' and i don't really like it. Don't get me wrong, i absolutely love rust, zig and go, but i believe, that you shouldn't pick a technology based on such subjective matter. A language is just a tool to solve problems, and you don't really want to screw in a screw with a hammer, do you?
- tobyhinloopen 4y ago
  Zig does look great

grogers 4y ago

Aren't pages in a buffer pool usually in a different representation than the on-disk format?