4 ms·

Out of curiosity, how many databases have you written? This is co-authored by Pavlo, Viktor Leiss, with feedback from Neumann. I'm sorry, but if someone on the

Out of curiosity, how many databases have you written? This is co-authored by Pavlo, Viktor Leiss, with feedback from Neumann. I'm sorry, but if someone on the internet claims to know better than those 3, you're going to need some monumental evidence of your credibility. Additionally, what you link here: > ... (See the slide at 21:25 into the video). Modern DBMSs spend 96% of their time managing buffers and locks, and only 4% doing actual useful work for the caller. Is discussing "Main Memory" databases. These databases do no I/O outside of potential initial reads, because all of the data fits in-memory! These databases represent a small portion of contemporary DBMS usage when compared to traditional RDBMS. All you have to do is look at the bandwidth and reads/sec from the paper when using O_DIRECT "pread()"s versus mmap'ed IO.

LAC-Tech 3y ago
This is a classic appeal to authority. Let's play the argument, not the man. (My understanding is that the GP wrote LMDB, works on openLDAP, and was a maintainer for BerkelyDB for a number of years. But even if he'd only written 'hello, world!' I'm much more interested in the specific arguments).
- hyc_symas 3y ago
  Correct, and thank you. I wrote LMDB, wrote a lot of OpenLDAP, and worked on BerkeleyDB for many years. And actually Andy Pavlo invited me to CMU to give a lecture on LMDB a few years back. https://www.youtube.com/watch?v=tEa5sAh-kVk https://www.youtube.com/watch?v=tEa5sAh-kVk Andy and I have had this debate going for a long time already.
  - gavinray 3y ago
    Well, I eat my shorts. Isn't LMBD closer to an embedded key-value store than an RDBMS, though? Also there's a section in the paper that mentions it's single-writer.
    - hyc_symas 3y ago
      Yes, LMDB is an embedded key/value store but it can be used as the backing store of any other DB model you care for. E.g. as a backend to MySQL, or SQLite, or OpenLDAP, or whatever.
- gavinray 3y ago
  The argument is that: - Queries can trigger blocking page faults when accessing (transparently) evicted pages, causing unexpected I/O stalls - mmap() complicates transactionality and error-handling - Page table contention, single-threaded page eviction, and TLB shootdowns become bottlenecks
  - hyc_symas 3y ago
    1 - for reading any uncached data, the I/O stalls are unavoidable. Whatever client requested that data is going to have to wait regardless. 2 - complexity? this is simply false. LMDB's ACID txns using MVCC are much simpler than any "traditional" approach. 3 - contention is a red herring since this approach is already single-writer, as is common for most embedded k/v stores these days. You lose more perf by trying to make the write path multi-threaded, in lock contention and cache thrashing.
    - pclmulqdq 3y ago
      It's kind of disingenuous to talk about how great your concurrency system is when you only allow a single writer. RCU (which I imagine your system is isomorphic to) is pretty simple compared to what many DB engines use to do ACID transactions that involve both reads and writes.
      - hyc_symas 3y ago
        You don't need more than single-writer concurrency if your write txns are fast enough. Our experience with OpenLDAP was that multi-writer concurrency cost too much overhead. Even though you may be writing primary records to independent regions of the DB, if you're indexing any of that data (which all real DBs do, for query perf) you wind up getting a lot of contention in the indices. That leads to row locking conflicts, txn rollbacks, and retries. With a single writer txn model, you never get conflicts, never need rollbacks.
        pclmulqdq 3y ago
        That's probably because your OpenLDAP benchmarks used a tiny database. If you have multi-terabyte databases, you will start to see huge gains from a multi-writer setup because you will be regularly be loading pages from disk, rather than keeping almost all of your btree/LSM tree in RAM.
- jemfinch 3y ago
  "Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers" is already an appeal to authority in itself. We shouldn't apply a higher bar to the counterargument than we applied to the argument in the first place.
- jandrewrogers 3y ago
  I think the real argument is more nuanced. Where you see mmap() fail badly on Linux, even for read-only workloads, is under a few specific conditions: very large storage volumes, highly concurrent access, non-trivial access patterns (e.g. high-dimensionality access methods). Most people do not operate data models under these conditions, but if you do then you can achieve large integer factor gains in throughput by not using mmap(). Interestingly, most of the reason for these problems has to do with theoretical limitations of cache replacement algorithms as drivers of I/O scheduling. There are alternative approaches to scheduling I/O that work much better in these cases but mmap() can’t express them, so in those cases bypassing mmap() offers large gains.
  - pclmulqdq 3y ago
    GP wrote a key-value store called LMDB that is constrained to a single writer, and often used for small databases that fit entirely in memory but need to persist to disk. There's a whole different world for more scalable databases.
    - hyc_symas 3y ago
      "fit entirely in memory" is not a requirement. LMDB is not a main-memory database, it is an on-disk database that uses memory mapping.
  - LAC-Tech 3y ago
    Can you explain "high-dimensionality access methods" to me? (Or if it's too big for an HN comment, maybe recommend a paper).
    - heisjustsosmart 3y ago
      This guy talks a lot of crap. See his website for examples, and don't waste your time with him <<<There is one significant drawback that should not be understated. Algorithm design using topology manipulation can be enormously challenging to reason about. You are often taking a conceptually simple algorithm, like a nested loop or hash join, and replacing it with a much more efficient algorithm involving the non-trivial manipulation of complex high-dimensionality constraint spaces that effect the same result. Routinely reasoning about complex object relationships in greater than three dimensions, and constructing correct parallel algorithms that exploit them, becomes easier but never easy.>>> http://www.jandrewrogers.com/2015/10/08/spacecurve/ http://www.jandrewrogers.com/2015/10/08/spacecurve/
ilyt 3y ago
Out of curiosity, do you have anything actually useful to add or are just throwing appeals to authority because you don't ?
Mikhail_Edoshin 3y ago
Even thought the data resides mostly in-memory they still have to write transactions to disk to preserve them, don't they?

LAC-Tech 3y ago

This is a classic appeal to authority. Let's play the argument, not the man. (My understanding is that the GP wrote LMDB, works on openLDAP, and was a maintainer for BerkelyDB for a number of years. But even if he'd only written 'hello, world!' I'm much more interested in the specific arguments).

hyc_symas 3y ago
Correct, and thank you. I wrote LMDB, wrote a lot of OpenLDAP, and worked on BerkeleyDB for many years. And actually Andy Pavlo invited me to CMU to give a lecture on LMDB a few years back. https://www.youtube.com/watch?v=tEa5sAh-kVk https://www.youtube.com/watch?v=tEa5sAh-kVk Andy and I have had this debate going for a long time already.
- gavinray 3y ago
  Well, I eat my shorts. Isn't LMBD closer to an embedded key-value store than an RDBMS, though? Also there's a section in the paper that mentions it's single-writer.
  - hyc_symas 3y ago
    Yes, LMDB is an embedded key/value store but it can be used as the backing store of any other DB model you care for. E.g. as a backend to MySQL, or SQLite, or OpenLDAP, or whatever.
gavinray 3y ago
The argument is that: - Queries can trigger blocking page faults when accessing (transparently) evicted pages, causing unexpected I/O stalls - mmap() complicates transactionality and error-handling - Page table contention, single-threaded page eviction, and TLB shootdowns become bottlenecks
- hyc_symas 3y ago
  1 - for reading any uncached data, the I/O stalls are unavoidable. Whatever client requested that data is going to have to wait regardless. 2 - complexity? this is simply false. LMDB's ACID txns using MVCC are much simpler than any "traditional" approach. 3 - contention is a red herring since this approach is already single-writer, as is common for most embedded k/v stores these days. You lose more perf by trying to make the write path multi-threaded, in lock contention and cache thrashing.
  - pclmulqdq 3y ago
    It's kind of disingenuous to talk about how great your concurrency system is when you only allow a single writer. RCU (which I imagine your system is isomorphic to) is pretty simple compared to what many DB engines use to do ACID transactions that involve both reads and writes.
    - hyc_symas 3y ago
      You don't need more than single-writer concurrency if your write txns are fast enough. Our experience with OpenLDAP was that multi-writer concurrency cost too much overhead. Even though you may be writing primary records to independent regions of the DB, if you're indexing any of that data (which all real DBs do, for query perf) you wind up getting a lot of contention in the indices. That leads to row locking conflicts, txn rollbacks, and retries. With a single writer txn model, you never get conflicts, never need rollbacks.
      - pclmulqdq 3y ago
        That's probably because your OpenLDAP benchmarks used a tiny database. If you have multi-terabyte databases, you will start to see huge gains from a multi-writer setup because you will be regularly be loading pages from disk, rather than keeping almost all of your btree/LSM tree in RAM.
jemfinch 3y ago
"Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers" is already an appeal to authority in itself. We shouldn't apply a higher bar to the counterargument than we applied to the argument in the first place.
jandrewrogers 3y ago
I think the real argument is more nuanced. Where you see mmap() fail badly on Linux, even for read-only workloads, is under a few specific conditions: very large storage volumes, highly concurrent access, non-trivial access patterns (e.g. high-dimensionality access methods). Most people do not operate data models under these conditions, but if you do then you can achieve large integer factor gains in throughput by not using mmap(). Interestingly, most of the reason for these problems has to do with theoretical limitations of cache replacement algorithms as drivers of I/O scheduling. There are alternative approaches to scheduling I/O that work much better in these cases but mmap() can’t express them, so in those cases bypassing mmap() offers large gains.
- pclmulqdq 3y ago
  GP wrote a key-value store called LMDB that is constrained to a single writer, and often used for small databases that fit entirely in memory but need to persist to disk. There's a whole different world for more scalable databases.
  - hyc_symas 3y ago
    "fit entirely in memory" is not a requirement. LMDB is not a main-memory database, it is an on-disk database that uses memory mapping.
- LAC-Tech 3y ago
  Can you explain "high-dimensionality access methods" to me? (Or if it's too big for an HN comment, maybe recommend a paper).
  - heisjustsosmart 3y ago
    This guy talks a lot of crap. See his website for examples, and don't waste your time with him <<<There is one significant drawback that should not be understated. Algorithm design using topology manipulation can be enormously challenging to reason about. You are often taking a conceptually simple algorithm, like a nested loop or hash join, and replacing it with a much more efficient algorithm involving the non-trivial manipulation of complex high-dimensionality constraint spaces that effect the same result. Routinely reasoning about complex object relationships in greater than three dimensions, and constructing correct parallel algorithms that exploit them, becomes easier but never easy.>>> http://www.jandrewrogers.com/2015/10/08/spacecurve/ http://www.jandrewrogers.com/2015/10/08/spacecurve/

ilyt 3y ago

Out of curiosity, do you have anything actually useful to add or are just throwing appeals to authority because you don't ?

Mikhail_Edoshin 3y ago

Even thought the data resides mostly in-memory they still have to write transactions to disk to preserve them, don't they?