3 ms·

Compression libraries, OpenSSL, ICU, etc. are all common dependencies for databases. Looking at the dependencies list (https://gist.github.com/tisonkun/06550d2

Compression libraries, OpenSSL, ICU, etc. are all common dependencies for databases. Looking at the dependencies list (https://gist.github.com/tisonkun/06550d2dcd9cf6551887ee6305e92edc https://gist.github.com/tisonkun/06550d2dcd9cf6551887ee6305e...) I see plenty of reasonable things like: * Base64/checksum/compression encoding libraries * Encryption/hash libraries * Platform-specific bindings (likely conditional dependencies) * Bit hacking/casting/zero-copy libraries like bytemuck, zerocopy, zero-vec, etc. * "Small"/stack allocated data structure libraries (smallvec, tinystr, etc.) * Unicode libraries There are certainly things that would add bloat too, but I think it's silly to pretend like everything here is something a database engine would need custom implementations of.

jandrewrogers 2y ago
I think you'd be surprised how many of these things are custom implementations in databases. The main motivation is performance. Databases tend to have detailed and well-specified constraints on each use case for data structures and algorithms that can be used to codegen narrowly optimized implementations. You can do significantly better than generic library codecs or data structures in most cases, those implementations lack the context and metaprogramming hooks to make it feasible. Combine this with the challenge of implementations being async, non-allocating, compatible with explicitly paged memory, etc and it generally becomes worth the effort. You'll find more libraries used at the periphery for integration and compatibility where it matters less but not in the core.
- 0x457 2y ago
  Pretty sure it's not due to performance, but due to age of most database code bases and in some cases licensing. How annoying it is to have dependencies in C and C++ also probably a contributing factor. I'd rather an author pulls in a tinyvec/serde than tries to make a bespoke implementation.
  - jandrewrogers 2y ago
    You are entitled to your take, but it is a demonstration that you’ve never been anywhere near a modern database engine code base. You can’t defend your opinion. It is what people with no experience would say. That’s fine, I have not objection to hot takes, but don’t conflate your limited experience with reality.
    - 0x457 2y ago
      Never claimed that I was an expert or anything. I interpolated my opinion from my experience with large C code bases. I get it that you work on a geospatial database and understand this field better than I do, but why so rude and dismissive? Work on that.

jandrewrogers 2y ago

I think you'd be surprised how many of these things are custom implementations in databases. The main motivation is performance. Databases tend to have detailed and well-specified constraints on each use case for data structures and algorithms that can be used to codegen narrowly optimized implementations. You can do significantly better than generic library codecs or data structures in most cases, those implementations lack the context and metaprogramming hooks to make it feasible. Combine this with the challenge of implementations being async, non-allocating, compatible with explicitly paged memory, etc and it generally becomes worth the effort. You'll find more libraries used at the periphery for integration and compatibility where it matters less but not in the core.

0x457 2y ago
Pretty sure it's not due to performance, but due to age of most database code bases and in some cases licensing. How annoying it is to have dependencies in C and C++ also probably a contributing factor. I'd rather an author pulls in a tinyvec/serde than tries to make a bespoke implementation.
- jandrewrogers 2y ago
  You are entitled to your take, but it is a demonstration that you’ve never been anywhere near a modern database engine code base. You can’t defend your opinion. It is what people with no experience would say. That’s fine, I have not objection to hot takes, but don’t conflate your limited experience with reality.
  - 0x457 2y ago
    Never claimed that I was an expert or anything. I interpolated my opinion from my experience with large C code bases. I get it that you work on a geospatial database and understand this field better than I do, but why so rude and dismissive? Work on that.