4 ms·

RE ScyllaDB, there will absolutely be no fork and it’s very unlikely they have ever had a meaningful contribution to Scylla OSS (which is not changing, just goi

RE ScyllaDB, there will absolutely be no fork and it’s very unlikely they have ever had a meaningful contribution to Scylla OSS (which is not changing, just going to bit rot and the enterprise version which was closed source was moved to source available). The reasoning being the bar is very high for contributions. It’s C++ 20/23 without virtual memory and a userland cooperative thread per core scheduler (this is the underlying seastar framework). The skills needed to do anything meaningful here is very high and I don’t see the motivation behind it as I would expect people having feature requests would be customers, have an easier time extending Cassandra or the feature would be available in the Enterprise version.

menaerus 2y ago
> It’s C++ 20/23 without virtual memory Not sure what you mean by this? Virtual memory is implied by the CPU MMU and consequently OS kernel. Perhaps you meant they use a lot of custom memory allocation schemes? Otherwise, I agree that the bar is quite high since (1) the problem at hand is already too complex (scalable LSM), and (2) pretty much anything in the code is custom made, e.g. avoiding the OS kernel as much as possible. And they pay peanuts for the skills needed to do the job.
- mike_hearn 2y ago
  Seastar runs everything in kernel mode.
  - menaerus 2y ago
    Makes no sense. Source?
    - rockwotj 2y ago
      I mean memory is completely allocated up front and fragmentation is something for application developers to deal with.
      - menaerus 2y ago
        I occasionally read their blogs, and I haven't looked into the source for a while, but application developers having to deal with the OOM because one is using custom memory allocation policy that doesn't deal with or suffers from fragmention would be a strange design choice. jmalloc IME works really well.
  - rockwotj 2y ago
    It tries to bypass the kernel as much as possible. It runs in userspace.
    - mike_hearn 2y ago
      I'm probably thinking of when it's combined with OSv or something. It's been years since I looked at this.

menaerus 2y ago

> It’s C++ 20/23 without virtual memory Not sure what you mean by this? Virtual memory is implied by the CPU MMU and consequently OS kernel. Perhaps you meant they use a lot of custom memory allocation schemes? Otherwise, I agree that the bar is quite high since (1) the problem at hand is already too complex (scalable LSM), and (2) pretty much anything in the code is custom made, e.g. avoiding the OS kernel as much as possible. And they pay peanuts for the skills needed to do the job.

mike_hearn 2y ago
Seastar runs everything in kernel mode.
- menaerus 2y ago
  Makes no sense. Source?
  - rockwotj 2y ago
    I mean memory is completely allocated up front and fragmentation is something for application developers to deal with.
    - menaerus 2y ago
      I occasionally read their blogs, and I haven't looked into the source for a while, but application developers having to deal with the OOM because one is using custom memory allocation policy that doesn't deal with or suffers from fragmention would be a strange design choice. jmalloc IME works really well.
- rockwotj 2y ago
  It tries to bypass the kernel as much as possible. It runs in userspace.
  - mike_hearn 2y ago
    I'm probably thinking of when it's combined with OSv or something. It's been years since I looked at this.