4 ms·

It's particularly flawed given: a) IO is such a large portion of the problem b) Hypertable isn't just way, way faster.

by cbsmith 11y ago

acconsta 11y ago
Are there any recent comparative benchmarks of hypertable? I looked around but couldn't find any.
- cbsmith 11y ago
  Not that I know of. Anecdotally though, it has some advantages, but doesn't exactly crush the competition.
_Codemonkeyism 11y ago
"a) IO is such a large portion of the problem" I'm very interested in a cluster benchmark therefor, say 10 servers, as Cassandra claims to scale very well. With a cluster IO has a higher performance impact than one server with local RAID IO.
- cbsmith 11y ago
  The way ScyllaDB works, it'd be really weird if it didn't scale at least as well as Cassandra.
glommer 11y ago
IO is not only a large part. It is the main part. That is why it is important to get it right : scylla for instance does not leave the cache to the OS. It has its own caches for everything. Never blocks on IO or page faults because all IO bypasses the kernel. And those are just two tiny examples.
- cbsmith 11y ago
  > scylla for instance does not leave the cache to the OS. It has its own caches for everything Uh-huh... that's all pretty common for databases. Cassandra would fit that description. > Never blocks on IO or page faults because all IO bypasses the kernel. That just seems nonsensical. Sometimes, you are waiting for IO. That's just reality. It is conceivable you bypass the kernel for I/O, but that creates a lot of complexity and limitations. Near as I can tell though, they do talk to the kernel for IO.
  - vitalyd 11y ago
    http://www.scylladb.com/technology/memory/ http://www.scylladb.com/technology/memory/ By the way, I think you're replying to one of the devs of Scylla.
    - cbsmith 11y ago
      I'm sure I am.
    - cbsmith 11y ago
      So, in general, I understand there is lots of stuff going on in Scylla that does distinguish it, at least from Cassandra. There is the user space networking logic for IO. However, a lot of the IO overhead with disk, for example.
      - vitalyd 11y ago
        >However, a lot of the IO overhead with disk, for example. That's why they benchmarked this workload on a 4x SSD RAID configuration :). Given that i/o bandwidth and throughput continues to increase, processor frequency isn't, and core counts are going up, it's prudent to design a system that can take advantage of this.
        cbsmith 11y ago
        Yeah, and a 4x SSD RAID configuration is kind of overkill in the extreme for most Cassandra set ups. I'm sure there is a way to set up IO subsystems so that Cassandra becomes a huge bottleneck, but that's a pretty specialized context.
  - glommer 11y ago
    Cassandra has a row cache that it does not necessarily use. Most of its data sits in the linux page cache. Because the SSTables are mmaped into memory, you can't get rid of that: even if you do use the row cache, you would be at best using twice as much cache memory. Scylla never touches the page cache. All IO in seastar is direct IO and then scylla caches everything itself. We always know when the disk access is going to happen. The OS paging mechanism does not do a thing. As for waiting for IO, of course IO does not complete immediately. But you can either block and wait for it, as Cassandra does (it doesn't even have the option not to in the case of the mmaped regions) or you can do something fully async like seastar that guarantees you never block waiting for IO.
- cbsmith 11y ago
  Even on the networking side, you can see from projects like this that you can get what should be enough messaging performance for any NoSQL store out of Java: https://github.com/real-logic/Aeron https://github.com/real-logic/Aeron Their Java throughput is about 70% of their C++11 throughput, and that's with a pretty synthetic benchmark where there is not any logic behind those messages. Once you add in some real logic there, it gets even thinner. They aren't doing user space networking, but that actually ought to allow Java to do even better.

It's particularly flawed given: a) IO is such a large portion of the problem b) Hypertable isn't just way, way faster.

by cbsmith 11y ago

It's particularly flawed given: a) IO is such a large portion of the problem b) Hypertable isn't just way, way faster.

acconsta 11y ago

Are there any recent comparative benchmarks of hypertable? I looked around but couldn't find any.

cbsmith 11y ago
Not that I know of. Anecdotally though, it has some advantages, but doesn't exactly crush the competition.

_Codemonkeyism 11y ago

"a) IO is such a large portion of the problem" I'm very interested in a cluster benchmark therefor, say 10 servers, as Cassandra claims to scale very well. With a cluster IO has a higher performance impact than one server with local RAID IO.

cbsmith 11y ago
The way ScyllaDB works, it'd be really weird if it didn't scale at least as well as Cassandra.

glommer 11y ago

IO is not only a large part. It is the main part. That is why it is important to get it right : scylla for instance does not leave the cache to the OS. It has its own caches for everything. Never blocks on IO or page faults because all IO bypasses the kernel. And those are just two tiny examples.

cbsmith 11y ago
> scylla for instance does not leave the cache to the OS. It has its own caches for everything Uh-huh... that's all pretty common for databases. Cassandra would fit that description. > Never blocks on IO or page faults because all IO bypasses the kernel. That just seems nonsensical. Sometimes, you are waiting for IO. That's just reality. It is conceivable you bypass the kernel for I/O, but that creates a lot of complexity and limitations. Near as I can tell though, they do talk to the kernel for IO.
- vitalyd 11y ago
  http://www.scylladb.com/technology/memory/ http://www.scylladb.com/technology/memory/ By the way, I think you're replying to one of the devs of Scylla.
  - cbsmith 11y ago
    I'm sure I am.
  - cbsmith 11y ago
    So, in general, I understand there is lots of stuff going on in Scylla that does distinguish it, at least from Cassandra. There is the user space networking logic for IO. However, a lot of the IO overhead with disk, for example.
    - vitalyd 11y ago
      >However, a lot of the IO overhead with disk, for example. That's why they benchmarked this workload on a 4x SSD RAID configuration :). Given that i/o bandwidth and throughput continues to increase, processor frequency isn't, and core counts are going up, it's prudent to design a system that can take advantage of this.
      - cbsmith 11y ago
        Yeah, and a 4x SSD RAID configuration is kind of overkill in the extreme for most Cassandra set ups. I'm sure there is a way to set up IO subsystems so that Cassandra becomes a huge bottleneck, but that's a pretty specialized context.
- glommer 11y ago
  Cassandra has a row cache that it does not necessarily use. Most of its data sits in the linux page cache. Because the SSTables are mmaped into memory, you can't get rid of that: even if you do use the row cache, you would be at best using twice as much cache memory. Scylla never touches the page cache. All IO in seastar is direct IO and then scylla caches everything itself. We always know when the disk access is going to happen. The OS paging mechanism does not do a thing. As for waiting for IO, of course IO does not complete immediately. But you can either block and wait for it, as Cassandra does (it doesn't even have the option not to in the case of the mmaped regions) or you can do something fully async like seastar that guarantees you never block waiting for IO.
cbsmith 11y ago
Even on the networking side, you can see from projects like this that you can get what should be enough messaging performance for any NoSQL store out of Java: https://github.com/real-logic/Aeron https://github.com/real-logic/Aeron Their Java throughput is about 70% of their C++11 throughput, and that's with a pretty synthetic benchmark where there is not any logic behind those messages. Once you add in some real logic there, it gets even thinner. They aren't doing user space networking, but that actually ought to allow Java to do even better.