9 ms·

Scylla – Real-Time Big Data Database

96 points by andrewstuart 5y ago

actually_a_dog 5y ago
Whenever I hear Scylla mentioned, I often wonder: where is Charybdis? They do work as a team, after all. :-)
- PeterCorless 5y ago
  Charybdefs is our fault injection filesystem. Though it's based in Thrift, the ol' API that predated CQL. https://github.com/scylladb/charybdefs https://github.com/scylladb/charybdefs More modern is Project Circe, our efforts to make Scylla into an even more monstrous database: https://www.scylladb.com/2021/01/12/making-scylla-a-monstrous-database-introducing-project-circe/ https://www.scylladb.com/2021/01/12/making-scylla-a-monstrou...
DSingularity 5y ago
Yeah not surprised. They squeeze all the performance out of a single node using the seastar framework.
jiggawatts 5y ago
(Just in case some Scylla employees see this YC News post) Sigh... I know this sounds like a nitpick, and I know it sounds like a broken record, and I know you probably work in a different team and there's "nothing" you can do about it... But. I went to your website, interested in your product. I'm your target market! This site needs to impress me, and people like me. What's the first thing that I see? The website oh-so-slowly animates, sliding down to show some TechCrunch ad. I don't care about TechCrunch. I care about high-performance databases. But okay, I move to close it, but "No!" says your website, helpfully overlapping it with another animated slider asking me to accept your cookie policy so that I can be tracked by your marketing group. Fine. I close both popups, and try to read the content of your site despite the animations every paragraph or so trying to distract me from the content. As soon as I scroll too far past the animations, a stupid chat bot pops up to overlay the bottom of the content as well. I figure I'll just go to the meat of it, some whitepaper or technical documentation. Despite their miriad flaws, PDFs are thankfully not commonly animated. The download link for your benchmarks asks for my contact details. It's not a link. It's a sign-up form for spam. I'm not an idiot. I don't want spam. I want to read about your database. Your marketing team actively stops your target market from looking at your products. Perhaps that's an issue you should look into, because right now there are potential customers that simply never get to find out just how amazing your technology is, because their first experience of your products makes used car salesmen look upstanding and trustworthy.
- PeterCorless 5y ago
  Peter Corless here from ScyllaDB's marketing team. Your feedback is 100% heard, understood & appreciated. I'll share it with the team. The good news is that we just provided a new benchmark today and it is 100% ungated. Scylla 4.4 vs Cassandra 4.0: https://www.scylladb.com/2021/08/24/apache-cassandra-4-0-vs-scylla-4-4-comparing-performance/ https://www.scylladb.com/2021/08/24/apache-cassandra-4-0-vs-... Also, here are a few other free and open blogs where we have benchmarked vs. competitors: Scylla vs. DynamoDB: https://www.scylladb.com/2018/12/13/scylla-vs-amazon-dynamodb/ https://www.scylladb.com/2018/12/13/scylla-vs-amazon-dynamod... Scylla vs. Google Cloud Bigtable: https://www.scylladb.com/2019/05/02/going-head-to-head-scylla-cloud-vs-google-cloud-bigtable/ https://www.scylladb.com/2019/05/02/going-head-to-head-scyll...
  - jiggawatts 5y ago
    To also provide some constructive feedback: A good point of comparison is this website: https://www.jetbrains.com/idea/ https://www.jetbrains.com/idea/ It's also a product targetted at technical people, including people working at startups and also large enterprise. No popups. When you click the community edition download link -- bam -- it is immediately downloading! No form to fill in. I don't even use their tools much any more, except for the Rust plugin. Nonetheless, I read through their "what's new" release notes, even for Java, because it is so well presented: https://www.jetbrains.com/idea/whatsnew/ https://www.jetbrains.com/idea/whatsnew/ I don't have to sit through an hour-long YouTube video watching some guy introducing some other guy I don't care about for five minutes. https://knowyourmeme.com/memes/the-wadsworth-constant https://knowyourmeme.com/memes/the-wadsworth-constant Instead they have GIFs showing short, to-the-point snippets of exactly what each feature does. Note that these don't animate by default! You have to click them to play the clip. I love that. They're helpful without being distracting while I'm reading nearby text. It also gives you an idea of what the product looks like in actual use. You have no idea how much crap people wade through to find this! I often spend hours googling terms like "Product X Screenshot", "Product X real-world", "Product X tutorial" in the futile attempt to just find out what the heck to expect. Is it a green-screen terminal app kept alive like some sort of crime against nature? Is it a web application? Does it come with a Windows-only GUI? If so, is it at least a usable one? Does it have command-line tools? PowerShell? Tab-complete? You go to a site like JetBrains, and you see exactly that! Real-world code being manipulated, showing you the product in all its glory. So show this! Show ScyllaDB doing something. Don't just talk about how it's 47% more snazzy than a competing product I haven't used. Show it doing a schema change nearly instantly on a terabyte of data, or whatever. But show me the product, or I walk away until I find a website that isn't afraid of letting me see what they're selling...
Notanothertoo 5y ago
Been using it since 2017 for IoT time series data. Works great, very fast. I think the one criticism is that it seems the company has some intentional rough edges on it, in favor of their SaaS over the open source release. It used to be around backing up and managing updates long term ect. They have gotten better but their helm chart for example is very opinionated and uses custom resources in place of say a statefulset. I think their SaaS is overpriced especially compared to reserved instances but that's just me. Also from my calls with support, they lack direction internally. One time our sales rep and customer success director argued on the phone in front of us if they could bill us for some migration work. We silently let them finish and then said we expect it to be included in our support package based on their argument.
- biggestdummy 5y ago
  So an open exchange of ideas, and some free assistance. Sounds like it was a good outcome for you. :)
andrewstuart 5y ago
I posted this because I'm interested to hear from anyone using it - how has it worked out for you? I note it's written in C++ which is a bit of a surprise - I'd expected Rust or Golang. Interesting as well is is AGPL - licensing is always contentious: https://github.com/scylladb/scylla/blob/master/LICENSE.AGPL https://github.com/scylladb/scylla/blob/master/LICENSE.AGPL
- deleted 5y ago
  [deleted]
- zinclozenge 5y ago
  I think the main reason it's in C++ is because of its async executor, Seastar. There's a similar Rust project called Glommio but seems still very early.
  - throwaway81523 5y ago
    Seastar is sort of a C++-ification of node.js. Now that C++20 has coroutines, I wonder if those could have been used instead of all that chained method stuff.
    - enedil 5y ago
      Seastar already uses coroutines, however coroutines without Seastar reactor (and all the utilities for IO) are useless by themselves. You still need a way to schedule what's being done when.
      - throwaway81523 5y ago
        Hmm ok I haven't looked at Seastar in a while, but it used to depend on Node-like control inversion where you'd pass an explicit lambda to each action, telling the action what to do next. That meant unwinding the handler for a given event into a bunch of nested lambdas. Coroutine would let you write them in a more traditional sequential style, where you'd have a return to the scheduler whenever something could block. Yes you have to write a layer of async io under everything, but that's how any OS works, more or less.
criticaltinker 5y ago
The benchmarks against DynamoDB, Bigtable, & CockroachDB [1] appear quite impressive - anyone have real world experience that can attest to these claims of improved performance and reduced cost? > Scylla vs DynamoDB – Database Benchmark > 20x better throughput in the hot-partition test > Scylla Cloud is 1/7 the expense of DynamoDB when running equivalent workloads > Scylla Cloud: Average replication latency of 82ms. DynamoDB: Average latency of 370ms. > Scylla vs Bigtable – Database Benchmark > Scylla Cloud performs 26X better than Google Cloud Bigtable when applied with real-world, unoptimized data distribution > Google BigTable requires 10X as many nodes to accept the same workload as Scylla Cloud > Scylla Cloud was able to sustain 26x the throughput, and with read latencies 1/800th and write latencies less than 1/100th of Cloud Bigtable > Scylla vs CockroachDB – Database Benchmark > Loading 10x the data into Scylla took less than half the time it took for CockroachDB to load the much lesser dataset. > Scylla handled 10x the amount of data. > Scylla achieved 9.3x the throughput of CockroachDB at 1/4th the latency. [1] https://www.scylladb.com/product/benchmarks/ https://www.scylladb.com/product/benchmarks/
- PeterCorless 5y ago
  We just posted this today. Latest Cassandra 4.0 vs. Scylla 4.4. Note: Cassandra 4.0 is a HUGE improvement over Cassandra 3.11. But we're still many times faster: https://www.scylladb.com/2021/08/24/apache-cassandra-4-0-vs-scylla-4-4-comparing-performance/ https://www.scylladb.com/2021/08/24/apache-cassandra-4-0-vs-...
  - deleted 5y ago
    [deleted]
- manigandham 5y ago
  Yes, Scylla does what it says. Used it in a prior adtech company and it beat everything else at that time for a very intensive key/value workload that also needed multi-regional replication. The adtech industry also uses Aerospike heavily but that (at the time) had many replication data model issues compared to Scylla/Cassandra.
- steeve 5y ago
mianos 5y ago
It is interesting in that this is here on the front page and an old article about Discord moving to Cassandra is also here considering Discord went from Cassandra to Scylla I beleive.

96 points by andrewstuart 5y ago

actually_a_dog 5y ago

Whenever I hear Scylla mentioned, I often wonder: where is Charybdis? They do work as a team, after all. :-)

PeterCorless 5y ago
Charybdefs is our fault injection filesystem. Though it's based in Thrift, the ol' API that predated CQL. https://github.com/scylladb/charybdefs https://github.com/scylladb/charybdefs More modern is Project Circe, our efforts to make Scylla into an even more monstrous database: https://www.scylladb.com/2021/01/12/making-scylla-a-monstrous-database-introducing-project-circe/ https://www.scylladb.com/2021/01/12/making-scylla-a-monstrou...

DSingularity 5y ago

Yeah not surprised. They squeeze all the performance out of a single node using the seastar framework.

jiggawatts 5y ago

(Just in case some Scylla employees see this YC News post) Sigh... I know this sounds like a nitpick, and I know it sounds like a broken record, and I know you probably work in a different team and there's "nothing" you can do about it... But. I went to your website, interested in your product. I'm your target market! This site needs to impress me, and people like me. What's the first thing that I see? The website oh-so-slowly animates, sliding down to show some TechCrunch ad. I don't care about TechCrunch. I care about high-performance databases. But okay, I move to close it, but "No!" says your website, helpfully overlapping it with another animated slider asking me to accept your cookie policy so that I can be tracked by your marketing group. Fine. I close both popups, and try to read the content of your site despite the animations every paragraph or so trying to distract me from the content. As soon as I scroll too far past the animations, a stupid chat bot pops up to overlay the bottom of the content as well. I figure I'll just go to the meat of it, some whitepaper or technical documentation. Despite their miriad flaws, PDFs are thankfully not commonly animated. The download link for your benchmarks asks for my contact details. It's not a link. It's a sign-up form for spam. I'm not an idiot. I don't want spam. I want to read about your database. Your marketing team actively stops your target market from looking at your products. Perhaps that's an issue you should look into, because right now there are potential customers that simply never get to find out just how amazing your technology is, because their first experience of your products makes used car salesmen look upstanding and trustworthy.

PeterCorless 5y ago
Peter Corless here from ScyllaDB's marketing team. Your feedback is 100% heard, understood & appreciated. I'll share it with the team. The good news is that we just provided a new benchmark today and it is 100% ungated. Scylla 4.4 vs Cassandra 4.0: https://www.scylladb.com/2021/08/24/apache-cassandra-4-0-vs-scylla-4-4-comparing-performance/ https://www.scylladb.com/2021/08/24/apache-cassandra-4-0-vs-... Also, here are a few other free and open blogs where we have benchmarked vs. competitors: Scylla vs. DynamoDB: https://www.scylladb.com/2018/12/13/scylla-vs-amazon-dynamodb/ https://www.scylladb.com/2018/12/13/scylla-vs-amazon-dynamod... Scylla vs. Google Cloud Bigtable: https://www.scylladb.com/2019/05/02/going-head-to-head-scylla-cloud-vs-google-cloud-bigtable/ https://www.scylladb.com/2019/05/02/going-head-to-head-scyll...
- jiggawatts 5y ago
  To also provide some constructive feedback: A good point of comparison is this website: https://www.jetbrains.com/idea/ https://www.jetbrains.com/idea/ It's also a product targetted at technical people, including people working at startups and also large enterprise. No popups. When you click the community edition download link -- bam -- it is immediately downloading! No form to fill in. I don't even use their tools much any more, except for the Rust plugin. Nonetheless, I read through their "what's new" release notes, even for Java, because it is so well presented: https://www.jetbrains.com/idea/whatsnew/ https://www.jetbrains.com/idea/whatsnew/ I don't have to sit through an hour-long YouTube video watching some guy introducing some other guy I don't care about for five minutes. https://knowyourmeme.com/memes/the-wadsworth-constant https://knowyourmeme.com/memes/the-wadsworth-constant Instead they have GIFs showing short, to-the-point snippets of exactly what each feature does. Note that these don't animate by default! You have to click them to play the clip. I love that. They're helpful without being distracting while I'm reading nearby text. It also gives you an idea of what the product looks like in actual use. You have no idea how much crap people wade through to find this! I often spend hours googling terms like "Product X Screenshot", "Product X real-world", "Product X tutorial" in the futile attempt to just find out what the heck to expect. Is it a green-screen terminal app kept alive like some sort of crime against nature? Is it a web application? Does it come with a Windows-only GUI? If so, is it at least a usable one? Does it have command-line tools? PowerShell? Tab-complete? You go to a site like JetBrains, and you see exactly that! Real-world code being manipulated, showing you the product in all its glory. So show this! Show ScyllaDB doing something. Don't just talk about how it's 47% more snazzy than a competing product I haven't used. Show it doing a schema change nearly instantly on a terabyte of data, or whatever. But show me the product, or I walk away until I find a website that isn't afraid of letting me see what they're selling...

Notanothertoo 5y ago

Been using it since 2017 for IoT time series data. Works great, very fast. I think the one criticism is that it seems the company has some intentional rough edges on it, in favor of their SaaS over the open source release. It used to be around backing up and managing updates long term ect. They have gotten better but their helm chart for example is very opinionated and uses custom resources in place of say a statefulset. I think their SaaS is overpriced especially compared to reserved instances but that's just me. Also from my calls with support, they lack direction internally. One time our sales rep and customer success director argued on the phone in front of us if they could bill us for some migration work. We silently let them finish and then said we expect it to be included in our support package based on their argument.

biggestdummy 5y ago
So an open exchange of ideas, and some free assistance. Sounds like it was a good outcome for you. :)

andrewstuart 5y ago

I posted this because I'm interested to hear from anyone using it - how has it worked out for you? I note it's written in C++ which is a bit of a surprise - I'd expected Rust or Golang. Interesting as well is is AGPL - licensing is always contentious: https://github.com/scylladb/scylla/blob/master/LICENSE.AGPL https://github.com/scylladb/scylla/blob/master/LICENSE.AGPL

deleted 5y ago
[deleted]
zinclozenge 5y ago
I think the main reason it's in C++ is because of its async executor, Seastar. There's a similar Rust project called Glommio but seems still very early.
- throwaway81523 5y ago
  Seastar is sort of a C++-ification of node.js. Now that C++20 has coroutines, I wonder if those could have been used instead of all that chained method stuff.
  - enedil 5y ago
    Seastar already uses coroutines, however coroutines without Seastar reactor (and all the utilities for IO) are useless by themselves. You still need a way to schedule what's being done when.
    - throwaway81523 5y ago
      Hmm ok I haven't looked at Seastar in a while, but it used to depend on Node-like control inversion where you'd pass an explicit lambda to each action, telling the action what to do next. That meant unwinding the handler for a given event into a bunch of nested lambdas. Coroutine would let you write them in a more traditional sequential style, where you'd have a return to the scheduler whenever something could block. Yes you have to write a layer of async io under everything, but that's how any OS works, more or less.

criticaltinker 5y ago

The benchmarks against DynamoDB, Bigtable, & CockroachDB [1] appear quite impressive - anyone have real world experience that can attest to these claims of improved performance and reduced cost? > Scylla vs DynamoDB – Database Benchmark > 20x better throughput in the hot-partition test > Scylla Cloud is 1/7 the expense of DynamoDB when running equivalent workloads > Scylla Cloud: Average replication latency of 82ms. DynamoDB: Average latency of 370ms. > Scylla vs Bigtable – Database Benchmark > Scylla Cloud performs 26X better than Google Cloud Bigtable when applied with real-world, unoptimized data distribution > Google BigTable requires 10X as many nodes to accept the same workload as Scylla Cloud > Scylla Cloud was able to sustain 26x the throughput, and with read latencies 1/800th and write latencies less than 1/100th of Cloud Bigtable > Scylla vs CockroachDB – Database Benchmark > Loading 10x the data into Scylla took less than half the time it took for CockroachDB to load the much lesser dataset. > Scylla handled 10x the amount of data. > Scylla achieved 9.3x the throughput of CockroachDB at 1/4th the latency. [1] https://www.scylladb.com/product/benchmarks/ https://www.scylladb.com/product/benchmarks/

PeterCorless 5y ago
We just posted this today. Latest Cassandra 4.0 vs. Scylla 4.4. Note: Cassandra 4.0 is a HUGE improvement over Cassandra 3.11. But we're still many times faster: https://www.scylladb.com/2021/08/24/apache-cassandra-4-0-vs-scylla-4-4-comparing-performance/ https://www.scylladb.com/2021/08/24/apache-cassandra-4-0-vs-...
- deleted 5y ago
  [deleted]
manigandham 5y ago
Yes, Scylla does what it says. Used it in a prior adtech company and it beat everything else at that time for a very intensive key/value workload that also needed multi-regional replication. The adtech industry also uses Aerospike heavily but that (at the time) had many replication data model issues compared to Scylla/Cassandra.
steeve 5y ago

mianos 5y ago

It is interesting in that this is here on the front page and an old article about Discord moving to Cassandra is also here considering Discord went from Cassandra to Scylla I beleive.