8 ms·
To a someone not versed in NoSQL: what are Cassandra and Scylla like to use? Would you use it instead of say, Mongo, when you have much more data?
by kreetx 5y ago
To a someone not versed in NoSQL: what are Cassandra and Scylla like to use? Would you use it instead of say, Mongo, when you have much more data?
- dreyfan 5y agoThey’re for teams that still refuse a proper RDBMS but desire an Apache-level of infrastructure complexity.
- zinclozenge 5y agoI love this comment, but presumably there's a need for databases that are well suited for write heavy workloads. Also there's no great story for starting out with an RDBMS and then scaling out. Right now if you're with MySQL you have some options to scale out with read replicas, and then eventually replicate to a TiDB cluster. PostgreSQL I guess there's cockroachdb, but it's kind of meh.
- skyde 5y agoI believe Cloudflare usage of PostgreSQL is "write heavy" and is a RDBMS that scaled out very well for them. They are using CitusDB but it's still RDBMS and support most PostgreSQL SQL query without the need to rewrite your query or change your schema.
- zinclozenge 5y agoThat's true, I forgot about Citus.
- skyde 5y agoIn fact Youtube for a long time was running on MySQL with a tiny service called "vitess" on top of it to handle (Sharding/partitioning) automatically. I dont know if they are still using Vitess or migrated to Google other relational Database "Spanner"
- jeffbee 5y agoYouTube switched some Vitess workloads over to Procella. http://www.vldb.org/pvldb/vol12/p2022-chattopadhyay.pdf http://www.vldb.org/pvldb/vol12/p2022-chattopadhyay.pdf In general I've noticed that people outside Google assume that Spanner is much more popular than it really is.
- doliveira 5y agoThey moved to Clickhouse recently, though: https://blog.cloudflare.com/http-analytics-for-6m-requests-per-second-using-clickhouse/ https://blog.cloudflare.com/http-analytics-for-6m-requests-p...
- deleted 5y ago[deleted]
- doliveira 5y agoThere are other requirements other than low-volume transactional data and CRUDs...
- dreyfan 5y agoI’m guessing your usage of a RDBMS hasn’t extended beyond an ORM library.
- doliveira 5y agoI'm obviously being hyperbolic, not sure about you. Point remains that scaling RDBMSs is not exactly trivial and it does look like most companies eventually give up.
- dreyfan 5y agoEh, there are numerous options today (vitess, citus, clickhouse, aurora, redshift, bigquery, snowflake). Clickhouse outperforms Scylla by a wide margin at a fraction of the cost and complexity. Any of the above makes your data accessible via standard SQL. You can hire and utilize proficient data analysts, not people writing esoteric queries in whatever flavor of NoSQL, while actually solving business needs.
- doliveira 5y agoNot sure what you're talking about then? It seems you're just talking about the syntax used to query (which I agree SQL is a no-brainer) and not really about the "R" in "RDBMS"
- deleted 5y ago[deleted]
- dreyfan 5y agoYou haven't made a point yourself, so honestly, I don't fucking know what we're talking about either. You came in with a hyperbolic comment that echoes the stance from the NoSQL community I've seen for the past decade. The one that basically says "We have zero fucking clue how to model our data so we're just going to jam everything into bunch of loosely connected JSON documents and spend the next decade adding basic SQL features from the 70s".
- VirusNewbie 5y agoYou remind me of some ex coworkers who thought everything could be done on RDBMS systems and then when they had to export large amounts of data topped out at 3500 tps. Meanwhile I'm over here using Scylla and 3 nodes can give me roughly 3.5M TPS for data extraction. RDBMS are great for certain use cases but they do not scale and if you think they do you don't really know what scale means. Sure you can cite things like Github using MySql, but i'll also point out they've invested thousands upon thousands of man hours customizing layers on top of MySql to support their sharding which also, oops, had a bug that gave them almost half a day of downtime at one point. This is not a good use of someone's time, GH definitely made the wrong decision. Don't steer other people into making similar mistakes especially given it seems you have a lack of experience on designing systems at scale.
- deleted 5y ago[deleted]
- dreyfan 5y agoExactly where did I suggest using a single mysql instance or bandaiding a fleet of them together? Clickhouse vastly outperforms and outscales Scylla with a fraction of the hardware and cost.
- PeterCorless 5y agoDifferent purposes — OLAP vs. OLTP. Reminder of this thread: https://news.ycombinator.com/item?id=22478407 https://news.ycombinator.com/item?id=22478407
- deleted 5y ago[deleted]
- VirusNewbie 5y agoClickhouse is in no way an RDBMS.
- 5y ago
- cocoonkid 5y agois there a way to have a short private chat? I am an AppSec Engineer. Really max 10 minutes of your precious time.
- seabrookmx 5y agoIn a way, yes. Cassandra's clustering is more flexible. But the "price" you pay is that every table has a partition key. This requires more data modeling and thought upfront than your typical "JSON in, JSON out" document store like MongoDB or firebase.
- fennecfoxen 5y agoThe important thing to understand from this is how it changes the way your application handles reads writes. In a normal SQL database, you'd try hard to normalize your data, write an update once, do joins, and add an index for things you frequently read by, so the query planner can come up with something reasonable. When that reaches its scalability limit, you'll grudgingly consider various denormalization strategies, copying some subset of your data somewhere so that it's faster to query. If you're using Cassandra and you're also not a fool, it's probably because you actually need a lot of scale, so you'll generally do the denormalization up front and you will denormalize everything. If you're writing an event, you will write it several places. For example, suppose you're WalMart recording sales. You might write to: store transactions by store/year/day/hour (the "master" record insofar as you have one), user transactions by user/year/month, product purchases by manufacturer/year/day/hour... When you write the transaction, you write to all of these locations. Each of these "by X" keys is a shard. Each can be located on a different set of ~3 machines (the number is configurable). Querying involves getting a copy of the ring topology, computing which integer shard-ID the key maps to, figuring out which machines in the ring own that integer, and then asking the machine for a whole bucketful of data, which should be a superset of what you're actually looking for. For something like a user's transactions you'll want to have basically everything there at once, so loading the "order history" page for the past month might be a single query that just returns a report: no joins at all, very fast, super scalable. Other lookup strategies might ask for a range within that bucket (the data within the bucket can be ordered by a single key; often this is a timestamp or time-based UUID). Anything that isn't a simple query of a few buckets like this has to be a map-reduce job and will be slow. All of this is pain. You should generally not invite pain into your organization. However, if pain has already found you, something like this may be the least painful option.
- 5y ago
- dagnabbit 5y agoColumn stores are particularly good for handling large analytics workloads. As the name suggests, the defining feature of a column store is that instead of storing each row of data sequentially, they store the _column_ sequentially, so it’s very very fast to grab a column of data, and quite slow to get a row (the opposite of a typical RDBMS).
- jandrewrogers 5y agoCassandra, and by extension Scylla, are commonly used for workloads you probably wouldn't use MongoDB for at any scale. The primary strength is the ability to reliably scale-out the landing of live data. It supports writing more complex data models than logging systems but is slower and more limited for querying complex data models than e.g. an RDBMS. Cassandra is commonly used as a sink for complex service telemetry e.g. mobile data from the carrier perspective.
- orthecreedence 5y agoI didn't understand Cassandra/Scylla until I realized that it's basically a k/v store that lets you append data to existing keys. If your storage needs fit that domain, then I can vouch for Scylla being amazing to work with.
- fennecfoxen 5y agoTo put it in slightly more detail, it's a two-level data structure with two keys, a Dict[SortedList[Any]], to use Python type syntax. The top level key is also the sharding key.
- jjirsa 5y agoModern Cassandra is more like Dict[SortedList[Row]], where the row is basically a SQL row just like you’d expect in any SQL database
- jjirsa 5y agoSQL-like query pattern on top of bigtable-like distribution. e.g. you're willing to model your reads such that you dont need joins, and your sorting/scanning is based on the idea that you cluster data together, sorted in order you'll read it, into something called a "partition", which is how data is found within very large clusters.