6 ms·
Cassandra can write 50GB of data in 0.12 milliseconds, more than 2,500 times faster than MySQL. Really? Cassandra can write 416TB, 666GB per second? If Cassa
by mdasen 17y ago
Cassandra can write 50GB of data in 0.12 milliseconds, more than 2,500 times faster than MySQL.
Really? Cassandra can write 416TB, 666GB per second? If Cassandra can write 50GB in 0.12ms, then it can write over 416TB in a second and 25petabytes per minute. Which clearly isn't true. Of course, that quote would also mean that MySQL can save 166GB per second.
The article is a piece about technology that the writer doesn't understand. They all have trade-offs. Key-value stores eliminate your ability to access data by anything other than its key. So, you have an article with an id of 5. What if you want to look up the articles by author 9? You're just unable. Column based databases aren't magic either. They just change the orientation of the data. Rather than seeing 1,Adam,West;2,Mark,Twain;3,Will,Smith you see 1,2,3;Adam,Mark,Will;West,Twain,Smith. That offers some advantages - such as being able to do metrics on a column easily - but it also means that if you want to get the data for one row, it has to do MORE work than a regular row-based database. To retrieve 1,Adam,West it has to do multiple lookups since the data isn't located next to each other. And frankly, random access of a row is what you're likely to want.
They run on clusters of cheap PC servers. PC clusters can be easily and cheaply expanded without the complexity and cost of "sharding," which involves cutting up databases into multiple tables to run on large clusters or grids.
That's like saying that if you plant a hamburger you'll get a McDonalds. They don't just magically run on clusters of cheap PCs. The fact is that there are tradeoffs. If you put 50% of the data on server1 and 50% on server2 you'll get faster query speeds except that you then have to know where to look for a specific piece of data.
So, sharding: basically, you split your data along logical (or non-logical) lines into separate databases. Like registration at a conference, users A-M are in DB1 and users N-Z are in DB2. So, when someone comes up with a query, you can easily tell which place to send them and then the query is run there. Or you can non-logically split and have a map that tells you where to go. So you query metadb and ask "where's Frank" and it says "Frank is in DB1" and then you query DB1 for Frank.
BigTable isn't so different. It does do some awesome auto-splitting of tables, but when it comes down to it, it's much the same. There's a single META0 tablet that gets queried to find the location of the META1 tablet which knows the location of the actual data. And, in order not to overload the META0 tablet, one needs to cache the location that you get back.
--
Often times, the problem is that you have web programmers who wouldn't know a B-Tree from a linked list. All of these technologies exist because there are certain things that they're good at. However, what this article purports to know is that these NoSQL databases are just superior. In most ways, they're inferior. A key-value store has its place, but it's severely limiting in a way that you can't just use a key-value store; you need more querying power. Likewise, whether a column-oriented database is right for you depends more on how you want to access the data (by column or by row).
Oh, and really, learn SQL indexing and check whether your queries are doing full table scans. That's the root of a lot of problems. I mean, a good index should do lookups in log(n) time which means that on 10,000 rows a query should take 1/769th the time; on a million rows a query should take 1/50,000th the time; on a billion rows it will take 1/33,333,333th the time of doing a full table scan of a table that size. Yeah, indexes make a huge difference, but they aren't magic either. They merely order the data in a certain way that makes it easy to pluck out certain rows since you don't have to look at every value. For example, dictionaries are alphabetized and so you don't have to look at every word. You just start jumping toward the area of the word you're looking for. Indicies work the same way. Now imagine if the dictionary was unordered. You'd have to look at every single word to see if it was the word you were looking for.
Oh, and just saying "put an index on stuff" isn't enough to know. It isn't too complicated, but there's a Google talk that goes over some of the mistakes one can make with assuming that indexes are doing what you want (http://www.youtube.com/watch?v=u70mkgDnDdU&feature=channel_page http://www.youtube.com/watch?v=u70mkgDnDdU&feature=chann...). It's in the middle somewhere, but it's a good watch for the whole thing.
--
Really, is this the type of article we've become? I mean, column-oriented datastores and key-value datastores are both valuable technologies, but a NoSQL flame written by someone who doesn't know that Facebook is still MySQL backed and uses Cassandra for only specific things and that they use many different technologies where they're good (like heavy use of memcached as a key-value store to reduce load on MySQL where queries aren't needed). Or perhaps that the article doesn't seem to acknowledge the drawbacks that the authors of these systems would talk about. For example, mongoDB states that they lack ACID integrity and that it's more suited for "High volume, low value data" - that should be read as whether someone has upvoted or not; if that data gets lost it doesn't really matter so much because the person can click the vote button again and all is well or just because it isn't super important data.
Why are we supporting "we hate this technology just because"? Technologies are tools that have their places. It's foolish to write off a good technology just because you want to make fun of it or deride it. Use the right tool for the job. In your web application, you might find that using multiple tools in concert is the way to go.
- archon810 17y agoI also noticed this outrageous claim and left a comment in the article. It's shocking that the author can just blurb out random BS and post it without verifying.
- drawkbox 17y ago"They all have trade-offs. Key-value stores eliminate your ability to access data by anything other than its key. So, you have an article with an id of 5. What if you want to look up the articles by author 9? " For cases like that you don't have one source of data, you do almost like an AFTER INSERT trigger to update other flattened tables of data that you need. For instance, where you would normally perform a JOIN you would just after save update a JOIN table or a VIEW that is created by incremental updates. So then you could filter by many other things and really aren't limited too much by space to make those duplications. It happens with large RDBMS systems anyways after about 5GB of data, same type of scalability decisions have to be made. But yes the article is off and you are right, most likely systems will approach this with multiple solutions for specific problems.
- gaius 17y agoIt happens with large RDBMS systems anyways after about 5GB of data, same type of scalability decisions have to be made. More like 5Tb these days.
- sho 17y agoWell, the disk space is not the controlling parameter usually. Getting a ridiculously huge amount of fast, reliable disk is pretty much a solved problem. If all you were doing was writing logs at a constant rate then you could "scale" pretty much indefinitely just adding larger and larger disks to the same poky DB server, and it wouldn't matter if it was 5GB or 5PB. The proper scaling issue is, of course, with the cpu and memory of your single DB server - and its disk speed - which can only be expanded so much, and at exponentially increasing cost. It's conceivable that even with a fixed amount of data, say 100G, you might have to confront scaling issues with increasing site usage, long before you reached anywhere near the limits of your raw disk space.