5 ms·
This looks like a big hack to compensate for using the wrong tool. Cassandra would have been a better solution IMO. With Cassandra, you can set replication fac
by halayli 11y ago
This looks like a big hack to compensate for using the wrong tool. Cassandra would have been a better solution IMO.
With Cassandra, you can set replication factors, speed up the writes, and automatically shard the data without having to manage your own "mapping tables".
- econner 11y agoCassandra was immature when this work was started (late 2011). The team was also much more familiar with the ins and outs of operating mysql.
- halayli 11y agoTechnical debt is not an excuse to use the wrong tool. This is how you end up having a hammer. Cassandra was mature by late 2011 and we were using it in production back then (billions of records).
- econner 11y agoThe mapping tables are used to store relationships between objects, not to define shard locations. The shard mapping is encoded in the object id. In any case, I would have a knee jerk reaction to not trust any database system that hit 1.0 a month before I wanted to start using it, especially for mission critical core data.
- acveilleux 11y agoDoubly so when everything is catching fire around you from how overloaded it all is.
- wereHamster 11y agoFor this very reason many projects nowadays jump directly from 0.x to 2.x :) 2 is the new 1.
- luckycharms810 11y agoTechnical debt doesn't necessarily arise from using the wrong (a.k.a pretty similar but with a few features less than what you need) tool. In my experience, it's more likely that technical debt arises from trying to use thew newest tool (which looks to be a silver bullet) rather than trusting your own experiences. It's surprising to me that people are willing to move to a completely different technology rather than extend and repurpose what your engineers are already familiar with.
- joantune 11y agoAlthough, by using an existing tool, as long as it fits your needs, you'll avoid reinventing the proverbial wheel, ofcourse at the expense of learning something new.. which might not be a bad thing..
- _up 11y agoBut if you also need to use MySql anyway, Cassandra would be the "wrong Tool". Because you would have to maintain two DBMS instead of one. You want to minimize complexity and this can mean implementing your own Sharding.
- richardwhiuk 11y agoThere's also no transactions in Cassandra, although that's questionable useful given that the unidirectional map may be on a different machine.
- brianwawok 11y agoCassandra has very lightweight transactions. What he lists as his transaction use case (update if unmodified) I do right now with Cassandra and an IF clause and a timestamp... i.e. update foo set x = y AND last_modified_timestamp = 456789 IF last_modified_timestamp = 12345 see http://www.datastax.com/dev/blog/lightweight-transactions-in-cassandra-2-0 http://www.datastax.com/dev/blog/lightweight-transactions-in...
- deleted 11y ago[deleted]
- meritt 11y agoWhile that would have made scaling easier they would then run into a different set of massive problems because Cassandra isn't a relational database.
- brianwawok 11y agoBut neither is a sharded database. You can't do fancy joins and groups across 400 shards. (Well you can, but you are writing your own code to split up requests and stick it back together.. which is the same thing you would need to do in Cassandra).
- nemothekid 11y agoEven today I'm not sure I'd recommend Cassandra for his use case. A stated requirement he needed was: >Support asking for N number of Pins in a board in a deterministic order (such as reverse creation time or user specified ordering). Same for Pinner to likes, Pinner to Pins, etc. This can be a pain to model in Cassandra. It would require denormalizing for every key you wanted to order on, and it generally makes updating data a pain.
- jlongtine 11y agoI don't think that the schema here allows you to do this kind of query even in MySQL. They are throwing everything in that blob, which makes it pretty difficult to sort on anything that's in the blob. So, I think they would have to do what you are describing, denormalize everything... or get all the data and do the sort in the app. Which is precisely what would be required with Cassandra or another NoSQL system. Am I missing anything that would invalidate what I've just said?
- brianwawok 11y agoIf you only need 1 ordering, not too hard in cassandra. If you need a few different orderings, you would need a side table - which seems to be what you need here with mysql.
- axiak 11y agoThey also use HBase for a lot of their non relational data.
- georgiecasey 11y ago> 'We had several NoSQL technologies, all of which eventually broke catastrophically'