3 ms·
Great options already exist: MemSQL for blazing fast distributed full SQL database with cross-datacenter replication and in-memory rowstore + disk-based column
by manigandham 10y ago
Great options already exist:
MemSQL for blazing fast distributed full SQL database with cross-datacenter replication and in-memory rowstore + disk-based columnstore.
ScyllaDB for Cassandra rewritten in C++ for blazing fast dynamo-style multi-master multi-datacenter disk-based wide-column database.
I'll also throw in AMPS (by 60East) as the best messaging platform that supports innovative SQL and real-time state-of-the-world queries on it's message streams (instead of using that rabbitmq or kafka crap).
- akbar501 10y agohttps://github.com/Netflix/dynomite https://github.com/Netflix/dynomite Dynomite for production proven, limitless scale for Redis. Provides ability to Redis for both in-memory and on-disk workloads. Dynamo-inspired linearly scalable, shared nothing multi-master architecture. Pairs well with Cassandra. Supports pluggable backends (ex. RocksDB). Been in production 2+ years. One of the larger clusters handles over 3.6 million sustained ops/sec in production, every day.
- ddorian43 10y agoRedis is only the api/driver, not the features, right ? Meaning it's just crud and not the many features/data-types that redis provides, correct ?
- akbar501 10y agoIt's the full Redis API and protocol. See https://github.com/Netflix/dynomite/blob/dev/notes/redis.md https://github.com/Netflix/dynomite/blob/dev/notes/redis.md for a list of all supported Redis features. The benefit of Dynomite's support for the Redis API + protocol is a.) you can use any Redis client and b.) the same code can be used for standalone Redis on your laptop and on a distributed Dynomite cluster.
- manigandham 10y agoThis is neat, but why use this over just using Cassandra-based DBs? Also you were a speaker at Data Layer right?
- akbar501 10y agoYes, I was a speaker at DataLayer. There are two answers to your first question. 1. Dynomite pairs well with Cassandra. At scale, Cassandra is not a speed deamon when it comes to reads. Dynomite helps to improve Cassandra's read performance. 2. The use case for Dynomite as the primary database is for workloads that require high throughput and low latency. In other words, Dynomite delivers consistently lower latency at any scale.
- manigandham 10y agoBetter to just use scylladb which natively handles it rather than run another layer of software.
- ddorian43 10y agoMemsql + AMPS have "contact us" pricing unfortunately. Do you know any sharded queue (like amazon sqs,iron-mq) open-source ?
- manigandham 10y agoYes, those are both closed source but great companies that will work with you. Happy to put you in touch if you want. MemSQL also has a completely free community edition. For open-source fast and clean messaging, I'd recommend NATS, they recently built NATS streaming that builds on top of the pub/sub to include kafka like persistence - http://nats.io/ http://nats.io/
- ddorian43 10y agoYes but memsql community doesn't have high availability (voltdb does the same thing). I don't think nats can be used like I said by looking at the docs.
- manigandham 10y agoIt has clustering: https://nats.io/documentation/server/gnatsd-cluster/ https://nats.io/documentation/server/gnatsd-cluster/ What are you looking to do? Sharding for what? Data size? Throughput?
- AliaksandrH 10y agoSome companies are closed source for a really good reason: it allows for Vertical Integration and Focused Execution. Bay Area companies spend millions of dollars worth of engineers salaries in projects that layer complexity upon complexity (and you need dozens of engineers to maintain them), just because they are in love with idea of open source. This is dangerous proposition, and it stifles innovation. There is an opportunity to dramatically simplify a lot of data pipelines. Check this out: http://docs.memsql.com/docs/pipelines-overview http://docs.memsql.com/docs/pipelines-overview
- 10y ago