5 ms·
The timing seems a little defensive, but I think this raises a good point. Databases, scaling, persistence, and synchronization are hobbies of mine. As a resul
by heretohelp 14y ago
The timing seems a little defensive, but I think this raises a good point.
Databases, scaling, persistence, and synchronization are hobbies of mine. As a result, I generally find the "FSYNC AFTER EVERY DATABASE OPERATION!" crowd rather tedious.
Even more so when they:
1. Don't know what they're talking about
2. Publish benchmarks of databases they don't understand compounded with a misunderstanding of statistics
With that said, I'm skeptical of how useful MemSQL's specific trade-offs are. They represent a nice contrast to how most other databases are currently configured, but I'm not sure what it's supposed to be for.
It seems as if it's designed for situations where you're doing a lot of table scans. This is rather odd. Are you optimizing for the lazy company that won't set up an OLAP for analytics and is trying to make their OLTP database do the work? The vast majority of transactional or just plain 'usual' work is indexed fetches by id of some row, document, kv blob, etc.
I know OP is actually the CTO of MemSQL and seems to be rather straightforward so I'd like an answer as to what MemSQL is trying to solve.
At present, it doesn't seem like MemSQL is actually better at the sundry "fetch by pk/guid" 95% use-case than MySQL. If it is, I'd like some evidence of that.
Answers that can be summarized as "BETTER THAN MYSQL!" will be met with derision though ;) Talk to me like an engineer...in trade-offs.
Edit: that "MySQL is a bazillion times faster than MemSQL" post appears to be the top Google result for MemSQL now. That's gotta sting. N.B. This is a common mistake database vendors make. Not explaining their niche/use-case clearly. They love to claim they're superior at everything. Basho is marginally less guilty of this than most.
- true_religion 14y ago> . Are you optimizing for the lazy company that won't set up an OLAP for analytics and is trying to make their OLTP database do the work? What you have described is pretty much every startup (rationale: moving too fast to bother, not enough time to do it).
- heretohelp 14y agoNope, the opposite. There are a lot of technologically incompetent non-startup SMBs that do this. Startups are much more likely not to jackass themselves into something stupid and just use Hadoop/Amazon Map-reduce/whatever.
- nikita 14y ago> It seems as if it's designed for situations where you're doing a lot of table scans. This is not quite true. Lockfree data structure work exceptionally well for a lot of small concurrent reads and writes. If you look at the benchmarking code: https://github.com/memsql/bench/blob/master/sql_database.py https://github.com/memsql/bench/blob/master/sql_database.py you will notice that there are a lot of queries like "fetch by pk/guid". The trade-offs are: taking a hit the first time you run the query, being limited to the amount of RAM, limited set of features for this release. The benefits are throughput for "fetch by pk/guid" transactions and predictably low latencies.
- heretohelp 14y agoExcellent, you've answered my question! Thank you. :)
- continuations 14y ago> being limited to the amount of RAM What will happen if the data grows to larger than the RAM?
- nikita 14y agoWe stop accepting write queries. We also work on sharding. RAM gets you pretty far 12K buys a terabyte these days.
- malkia 14y agoSo you are only safe, as the speed of HDD writing sequentially to the disk (it could be even directly to the volume, not even running from fs)?
- moe 14y agoOut of curiosity, where do you get an 1T-box for 12K? Spec'ing those out at Dell I arrive at >40k. I know channel-partner or own build can trim that significantly, but your price sounds pretty amazing.