3 ms·
This sounds much like the way we were using MySQL when I worked on a large consumer site. We both sharded and replicated to scale out. And, we used MemCached t
by ShabbyDoo 17y ago
This sounds much like the way we were using MySQL when I worked on a large consumer site. We both sharded and replicated to scale out. And, we used MemCached to bridge the replication lag. For better or worse, all writes auto-committed independently. What strategies should Keyspace users employ to counteract similar replication delay issues?
I presume Keyspace would offer much better performance because it doesn't have to worry about 95% of the stuff a SQL db does. Drizzle seems like it falls somewhere in the middle of the solution in the above paragraph and Keyspace. Is that a fair assessment?
One thing I wanted (among many) with our solution was the ability to specify async disk forcing (an oxymoron?). I'm willing to allow a database to return control to me before writing if it thinks it will be able to write something to disk within N seconds. The increase in performance for most non-transactional systems (meaning that a few seconds of data loss occurring once a year wouldn't hurt too badly) would more than make up for the risk. Terracotta, Gigaspaces, etc. are interesting to me because they facilitate async persistence.
- agazso 17y agoWithout being familiar with the details of your use-case, one of the nice properties of Keyspace is that it is a high-performance (async, C/C++, epoll on Linux) UNIX server --- we are able to get 100.000 writes/sec with n=3 replication on fast Linux PCs with SCSI disks. If you use memcached appropriately, then I suppose Keyspace should be able to handle the "remaining" reads and writes? With Keyspace, you currently cannot turn off disk syncs --- note that Keyspace is still very fast with disk syncs --- as this would break our replication algorithm, Paxos. We might make this an option if users request, which would result in a weird replication consistency somewhat like "eventual consistency", but differing from what is usually refered to under this term. Btw, making this an option is quite easy, roughly 2 minute of work (TXN_NOSYNC with BDB).
- andrewf 17y agoSomething I noticed here and on your blog - I recommend you adopt the American comma convention for writing "100.000". Many people will read that as one hundred (with a decimal point); 10e6 is a lot more interesting than 10e3!
- sparky 17y ago1.0e5 is a lot more interesting than 1.0e2 :P
- ShabbyDoo 17y agoWe were writing first to the master MySQL instance and then immediately putting the same data in memcached. When we read, we'd go to memcached first and then go to one of the read slaves. This way we would limit the effects of replication delay. Note that I don't necessarily like this scheme, but it's the one we used. Is there a replication delay issue with Keystore?