3 ms·
Tough call. If you did want to use Riak for fast bucket-drop, your best bet might be to: a.) Run multiple clusters--not too difficult. Just give each one a dif
by aphyr 14y ago
Tough call. If you did want to use Riak for fast bucket-drop, your best bet might be to:
a.) Run multiple clusters--not too difficult. Just give each one a different erlang cookie and run em on subsequent ports.
b.) Take bitcask_backend or leveldb_backend and add drop-bucket functionality. Custom backends are more difficult than running multiple clusters, but certainly not impossible. You could build it on top of fold or split writes up into, say, one leveldb per bucket. Don't recall if the vnode interface has drop-bucket so you might have to write some plumbing alongside Riak. jrecursive has done this in Mecha.
If I were building something like this, I might look first at Cassandra or Hbase, or possibly sharded master-slave postgres.
- ithkuil 14y agoThank you for your answer. The problems I see with (a) from the top of my head are: 1. Even if it's easy, somebody has to do that. 2. Setting up all the monitoring etc for each instance 3. Running more than one riak daemon on the same machine means that the riak daemon is unaware of the IO operations performed by the other one, hence IO throughput could suffer. This means that in practice you would need to mount separate disk heads (and we are back to 1.) 4. Each riak instance will require some RAM as well, so memory has to be allocated and there is the risk that's over-allocated. 5. Port allocation. I fear it would end up with smth like: "just keep a internal wiki page where each 'db space' is mapped to a port number" Well, the problem with (b) is of course that I don't have time to do that. For now we stick to cassandra, but Riak is so nice in many aspects that I really hope that at some point, as the product matures, more resources can be invested in aspects which are not currently perceived as "selling points" for riak, but are important for some scenarios and not technically impossible.
- aphyr 14y agoYeah, if you're using Cassandra and the GC/rebalancing issues aren't affecting you, you're probably fine sticking with it. Both are Dynamo-structured, so your consistency/failover model advantages are similar.