Probably a bit of both - I'm no SQL wizard, and I'm indexing 300 million+ Twitter messages and storing the @reply relationships (or more generally analyzing mas

Probably a bit of both - I'm no SQL wizard, and I'm indexing 300 million+ Twitter messages and storing the @reply relationships (or more generally analyzing massive communications stores, eg an organization's email server). I'll try to put together some reproducible code, but for example I had a relationship table that once it grew into millions of entries, took noticeable wall-time to simply access via a primary key. Switching to InnoDB after spotting a note in the memcache FAQ that it supports much faster primary key fetches fixed that, but before that I was looking into the black box with no good leads on fixing it.