you really, really don't want to be using a relational database here. hitting disks a million times per minute would be disastrous, if not for performance then

you really, really don't want to be using a relational database here. hitting disks a million times per minute would be disastrous, if not for performance then for blocking IO, even with only row-level locking. I'd look into memcached and write state to disk every once in a while to back up, kind of a reverse from what's usually done. Or, if thats really not enough, I'd look up Hadoop riding on GlusterFS. You're going to have synchrony and blocking problems no matter what, though, without really creative keying (bigtable style?). Both Facebook and Google have run up against these kinds of issues and have written and/or released source about it. Facebook has OSS'd Cassandra, their distributed database, and Google hasn't released much code but has papers out on their implementations of MapReduce (open source port->Hadoop, as suggested above) and BigTable (which the Hadoop team are in the process of cloning with HBase).