Yeah, the story is a bit incomplete. The full chain was application->transaction master -> 20 or so slave databases. It was a weekly batch dispatching data t

Yeah, the story is a bit incomplete. The full chain was application->transaction master -> 20 or so slave databases. It was a weekly batch dispatching data to the 20 slaves, starting with a full delete of each slave. Someone decided to literally pull the plug and replace the master database server node from the rack, while the batch was still running. He assumed the other server nodes would pick up where this one left off. So the batch log of the application first complained about the master disappearing, then about the rollback failing on another master node because it wasn't the coordinator and had no idea of this transaction. It also means the decision about the commit/rollback was irrelevant, as next week's batch run had deleted the records in question. Presumably, some ephemeral records were hanging around, deciding if they were deleted either in week X or week X+1.