"A research project had to load the full follower graph in memory" Really? It had to? There was no possible alternative? As often as I get the "fail whale"

"A research project had to load the full follower graph in memory" Really? It had to? There was no possible alternative? As often as I get the "fail whale" page on Twitter, I'm always skeptical seeing them present stuff like this and release code. Maybe I'm just not grasping how large and popular Twitter is, but of the popular web services I use, Twitter fails more than all the others combined. Though it was an interesting read, it seems suspect that they're having so many more problems than everybody else.