I read it and I am like ... we already do this. This is common and obvious. Maybe I am missing something. Before worker nodes had as much memory as they have n

I read it and I am like ... we already do this. This is common and obvious. Maybe I am missing something. Before worker nodes had as much memory as they have now, almost everything needed to use small buffers and spill to disk. BDB (Berkeley DB) was an extremely common tool for doing out of core data operations. Because the ETL tools I was writing needed to run on machines with 512MB of ram, it required out of core algorithms. We easily had jobs processing 10-20GB with only 512M of ram. I am sure I am missing something, reading the paper now. http://kth.diva-portal.org/smash/get/diva2:1334587/FULLTEXT01.pdf http://kth.diva-portal.org/smash/get/diva2:1334587/FULLTEXT0...