Spark is nice, but its memory model almost requires a full cluster overhaul. We looked at it at my last project. Our cluster nodes only had 64 GB of RAM. That w

Spark is nice, but its memory model almost requires a full cluster overhaul. We looked at it at my last project. Our cluster nodes only had 64 GB of RAM. That was carved into 4 GB workers. In order to use Spark we'd have to halve our number of workers because of the memory requirements. Neat project. Has its place. Requires a different cluster configuration which might limit its utility.