3 ms·

Dask anyone? Allowed me to handle a 60 Gb dataset on a machine with 16 Gb RAM. High level of compatibility with Pandas dataframes.

• xiaodai 7y ago
  I mean, don't reach for Spark. Just get Dask, a databased or R's disk.frame is mostly fine.
• roaur 7y ago
  I'm a Dask evangelist. It's a remarkable tool and is one of the first I reach for when this problem arises. Maybe it's not well known?