3 ms·
> YAGNI - For lots of small data tasks, pandas is perfectly fine. There’s a real possibility the growth of single node RAM capacity will perpetually outpace th
by halfcat 2y ago
> YAGNI - For lots of small data tasks, pandas is perfectly fine.
There’s a real possibility the growth of single node RAM capacity will perpetually outpace the growth of a business’s data. AWS has machines with 32 TB RAM now.
It’s a real question as to whether big data systems become more accessible before single node machines become massive enough to consume almost every data set.
(Also love your work, thank you)
- __mharrison__ 2y agoBut... but we have to have big data and a cluster... The hoops folks will jump through... (Thanks!)