3 ms·
I only know Hadoop from reading about it on HN, and the blog post I remember the most is "Don't use Hadoop - your data isn't that big" https://www.chrisstucchi
by jakub_g 10y ago
I only know Hadoop from reading about it on HN, and the blog post I remember the most is "Don't use Hadoop - your data isn't that big"
https://www.chrisstucchio.com/blog/2013/hadoop_hatred.html https://www.chrisstucchio.com/blog/2013/hadoop_hatred.html
TL;DR
> But my data is 100GB/500GB/1TB!
> A 2 terabyte hard drive costs $94.99, 4 terabytes is $169.99. Buy one and stick it in a desktop computer or server. Then install Postgres on it.
- sn9 10y agoThere's also this [0] on how to outperform Hadoop with command line tools. Obviously, you should look at the role I/O costs can play. A lot of RAM and some SSD drives might be a better idea than Hadoop at this scale. [0] http://aadrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html http://aadrake.com/command-line-tools-can-be-235x-faster-tha...
- greendragon 10y agoIt remains true. 4TB now is like $100 too, and SSDs that size are around even if they're an order of magnitude more expensive. (And then there's Seagate's 60 TB SSD prototype, clearly the future is with SSDs.)