3 ms·
I had prior industry experience. Eventually it was realized that getting a larger box and just spend sometime to think about cleaning the data is enough. But t
by Aperocky 6y ago
I had prior industry experience.
Eventually it was realized that getting a larger box and just spend sometime to think about cleaning the data is enough. But that didn't sound as good.
- riku_iki 6y agoYou can also think about two layers infra: have large big-data storage, have simple logic of extraction of aggregated/filtered data from it, and do complex work on your large box within single process.
- Aperocky 6y agoIt still made sense to store stuff in Hadoop - but it didn't make much sense to do anything further than extraction in the cluster, which we definitely did try with Spark + mllib.