I'd add that the benefits of avoiding bigdata(tm) can extended further into the 5TB+ space if you're happy to run at a bit of a delay. I.e. If it's ok for the

I'd add that the benefits of avoiding bigdata(tm) can extended further into the 5TB+ space if you're happy to run at a bit of a delay. I.e. If it's ok for the crunching to take 6 hours to produce intermediary aggregates which can then be crunched in less time , you can avoid spark and Hadoop for much longer.