In my experience, if you don't have security or privacy requirements, you're much better off putting your data in an easy to autoscale environment like BigQuery

In my experience, if you don't have security or privacy requirements, you're much better off putting your data in an easy to autoscale environment like BigQuery, RedShift etc. than trying to fiddle with on prem or cloud Hadoop clusters. Hadoop requires non trivial amount of management and fine-tuning which you can do if you have the scale to deploy a specialist platform team.