This goes back to the old adage "the right tool for the job". As @davidmr points out, there are jobs on smaller data sets that can still benefit from the distr

This goes back to the old adage "the right tool for the job". As @davidmr points out, there are jobs on smaller data sets that can still benefit from the distributed nature of HPC. That said, my own experience with startups echoes much more what OP writes - python scripts and csv processing has saved me days of headaches in resource constrained environments. I was able to quickly produce analysis and make crucial scaling decisions using data that would have taken days of engineering resources to produce. I happened to have the right tool for that job handy, and it worked out great. You really have to think things through before restricting yourself to any specific direction.