Spark is still commonly used for data processing and large-scale transform jobs (I see it often for parsing and converting datasets from one format to another).

Spark is still commonly used for data processing and large-scale transform jobs (I see it often for parsing and converting datasets from one format to another). Given that this survey is from Kaggle, I won't be surprised if there's some selection bias going on with the results. I suspect most of the respondees primarily work with smaller datasets at work with limited big data or ML opportunities, and hence were on Kaggle to get experience in those areas? According to the survey the most common query language is MySQL, which isn't great for actual big data.