This article has valid points but does not understand the perspective of companies. Companies do not buy technologies. Companies buy solutions. - Companies do

This article has valid points but does not understand the perspective of companies. Companies do not buy technologies. Companies buy solutions. - Companies do not buy Spark, they buy the ability to process their data and to have multiple personas collaborate (data scientists, data engineers, ...) - You can do it yourself. It will be cheaper but it will require time, expertise and money, all things that companies do not give easily - Snowflake and Databricks are elastic: you can start small and grow as you need. This is much easier than justifying the upfront cost of hiring specialized people or asking for trust that your ad-hoc solution will respect whatever enterprise governance rules (disclaimer: I worked at Databricks for 6 years and talked to hundreds of prospect and actual Databricks users and customers)