The benefit of Spark and related systems is you get a flexible infrastructure that can handle a wide range of tasks reasonably well. You pay once for infrastruc

The benefit of Spark and related systems is you get a flexible infrastructure that can handle a wide range of tasks reasonably well. You pay once for infrastructure, training, devops, and so on. You can optimise any particular use case to perform better than Spark, but then you are going to incur the above costs for every project you create.