3 ms·
We're currently moving into a different direction, removing Spark Code to move most of the stuff into BigQuery SQL (which can use structs, one of the points of
by KingOfCoders 5y ago
We're currently moving into a different direction, removing Spark Code to move most of the stuff into BigQuery SQL (which can use structs, one of the points of the article), because it's easier for Data (Engineers|Analysts|Scientists) to write SQL than e.g. Scala.
- dominotw 5y agoSame here. Lots of places are doing this, afaik. shopify from pyspark -> sql https://shopify.engineering/build-production-grade-workflow-sql-modelling https://shopify.engineering/build-production-grade-workflow-...