3 ms·
I built daria (https://github.com/MrPowers/spark-daria https://github.com/MrPowers/spark-daria) to make it easier to write Spark and spark-fast-tests (https://g
by MrPowers 5y ago
I built daria (https://github.com/MrPowers/spark-daria https://github.com/MrPowers/spark-daria) to make it easier to write Spark and spark-fast-tests (https://github.com/MrPowers/spark-fast-tests https://github.com/MrPowers/spark-fast-tests) to provide a good testing workflow.
quinn (https://github.com/MrPowers/quinn https://github.com/MrPowers/quinn) and chispa (https://github.com/MrPowers/chispa https://github.com/MrPowers/chispa) are the PySpark equivalents.
Built bebe (https://github.com/MrPowers/bebe https://github.com/MrPowers/bebe) to expose the Spark Catalyst expressions that aren't exposed to the Scala / Python APIs.
Also build spark-sbt.g8 to create a Spark project with a single command: https://github.com/MrPowers/spark-sbt.g8 https://github.com/MrPowers/spark-sbt.g8