4 ms·
Would love to see Lake Sail overtake Spark, so we could generally dodge tuning the JVM for big Spark jobs. https://docs.lakesail.com/sail/latest/ https://docs.
by ZeroCool2u 2y ago
Would love to see Lake Sail overtake Spark, so we could generally dodge tuning the JVM for big Spark jobs.
https://docs.lakesail.com/sail/latest/ https://docs.lakesail.com/sail/latest/
- esafak 2y agoHave you used it? How does it compare to other Rust-based ETL frameworks like Arroyo?
- ZeroCool2u 2y agoI haven't personally used it and I believe it's still relatively early in development, but it provides a pyspark compatible python API, so the idea is you should be able to migrate your Spark/Databricks workloads easily. Arroyo seems to be focused on Streaming. Lake Sail targets streaming and batch workloads.