4 ms·
Previous discussion in September when they didn't have distributed processing: https://news.ycombinator.com/item?id=41496033 https://news.ycombinator.com/item?i
by chenxi9649 2y ago
Previous discussion in September when they didn't have distributed processing: https://news.ycombinator.com/item?id=41496033 https://news.ycombinator.com/item?id=41496033
Github Repo: https://github.com/lakehq/sail https://github.com/lakehq/sail
Few interesting notes:
- Benchmarks show 4x faster than Spark on TPC-H with 94% cost reduction.
- Currently at 65.7% PySpark test compatibility(they talk about this in more detail in the post)
- Built in Rust using Tokio runtime and Arrow IPC for high performance
- Already supports 79/99 TPC-DS queries
- chenxi9649 2y agoAlso, some discussions on Reddit from yesterday/today. https://www.reddit.com/r/dataengineering/comments/1gv840u/introducing_distributed_processing_with_sail_v02/?share_id=sfFIu7JG2DyFn9c63DOfk&utm_content=1&utm_medium=ios_app&utm_name=ioscss&utm_source=share&utm_term=1 https://www.reddit.com/r/dataengineering/comments/1gv840u/in... https://www.reddit.com/r/rust/comments/1gwayz6/introducing_distributed_processing_with_sail_v02/ https://www.reddit.com/r/rust/comments/1gwayz6/introducing_d...