6 ms·
There is an ongoing work on creating Spark alternative in Rust[1][2]. Hopefully, that could help. [1] https://github.com/rajasekarv/native_spark https://github
by xvilka 7y ago
There is an ongoing work on creating Spark alternative in Rust[1][2]. Hopefully, that could help.
[1] https://github.com/rajasekarv/native_spark https://github.com/rajasekarv/native_spark
[2] https://medium.com/@rajasekar3eg/fastspark-a-new-fast-native-implementation-of-spark-from-scratch-368373a29a5c https://medium.com/@rajasekar3eg/fastspark-a-new-fast-native...
- pkolaczk 7y agoWill take years to get there, if they even get there at all. This is a single-man side project now. Also benchmarking something that has 1/100 of the original functionality and is not production ready is very unfair. It is much easier to get a specialized use case faster than a general purpose system.
- playing_colours 7y agoHe is not the first, Andy Grove worked on a similar thing last years, but he worked mostly alone, and without community you cannot take off such ambitious project: https://andygrove.io/2018/11/datafusion-2019/ https://andygrove.io/2018/11/datafusion-2019/