5 ms·
Every big data platform I know of is or already has implemented manual memory management (they all run on the JVM). I can see Rust doing well in this world. It'
by noelwelsh 11y ago
Every big data platform I know of is or already has implemented manual memory management (they all run on the JVM). I can see Rust doing well in this world. It's hard to say things like Spark are undervalued. (Two data points: ~$50M investment in Databricks, the company commercialising Spark. IBM just committed ~3.5K developers to working on Spark projects.)
- kibwen 11y agoBig data could be huge for Rust. I have a friend who works in the Hadoop ecosystem who's about to take a 90% pay cut to leave his current job in favor of using Rust at a startup. If I had gratuitous amounts of money, this is the field that I'd be investing in.
- frankmcsherry 11y agoShameless self-promotion for big data in Rust: https://github.com/frankmcsherry/timely-dataflow https://github.com/frankmcsherry/timely-dataflow https://github.com/frankmcsherry/differential-dataflow https://github.com/frankmcsherry/differential-dataflow Nothing much useful to contribute, except that they mostly work, are largely unsafe-free (unsafe in some sorting, and a Drain replacement), and build on 1.0 stable.
- pella 11y agoThe new Cockroachdb implementing in GOlang https://github.com/cockroachdb/cockroach https://github.com/cockroachdb/cockroach