3 ms·
So what we were going for with the core API is that it should be expressive enough for all kinds of distributed programming tasks, not just Spark-like or batch
by pchiusano 5y ago
So what we were going for with the core API is that it should be expressive enough for all kinds of distributed programming tasks, not just Spark-like or batch computing stuff.
We’re planning on doing several of these articles to show different applications. I think some distributed stream processing library would be super cool and there are lots of sources of inspiration for that like you mention. It would be a more interesting translation than the Spark-like stuff though. Likely using the channels ability for message passing.
> By the way, I really loved the FP in Scala book, it was my first step into FP -- thank you and Runar for writing it
That’s great to hear!
- lidatong 5y agoThank you both for the answers! > the thing that would have helped me most in avoiding some of the missteps might have been a richer test interpreter, where I could have "mocked" a network This is a really intriguing idea. Debugging distributed can be a huge headache, and I can see a lot of value in having something like that. > We’re planning on doing several of these articles to show different applications. Looking forward to future articles and definitely will be following this project!