3 ms·
> A Rama operation does not return a value to its caller. It emits values to its continuation. This is a critical distinction, as part of what makes Rama operat
by beders 2y ago
> A Rama operation does not return a value to its caller. It emits values to its continuation. This is a critical distinction, as part of what makes Rama operations more general than functions is how they can emit multiple times, not emit at all, or emit asynchronously.
- thom 2y agoWhich is obviously very similar to how transducers already work in Clojure, but they still lack some of concurrency options of reducers. Getting all this on a smart, distributed runtime seems very promising.
- mschaef 2y ago> Getting all this on a smart, distributed runtime seems very promising. Hopefully it is. This CPS article is the first of the Rama blog posts where it seemed like there might be something there. The earlier posts - "I built Twitter scale Twitter in 10Kloc" - were never really all that convincing. The thing they claimed to have built was too ambitious a claim.
- thom 2y agoOh I think there’s a lot of good stuff baked in there. The big idea downstream is that you have incrementally calculated, indexed data structures to query all the results of this fancy CPS logic. It’s all slightly esoteric even coming from a Clojure background but it ticks every box I want from a modern data platform, short of speaking SQL.
- dig1 2y agoThere is a library called tesser [1] (by Jepsen/Riemann author) that behaves like parallel transducers with more Clojure "native" syntax. With transducers, you have to use "comp," and with tesser, you use "->>" as you'd use with lazy functions. Sadly, tesser is not advertised as it should; I find it much more flexible than transducers. E.g. you could parallelize tesser code over Spark/Hadoop cluster. [1] https://github.com/aphyr/tesser https://github.com/aphyr/tesser
- aeonik 2y agoSee the injest library as well, I've had pretty good experience with ita performance. https://github.com/johnmn3/injest https://github.com/johnmn3/injest I can squeeze more performance out of tesser, but injest gives me a surprising boost with very little ceremony most of the time.
- thom 2y agoAs I understand it Tesser doesn't support ordering which makes it tricky for many types of logic. I'd like to be able to control exactly which parts of a pipeline are parallelisable without too much ceremony, but I can't say I've used Rama in anger or that it makes this ergonomic.