3 ms·
Cool, nice idea. Can you sub in different backend like bytewax (https://github.com/bytewax/bytewax https://github.com/bytewax/bytewax) for stateful processing?
by amath 4y ago
Cool, nice idea. Can you sub in different backend like bytewax (https://github.com/bytewax/bytewax https://github.com/bytewax/bytewax) for stateful processing?
- calebtv 4y agoThanks! Currently you can't, right now your only option is to use our ray runner. But we have talked about supporting different runner options similar to how Beam can be run on Spark, Dataflow, etc. And ultimately it would be nice if folks could implement their own runners, but I think we're still a ways out on that.
- calebtv 4y agoI should also mention BuildFlow does support stateful processing with the Processor class API: https://www.buildflow.dev/docs/processors/overview#processor-class https://www.buildflow.dev/docs/processors/overview#processor...