3 ms·
Is there an underlying stream processor (e.g. Flink)? How many messages per second can it process?
by jcnnghm 4y ago
Is there an underlying stream processor (e.g. Flink)? How many messages per second can it process?
- calebtv 4y agoAll of our processing is done via Ray (https://www.ray.io/ https://www.ray.io/). Our early benchmarks are about 5k mesesages per second on a single 4 core VM, but we believe we can increase the with some more optimizations. This bench mark was consuming a Google Cloud Pub/Sub stream and outputting to BigQuery.
- dmatrixjsd 4y agoDelighted to hear your choice of Ray and building atop Ray.