3 ms·
Pull/Push itself is not the problem unless you have stats to back such implied claims. It is far more likely that your highly inefficient Pull/Push implementati
by tw1984 3y ago
Pull/Push itself is not the problem unless you have stats to back such implied claims. It is far more likely that your highly inefficient Pull/Push implementation caused the problem. Without any real insight on why the pull/push based approach is slower, there is nothing the "Go community" can get.
All valuable details were omitted, e.g. 1) how much data is being moved? a few megabytes? what is the state of art latency? maybe 0.001 second for that? when latency is still seconds huge, what can be further improved, in the go runtime or in the apps? etc
A highly efficient pull/push model without using go channel can easily beat your 1-5 seconds latency. A crappy implementation using channel can get worse.
Delay of several seconds is when you pass on information mouth to mouth, in 2023, that kind of delay doesn't justify a blog article on such fancy implementation. I'd suggest you to delete the implementation to rebuild from ground zero.
- saisrirampur 3y agoThanks for the comment. We will aim to write a more deeper blog on this in the future. We were observing 60<>40 ratio for Pull/Push, as specified in the blog and majority of the Push was target data-store (network) bound. So there was less room of optimization on the Go side. The test was done with ~10-15K TPS on Postgres and the target was Azure Event Hubs, which has a limit of 1MB write batch size.
- earthboundkid 3y agoYes, thank you.