3 ms·
Given most of the backends use round robin for loadbalancing, having in-memory counter should be enough. Removing redis as downstream dependency is a big win.
by b1-88er 2y ago
Given most of the backends use round robin for loadbalancing, having in-memory counter should be enough. Removing redis as downstream dependency is a big win.
For the redis implementation, there should be fallback to in-memory counting instead blocking altogether. Currently the redis is a SPOF for the entire service.
- maerF0x0 2y agoif you're round robining clients w/o sticky assignment then you're going to get nodes*limit consumption. Not correct. Also if you give limit/nodes per node and random assign a connection, you get correct answers on average, but a really janky pattern at the edge case (a user gets a 429, and retries and succeeds, then gets 429 again as they consume those last few requests).
- b1-88er 2y ago> if you're round robining clients w/o sticky assignment then you're going to get nodes*limit consumption. Not correct. Fair point, using in-mem storage changes the meaning of the limit, since accounting changes to local. Something to consider in the library API.
- dasubhajit 2y ago> Given most of the backends use round robin for loadbalancing, having in-memory counter should be enough. Removing redis as downstream dependency is a big win. thanks for the feedback. planning to make redis optional in next release.