2 ms·
Distributed rate-limiting is intentionally not in the core library, Reader focuses on the scraping primitives and stays unopinionated about orchestration. For
by nihalwashere 9mo ago
Distributed rate-limiting is intentionally not in the core library, Reader focuses on the scraping primitives and stays unopinionated about orchestration.
For multi-node rate limiting, you'd layer that on top: Redis + a simple limiter that gates calls to reader.scrape().
For k8s resource settings, the Docker guide is a good starting point: https://docs.reader.dev/documentation/guides/deployment https://docs.reader.dev/documentation/guides/deployment
But I will add some reference examples on how to build a rate-limiting and K8s orchestration layer on top of Reader...
Thanks for sharing this :)
- verdverm 9mo agoThere's plenty of what you've built here to go around. It's trivial now to reproduce the basics. Distributed rate-limiting is a hard problem, one people may pay for
- nihalwashere 9mo agoI will add some reference examples on how to build a rate-limiting and K8s orchestration layer on top of Reader soon :)
- verdverm 9mo agoYou're missing the point, I don't want to build it myself. The framework I will actually use will do it for me. If yours does not, it will not be in consideration.