3 ms·
The outbound dispatcher here seems like a prime candidate to write in a different language (Go for instance). Presumably it just makes HTTP requests so can be v
by obeattie 11y ago
The outbound dispatcher here seems like a prime candidate to write in a different language (Go for instance). Presumably it just makes HTTP requests so can be very simple -- building the request payload for instance could still happen in Python. And as you use Celery you already have a message queue over which it could communicate with the rest of your application.
- sametmax 11y agoThere is not need to overdo it. Asyncio + threads will do nicely. 500 req/seq is not the end of the world. I have similar load on purely threaded systems and they work fine. Crossbar.io would help though, but is not even necessary.
- robgolding 11y agoWe have considered this actually, and I think it will be something we look into more as our scale increases. As you say, we already have the communication infrastructure in place via Celery/RabbitMQ.