Immediately starts to doubt OP's assumption/implementation of the "where it works great" Jokes aside, agree with others. For the 7500/day, I would just push th

Immediately starts to doubt OP's assumption/implementation of the "where it works great" Jokes aside, agree with others. For the 7500/day, I would just push these into an S3/minio folder. And then dequeue 100 or N once every 10/30/60/T seconds. play around for the right N,T. Then again am sure there maybe reasons/context/constraints unaware to us. Eg - the ingestion is spiky, with the possibility of all 7500 in a few secs/minute, you would want to first make sure that the http traffic can scale before getting to the point where it can actually connect and push to the queue Possible reason#2 - an intern who just finished up their first Kafka task; just got freed up #3 - or this was the only infra available and a choice had to be made with the time available at hand #4 - or this was an experiment to see for yourself A majority may not agree with your view, but none of us really are in your shoes. So I applaud you for sharing your thoughts anyway.