3 ms·
I wrote a message queue in Python called StarQueue. It’s meant to be a simpler reimagining of Amazon SQS. It has an HTTP API and behaves mostly like SQS. I w
by andrewstuart 4y ago
I wrote a message queue in Python called StarQueue.
It’s meant to be a simpler reimagining of Amazon SQS.
It has an HTTP API and behaves mostly like SQS.
I wrote it to support Postgres, Microsoft’s SQL server and also MySQL because they all support SKIP LOCKED.
At some point I turned it into a hosted service and only maintained the Postgres implementation though the MySQL and SQL server code is still in there.
It’s not an active project but the code is at https://github.com/starqueue/starqueue/ https://github.com/starqueue/starqueue/
After that I wanted to write the worlds fastest message queue so I implemented an HTTP message queue in Rust. It maxed out the disk at about 50,000 messages a second I vaguely recall, so I switched to purely memory only and in the biggest EC2 instance I could run it on it did about 7 million messages a second. That was just a crappy prototype so I never released the code.
After that I wanted to make the simplest possible message queue so I discovered that Linux atomic moves are the basis of a perfectly acceptable message queue that is simply file system based. I didn’t put it into a message queue, but close enough to be the same I wrote an SMTP buffer called Arnie. It’s only about 100 lines of Python. https://github.com/bootrino/arniesmtpbufferserver https://github.com/bootrino/arniesmtpbufferserver