4 ms·

Show HN: StarQueue – a free hosted HTTP message queue with a Postgres back end

43 points by andrewstuart 5y ago

andrewstuart 5y ago
Hi folks OP here. Here's my side project StarQueue which is a free message queue. I built it because I wanted to see if it was possible to implement something similar to SQS using Postgres as a back end. I actually implemented the back end in Postgres, MySQL and also SQL server and they all work fine. I chose Postgres in the end because of personal preference. I tried to implement the same back end in Oracle and the experience or using Oracle was horrifying so I dropped that. I tried to implement for IBM DB2 but that got hard for reasons I can't recall so I dropped that too. It's written in async Python using the Starlette framework. It's in beta at the moment. Let me know if you notice any problems.
- dissent 5y ago
  Is it too late to rename it FarQueue?
  - andrewstuart 5y ago
    Ha ha! I did consider “NearQueue”.
    - grzm 5y ago
      I encourage you to nickname it Tony.
- mdaniel 5y ago
  Congratulations on your launch, and for providing a public utility :-) I would love to see metrics on how hard the wild Internet makes your python and PG work, if you wanted to create /stats.html one day in the future It wasn't clear if you wanted commentary here, or via your contact form, so ... I was surprised that the JWT is being sent as a POST param, instead of its more traditional place in an Authorization header Kind of related, kind of not: I was also surprised to see the API request include the accountId in the URI when it's present in the JWT -- is that an implementation concession allowing a future where one AccountId (from the auth perspective) is able to control multiple "{accountId}" URIs? Definitely related to that: I was surprised to see the QueuesList response send back what I presume is the same accountid that was present in the URI, repeated for every QueueName I think the MessageReceive example copy-pasted the "MessageBody" field from its MessageSend friend, and the MessageDelete example is missing its "ReceiptHandles" that is mentioned in the description Good luck on your adventure!
  - andrewstuart 5y ago
    Great questions. I added the JWT late in the project and it needed the accountid in it which explains the duplication. As to why the token is in the body not the header, honestly it was expedience. Thanks for your thoughtful analysis.
encoderer 5y ago
Hey, nice work on this. Projects like this can be excellent for learning, landing a good job, and trying out new tech. I was surprised to not see a Github link. Are you planning to open source it?
- andrewstuart 5y ago
  First step was to get it running hosted and see if anyone is interested. It’s such a huge amount of work to get it to this point I had to stop at some point and say “make it public”. The original plan was open source only, not hosted so yes likely I’ll open source it at some point if people find it interesting.
  - encoderer 5y ago
    Right on, congrats on getting it there, I do know how hard that is.
tomberek 5y ago
Possible to receive messages using SSE (server-sent events)? I implemented it with a Postgres backend for a project once, but it was not a general solution. SSE provides many niceties for a queue like this.
- andrewstuart 5y ago
  I’m interested to hear more about what SSE benefits would be. I have looked into them a little. Are you familiar with long polling? It’s very efficient for a lot of applications. I’d be interested to hear what further benefit SSE would add.
  - SahAssar 5y ago
    With SSE you don't need to send a new request after you have received data to receive more. SSE also defines a format that allows for pings and the server to define how/when the client should retry failed connections.
brunojppb 5y ago
This looks really impressive. Congrats on the Launch! Minor nit: On the `div`s with the class name `code`, I see the curl demo is overflowing its contents. It could be fixed by adding this to its css: ``` overflow: scroll; ```
piterrro 5y ago
Hi OP congrats on launching the project, I think we share similar idea, my project is https://qutrunk.com https://qutrunk.com which is basically a queue with Http interface. It comes with multiple backends, so far its Rabbitmq and Mongo. I also plan to provide gRPC interface.

43 points by andrewstuart 5y ago

Hi folks OP here. Here's my side project StarQueue which is a free message queue. I built it because I wanted to see if it was possible to implement something similar to SQS using Postgres as a back end. I actually implemented the back end in Postgres, MySQL and also SQL server and they all work fine. I chose Postgres in the end because of personal preference. I tried to implement the same back end in Oracle and the experience or using Oracle was horrifying so I dropped that. I tried to implement for IBM DB2 but that got hard for reasons I can't recall so I dropped that too. It's written in async Python using the Starlette framework. It's in beta at the moment. Let me know if you notice any problems.

dissent 5y ago
Is it too late to rename it FarQueue?
- andrewstuart 5y ago
  Ha ha! I did consider “NearQueue”.
  - grzm 5y ago
    I encourage you to nickname it Tony.
mdaniel 5y ago
Congratulations on your launch, and for providing a public utility :-) I would love to see metrics on how hard the wild Internet makes your python and PG work, if you wanted to create /stats.html one day in the future It wasn't clear if you wanted commentary here, or via your contact form, so ... I was surprised that the JWT is being sent as a POST param, instead of its more traditional place in an Authorization header Kind of related, kind of not: I was also surprised to see the API request include the accountId in the URI when it's present in the JWT -- is that an implementation concession allowing a future where one AccountId (from the auth perspective) is able to control multiple "{accountId}" URIs? Definitely related to that: I was surprised to see the QueuesList response send back what I presume is the same accountid that was present in the URI, repeated for every QueueName I think the MessageReceive example copy-pasted the "MessageBody" field from its MessageSend friend, and the MessageDelete example is missing its "ReceiptHandles" that is mentioned in the description Good luck on your adventure!
- andrewstuart 5y ago
  Great questions. I added the JWT late in the project and it needed the accountid in it which explains the duplication. As to why the token is in the body not the header, honestly it was expedience. Thanks for your thoughtful analysis.

encoderer 5y ago

Hey, nice work on this. Projects like this can be excellent for learning, landing a good job, and trying out new tech. I was surprised to not see a Github link. Are you planning to open source it?

andrewstuart 5y ago
First step was to get it running hosted and see if anyone is interested. It’s such a huge amount of work to get it to this point I had to stop at some point and say “make it public”. The original plan was open source only, not hosted so yes likely I’ll open source it at some point if people find it interesting.
- encoderer 5y ago
  Right on, congrats on getting it there, I do know how hard that is.

tomberek 5y ago

Possible to receive messages using SSE (server-sent events)? I implemented it with a Postgres backend for a project once, but it was not a general solution. SSE provides many niceties for a queue like this.

andrewstuart 5y ago
I’m interested to hear more about what SSE benefits would be. I have looked into them a little. Are you familiar with long polling? It’s very efficient for a lot of applications. I’d be interested to hear what further benefit SSE would add.
- SahAssar 5y ago
  With SSE you don't need to send a new request after you have received data to receive more. SSE also defines a format that allows for pings and the server to define how/when the client should retry failed connections.

brunojppb 5y ago

This looks really impressive. Congrats on the Launch! Minor nit: On the `div`s with the class name `code`, I see the curl demo is overflowing its contents. It could be fixed by adding this to its css: ``` overflow: scroll; ```

piterrro 5y ago

Hi OP congrats on launching the project, I think we share similar idea, my project is https://qutrunk.com https://qutrunk.com which is basically a queue with Http interface. It comes with multiple backends, so far its Rabbitmq and Mongo. I also plan to provide gRPC interface.