3 ms·

Having both postgres and rabbitmq strikes me as overkill - I wonder if they'd be open to a PR to harmonize to just PG, slimming down the ops burden. I'll have t

Having both postgres and rabbitmq strikes me as overkill - I wonder if they'd be open to a PR to harmonize to just PG, slimming down the ops burden. I'll have to dig into it when I'm back on my desktop to see what exactly RMQ is doing for a media server

deleted 3y ago
[deleted]
zoriya 3y ago
rabbitmq is used to communicate between services. I just introduced it but it will be used to: - handle websockets communication with the client - create workqueues for new items creation/rescan requests - Watch list syncing with external services - and to synchronize different replicas when deployed on k8s (still need work on this point tho)
- JeremyNT 3y ago
  I'm not here to talk you out of rabbitmq now that you've already added it for this project, but IMO it's worth taking a look at PG notify/listen and keeping it in mind for simple use cases. It's not nearly as powerful but it can be a pretty good starting point before pulling in more dependencies.
- cchance 3y ago
  Are you running your own websocket server or just spinning up soketi in a container?
  - zoriya 3y ago
    For now, I'm not doing anything, websocket is next-release big change. I did not know about soketi tho, i'll take a look.
    - eyegor 3y ago
      If you're in dotnet land, I'd recommend looking at magic onion [0] or memorypack [1]. It'll be much nicer to work with once you have a decent pile of message types. Otherwise it's easy to end up in an entity framework like situation where you're constantly serializing and deserializing, generating serializer wrappers, doubling up on dtos and schemas, etc. Although any of the cysharp libs are great in dotnet, definitely recommend checking out their back catalogue. [0] https://github.com/Cysharp/MagicOnion https://github.com/Cysharp/MagicOnion [1] https://github.com/Cysharp/MemoryPack https://github.com/Cysharp/MemoryPack
roomey 3y ago
Rabbit mq is great. I found it particularly useful when some parts of a program are inherently slower than other parts. It offers a very robust and tested queuing system, and is pretty tiny to run. It lets you keep things very simple.
- NortySpock 3y ago
  How tiny is "pretty tiny" in your opinion? I've been playing around with NATS, which is hovering at just under 10MB of RAM on each of 3 nodes, while passing an average of 1 message per second. I am liking it, though NATS clearly has a less strict queue ordering policy when I throw a burst of messages at it. Edit: actually another node is at 22MB, so take what you will from that.

deleted 3y ago

[deleted]

zoriya 3y ago

rabbitmq is used to communicate between services. I just introduced it but it will be used to: - handle websockets communication with the client - create workqueues for new items creation/rescan requests - Watch list syncing with external services - and to synchronize different replicas when deployed on k8s (still need work on this point tho)

JeremyNT 3y ago
I'm not here to talk you out of rabbitmq now that you've already added it for this project, but IMO it's worth taking a look at PG notify/listen and keeping it in mind for simple use cases. It's not nearly as powerful but it can be a pretty good starting point before pulling in more dependencies.
cchance 3y ago
Are you running your own websocket server or just spinning up soketi in a container?
- zoriya 3y ago
  For now, I'm not doing anything, websocket is next-release big change. I did not know about soketi tho, i'll take a look.
  - eyegor 3y ago
    If you're in dotnet land, I'd recommend looking at magic onion [0] or memorypack [1]. It'll be much nicer to work with once you have a decent pile of message types. Otherwise it's easy to end up in an entity framework like situation where you're constantly serializing and deserializing, generating serializer wrappers, doubling up on dtos and schemas, etc. Although any of the cysharp libs are great in dotnet, definitely recommend checking out their back catalogue. [0] https://github.com/Cysharp/MagicOnion https://github.com/Cysharp/MagicOnion [1] https://github.com/Cysharp/MemoryPack https://github.com/Cysharp/MemoryPack

roomey 3y ago

Rabbit mq is great. I found it particularly useful when some parts of a program are inherently slower than other parts. It offers a very robust and tested queuing system, and is pretty tiny to run. It lets you keep things very simple.

NortySpock 3y ago
How tiny is "pretty tiny" in your opinion? I've been playing around with NATS, which is hovering at just under 10MB of RAM on each of 3 nodes, while passing an average of 1 message per second. I am liking it, though NATS clearly has a less strict queue ordering policy when I throw a burst of messages at it. Edit: actually another node is at 22MB, so take what you will from that.