4 ms·

If someone want to hear some experience using NATS in production(this is based on NATS): * standalone server is mature and very stable, crunching messages with

If someone want to hear some experience using NATS in production(this is based on NATS): * standalone server is mature and very stable, crunching messages with incredible speed * server side error handling is ok, no problems here * very simple, text-based protocol, even simplier than STOMP * c, rust, java, python client libraries are weak; c library even contains some state transition errors when the library can lock on reconnect( https://github.com/nats-io/nats.c/issues/217 https://github.com/nats-io/nats.c/issues/217 ). (unofficial) rust library is unusable. some netowork errors will not make nats.c reconnect what can be a surprise in prod. * arbitrary hardcoded limit on the message size - 1M

ikozlovic 7y ago
Regarding the NATS C client and issue reported here, this is being worked on and should be addressed in v2.1.0.
KaiserPro 7y ago
We've been using NATS in prod for a very specific usecase: request reply. We'd got tired of trying to hack around SQS. After a boat load of testing, we settled on NATS. It was our big "innovation token" for that project. It has paid off wonderfully. The server clusters very well, and is very simple to peer. We've joined two clusters across two regions, and we've not had any issues. If you need a fast, low latentcy, non persistant, non guarenteed message delevery system NATS is for you. If you like to store messages in your queues for later consumption, or need _every_ message, NATS isn't for you.
- krz 7y ago
  What problems did you have with SQS?
  - KaiserPro 7y ago
    Nothing really, We were asking it to do something its not really supposed to Basically we needed a request reply semantic. (forgive the explanation) Its where we have an instance of the front end sending a request to any of the back end servers. A single backend replies directly to the same front end instance. Basically we needed something similar semantic to a load balancer, but in messaging form. SQS is only a 1-n Queue system. So we can deliver a message to a single backend in a cluster, but we couldn't get it back to the producer without making a message routing system. We used Dynamodb and repeated polling, but that wasn't cheap to scale. We might have been able use SNS as a broadcast bus of somesort, but that seemed like a backwards step. We still use SQS for virtually everything else. NATS is kept for that one use case.
    - iddqd 7y ago
      > Basically we needed something similar semantic to a load balancer, but in messaging form. Why not stick with a load balancer? I've been struggling to find a good use case for request-response over message queues so I'm curious.
      - KaiserPro 7y ago
        Let me try and draw a diagram: Api-gateway -> Lambda -> NATS -> docker backend on GPU The backend is C++ and needs to be shielded as much as possible from the web. It also takes a fair wedge of time to deal with requests (1-5 seconds) THe rationale is the Lambda front end validates and routes, the backend replies. Before we moved to NATs we had the added bonus that using SQS meant that there was no possible link between front-backend apart from SQS. (both lambda and SQS are outside of the VPC by default) the backend has a strict schema, and to keep moving parts down it seemed wise to avoid having to plumb in a webserver as well. They really are badly suited to this sort of thing. They also add a boat load of extra latency. We have a large number of machines, all of which deal with certain areas, This allows us to intelligently cache hotter areas closer to clients. Request reply is a reasonable pattern for interacting with backends that are constantly changing capacity based on demand. It also, if done correctly can be a way to optimise for latency over capacity (depending on how you do it.) With NATS you can ask for more than one response, which might also be useful. Some other systems use a "fastest reply wins" which guarantees a fast response at the expense of capacity.
atombender 7y ago
NATS is a solid piece of engineering. I rank it alongside Postgres and Memcached in terms of reliability and convenience. It just works. I have an app in production that publishes real-time events via NATS at high volume, and it has never failed. You do have to understand its design -- NATS can drop messages, and your application has to deal with this. One lesser-known feature of NATS is that it's embeddable -- if you're writing a Go app, you can use it as a library. You end up with a single statically linked binary that can do everything out of the box. Liftbridge looks like a neat way to add a layer of statefulness to NATS. I have an app that will need a log abstraction, and have been hoping to avoid integrating any oversized guns like Kafka (whose dependency on ZooKeeper makes it operationally more complex) and Pulsar for what is a fairly simple system.
derekcollison 7y ago
We are always interested in feedback on how we can do better. Feel free to jump on our slack channel to join the community. We are actively working on the C client. Interested in more details on Python and Java issues as well.

ikozlovic 7y ago

Regarding the NATS C client and issue reported here, this is being worked on and should be addressed in v2.1.0.

KaiserPro 7y ago

We've been using NATS in prod for a very specific usecase: request reply. We'd got tired of trying to hack around SQS. After a boat load of testing, we settled on NATS. It was our big "innovation token" for that project. It has paid off wonderfully. The server clusters very well, and is very simple to peer. We've joined two clusters across two regions, and we've not had any issues. If you need a fast, low latentcy, non persistant, non guarenteed message delevery system NATS is for you. If you like to store messages in your queues for later consumption, or need _every_ message, NATS isn't for you.

krz 7y ago
What problems did you have with SQS?
- KaiserPro 7y ago
  Nothing really, We were asking it to do something its not really supposed to Basically we needed a request reply semantic. (forgive the explanation) Its where we have an instance of the front end sending a request to any of the back end servers. A single backend replies directly to the same front end instance. Basically we needed something similar semantic to a load balancer, but in messaging form. SQS is only a 1-n Queue system. So we can deliver a message to a single backend in a cluster, but we couldn't get it back to the producer without making a message routing system. We used Dynamodb and repeated polling, but that wasn't cheap to scale. We might have been able use SNS as a broadcast bus of somesort, but that seemed like a backwards step. We still use SQS for virtually everything else. NATS is kept for that one use case.
  - iddqd 7y ago
    > Basically we needed something similar semantic to a load balancer, but in messaging form. Why not stick with a load balancer? I've been struggling to find a good use case for request-response over message queues so I'm curious.
    - KaiserPro 7y ago
      Let me try and draw a diagram: Api-gateway -> Lambda -> NATS -> docker backend on GPU The backend is C++ and needs to be shielded as much as possible from the web. It also takes a fair wedge of time to deal with requests (1-5 seconds) THe rationale is the Lambda front end validates and routes, the backend replies. Before we moved to NATs we had the added bonus that using SQS meant that there was no possible link between front-backend apart from SQS. (both lambda and SQS are outside of the VPC by default) the backend has a strict schema, and to keep moving parts down it seemed wise to avoid having to plumb in a webserver as well. They really are badly suited to this sort of thing. They also add a boat load of extra latency. We have a large number of machines, all of which deal with certain areas, This allows us to intelligently cache hotter areas closer to clients. Request reply is a reasonable pattern for interacting with backends that are constantly changing capacity based on demand. It also, if done correctly can be a way to optimise for latency over capacity (depending on how you do it.) With NATS you can ask for more than one response, which might also be useful. Some other systems use a "fastest reply wins" which guarantees a fast response at the expense of capacity.

atombender 7y ago

NATS is a solid piece of engineering. I rank it alongside Postgres and Memcached in terms of reliability and convenience. It just works. I have an app in production that publishes real-time events via NATS at high volume, and it has never failed. You do have to understand its design -- NATS can drop messages, and your application has to deal with this. One lesser-known feature of NATS is that it's embeddable -- if you're writing a Go app, you can use it as a library. You end up with a single statically linked binary that can do everything out of the box. Liftbridge looks like a neat way to add a layer of statefulness to NATS. I have an app that will need a log abstraction, and have been hoping to avoid integrating any oversized guns like Kafka (whose dependency on ZooKeeper makes it operationally more complex) and Pulsar for what is a fairly simple system.

derekcollison 7y ago

We are always interested in feedback on how we can do better. Feel free to jump on our slack channel to join the community. We are actively working on the C client. Interested in more details on Python and Java issues as well.