10 ms·
High-Performance server for NATS.io, the cloud and edge native messaging system
- jitl 3y agoWith all the message queue whatever thingies, as an outside I’m quite confused about ideal use cases. NATS vs MQTT vs Kafka vs Redis Queue vs Amazon SQS - how do they all stack up?
- Xeoncross 3y agoQueues are often used for two reasons: as a way to throttle processing (just throw it into the queue and drain as you have time) and a way to avoid state loss (process the queue item and only ack/clear it once you've done X, Y, & Z. Of course there are more uses, but these are the two most common. Depending on the throughput you need, the number of clients pulling/putting into the queue, and other things you can rule out certain options. For example, Redis queue will be one of the fastest choices - but is very limited in capacity (memory size). SQS is basically unlimited capacity, but can often have 150ms or more of time elapse between when you put the item in and when it's available.
- paulgb 3y agoMQTT is a protocol rather than an alternative to NATS; NATS supports MQTT (https://docs.nats.io/running-a-nats-service/configuration/mqtt https://docs.nats.io/running-a-nats-service/configuration/mq...) As far as ideal use cases, we use NATS for https://plane.dev https://plane.dev in two ways: - As a message bus, it is a layer of abstraction on top of the network. Instead of each node needing to establish a connection to every node it needs to connect to, it just connects to a NATS cluster and messages are routed by subject. This is great for debugging because we can "wiretap" messages on a given subject pattern and verify what's being sent. We even have a service that listens on NATS subjects and conditionally turns events into Slack messages. - It has a built-in RAFT implementation (via JetStream), which we piggyback on when we need to create consensus among nodes.
- atombender 3y agoNATS is not a queue. It's distributed pub/sub message broker for communicating between applications. NATS's only responsibility is to route messages in near-real-time from publishers to consumers. Messages are ephemeral and dropped immediately after delivery; if nobody is listening, the messages vanish. Messages are only queued temporarily in RAM if the consumer is busy, and they can get dropped if a consumer doesn't handle them fast enough. In short, NATS is very lightweight and fast, and designed for things that are lightweight and fast. It's like a kind of distributed socket mechanism, and works best as a communication primitive you build stuff on top of (like TCP or UDP) rather than a fully fledged system. So it's very different from Kafka and other types of queues that are durable and database-like. Kafka is good for "fat pipes" that centralize data from producers into a log which is then consumed by massively parallel sets of consumers, and you don't constantly change this topology. NATS is good for networks of fast-changing producers and consumers that send small messages to each other, often one-on-one, although any fan-out topology works. It's great for firehose-type routing. For example, imagine you want your app to produce lots of different kinds of telemetry. Your app just sends messages to a topic "telemetry" or maybe a dotted topic like "telemetry.iostats" or "telemetry.errors". Then a client can "tap" into that topic by listening to it. If no client is listening, the firehose goes nowhere. But then a client can tap into "telemetry.errors" and get just the stream of error messages. Topics are just strings, so you can create unique topics for temporary things; an app can send a message to another app like "hey, do some work and then send the result to my temporary topic foobar726373". NATS is particularly notable for its "just works" design. The clustering, for example, ties together brokers with no effort. Clients typically don't need any configuration at all, other than the name of a NATS server. NATS can be used as a low-level component to build stateful stuff. NATS Jetstream is a Kafka-like solution that stores durable logs, and uses NATS as its communication protocol. Liftbridge is another one.
- zerbinxx 3y agoIt is worth noting that the jetstream API (to me) (not that experienced) to lack some of the features of Kafka w.r.t. replayability - for instance I can’t easily say “go back and re-run messages from X point-in-time”. Instead it may be necessary to write custom handlers for replay or safeguard your systems to be very idempotent (a good pattern in event driven systems, but not one that is explicitly required by Kafkaesque messaging systems).
- sofixa 3y ago> NATS vs MQTT vs Kafka vs Redis Queue vs Amazon SQS - how do they all stack up I'm curious why are RabbitMQ and AMQP no longer part of such comparisons (not only your comment, nobody else on this thread has mentioned them).
- jitl 3y agoAt my past job RabbitMQ was an infrequent source of painful surprises. I think it's kinda like HAProxy, a venerable solution that's well known but being replaced in new work more trendy implementations of the same kind of thing.
- sofixa 3y agoInteresting, at my past job we used RabbitMQ at decent scale and it was rock solid once the initial kinks around scaling were ironed out (there weren't enough resources for the amount of messages that were thrown at it).
- pnpnp 3y agoI have been a huge advocate of NATS. For anyone looking to support multiple message patterns on one message bus, this is what you want to check out. In AWS terms, it’s like SNS/SQS/Kinesis all rolled into one bus & very intuitive to work with.
- vosper 3y agoAny thoughts on NATS' JetStream? Looks quite compelling, to me. https://docs.nats.io/using-nats/developer/develop_jetstream https://docs.nats.io/using-nats/developer/develop_jetstream
- protochron 3y agoJetstream is the Kafka or Kinesis bit built out on top of the NATS core protocol. It's much easier to work with than either of them in my experience
- vosper 3y agoThanks, do you know if it's a good idea to use JetStream's KV or Object stores to implement a join? I'm thinking of "fan-in", where multiple streams carrying different pieces of data about the same object all meet in the same place, and the output is a unified object carrying all those bits of data. Seems the store could be used by the joining consumer to hold onto the various pieces of data related to an object until it has the full set, then emit the object into the next stream?
- yes29922 3y agoAt this point, where you are doing KSQL level stuff, isn't it better to just use Kafka, NATS is very appealing for it's simplicity but if I reach a point of re-implementing KSQL or spending 3 hours setting up zookeeper and understanding how to do Kafka distributed for a devops point then Im choosing the latter
- 3y ago
- clumsysmurf 3y agoI'm curious how this compares with MQTT, there seems to be some overlap. I was looking to stream sensor data from a mobile device using MQTT, but the Eclipse Paho Java client (1.2.5) hasn't seen a release in 3 years and I found it to be pretty buggy. There are lots of open issues on the GitHub page.
- ozarker 3y agoMQTT is a protocol. Nats is a server that supports MQTT among other things
- xupybd 3y agoI'm using this in production. So far it's worked very well. Took a few hours to setup with certificates. Now it just works.
- zerbinxx 3y agoWhat do you run it on?
- xupybd 3y agoWe have a manufacturing operation with a public B2B ecommerce platform. It's just a dumb communications buss for us. It's replaced most of our ETL needs with Realtime events. If an order is placed online an event is raised. Our accounting system can consume that event to create the sales order. Then the production system consume the same event to add the job to the next production batch. Each production step produces events that can be used to update other systems, including Realtime updates on the ecommerce system. We use Jetstream so consumers don't need to be awake when producers create events. This system spans the cloud and 3 physical locations. But, to the consumers and producers it's one buss that they only have to authenticate with once.
- gjvc 3y agonot to be confused with https://www.nats.aero/ https://www.nats.aero/
- zmmmmm 3y agoStruggling to figure out what "cloud native" actually means. Unfortunately their own web site doesn't say and the CNCF web site they link to leaves me none the wiser. At a guess they are talking about applications being built from the ground up to dynamically allocate resources using cloud providers APIs directly rather than relying on an assumption fixed resources are already provisioned and the application runs within them. I wonder if I'm right ...
- deleted 3y ago[deleted]
- jerrygenser 3y agoThere are a bunch of OSS that solve problems when running distributed scale out workloads in "the cloud". For example prometheus is a metric db for storing monitoring data. Ceph is a storage backend. Most of them tend to be built for or work well with kubernetes.
- re-thc 3y ago> Struggling to figure out what "cloud native" actually means. - very fast startup - low memory - can be easily distributed or is stateless
- xuancanh 3y agoThat is not what cloud native means. It is an overloaded term, but the basic idea is that the application is designed to operate in the cloud environment, utilizing cloud's capabilities such as elastic and automated infrastructure management.
- re-thc 3y ago> That is not what cloud native means. > utilizing cloud's capabilities such as elastic and automated infrastructure management. Contradictory perhaps? So it's not what it means but it is what it means? In order to achieve what you've said in vague terms you definitely do need to try to be stateless, have fast startup and reduce memory. Clear examples are things like lambda and fargate. How can you be elastic if you have an application server that takes 10 minutes to start?
- lakomen 3y agoI got into it again about 2 weeks ago. Been doing a small project with the microservice api. Compared to HTTP/REST I miss the debugabilty of it. Since all messages are sent over websocket and received, in binary form, since there is no support from Chrome or Firefox, one has to tediously manually extract the json payload and try to make sense of it. Jetstream is still quite the mystery. It just doesn't want to work the way I want it to, especially on the JS client side, blocking script execution and timing out. Then there's auth, after a few shutdowns and reboots I got locked out of my local installation. Oidc subject to nats user mapping doesn't exist and has to be done manually. So TL;DR the core functionality is great. Everything else seems to be WIP.
- devmunchies 3y agoYou can subscribe to all topics by using a *. Why not have a debug subscriber that listens to all messages and plays it in the terminal? I agree that the auth system is cumbersome. I wanted to use it on the edge for IoT devices where the device only has the same permissions as the user who the device belongs to, but not very easy. Their auth isn’t very customizable.
- lakomen 3y agoThis is actually a good idea, I'll try that, thanks :) But, you know, it's still a bit more complicated than opening the browser's debug console and inspecting request and response in the networking tab. Yeah you have operators, which are essentially the auth admins or orgs, however you'd like to look at it, then there are accounts which are an alias for "project name" and then you have users, which are "client name"s. And that's fine for their infrastructure only. The problem is, in the real world you have an external identity service (OIDC, IAM,...) and the JWT this service creates includes a subject, but the NATS auth system has no support for external bridges. Also you have to decide, would you like to use operator/account/user or single JWT, or passphrase or manually managed user/password accounts? In a professional world, you'd have various customer applications each divided by their operators, accounts and users. And then fine grained user capabilities.. headache time. So what you have to do is distribute a default user certificate with a client (client meaning actual client in the client/server sense), then do the signin/sso process and have some middleware checking this token from the OIDC auth process and check is a user with this subject exists, if not, create a new user and cert and send that cert back to the client which will then create a new connection with the new certificate. Very complicated. The initial default client certificate is required for "logging in" via NATS. I guess it could be a standalone program, the oidc to nats user mapper/manager. So compared to HTTP/REST, more work involved.
- npace12 3y agonats is phenomenal. Libraries and docs are not great, but you won't need them much as you just forget it's even there.
- protochron 3y agoI work quite a bit with CNCF wasmCloud (https://wasmcloud.com/ https://wasmcloud.com/) and that project wouldn't exist without NATS. You can think of wasmCloud as a distributed compute lattice using WebAssembly on the server for providing the compute and NATS to provide the interconnectivity between your various WebAssembly modules. NATS certainly has its quirks, but I can't recommend it highly enough if you need any sort of pub/sub or stream processing. It even has built-in key-value and object storage for when you need to store larger messages or content. I definitely prefer Jetstream to Kafka in pretty much every use case I can think of. At my current employer (https://cosmonic.com https://cosmonic.com) we use NATS not only for wasmCloud, but we also stream log data and metrics and it keeps up with everything we throw at it with a very low footprint. Auth is kind of counterintuitive until you've spent some time with it, but NATS provides you with a ton of flexibility (docs here: https://docs.nats.io/running-a-nats-service/configuration/securing_nats/auth_intro/jwt https://docs.nats.io/running-a-nats-service/configuration/se...). https://natsbyexample.com/ https://natsbyexample.com/ is a great resource, and can do a better job than I can in illustrating the various ways NATS can be used along with different deployment topologies.
- no_circuit 3y agoTIBCO Rednezvous, https://www.tibco.com/products/tibco-rendezvous https://www.tibco.com/products/tibco-rendezvous, is the first thing that came to my mind from previous experience in the financial industry working with real-time market data. Although I'm not sure if it has built-in KV support for dealing with large payloads like NATS does, TIBCO RV and their related software packages are worth checking out to see what an long time established commercial product offers. Which leads me to... ... the protocol is text-based like HTTP with CR LF for field both for the client, https://docs.nats.io/reference/reference-protocols/nats-protocol#client-protocol https://docs.nats.io/reference/reference-protocols/nats-prot..., and cluster protocols, https://docs.nats.io/reference/reference-protocols/nats-server-protocol https://docs.nats.io/reference/reference-protocols/nats-serv... -- which means encoding overhead if your payloads are binary. So depending on your definition of performance, ymmv. I really do not see how implementing an API across multiple languages is easier by making a new linefeed-based protocol, https://github.com/nats-io/nats-server/blob/0421c65c888bf38125b7e8aac006d6e00d7a8359/server/parser.go https://github.com/nats-io/nats-server/blob/0421c65c888bf381..., than just using code-generated JSON or gRPC (Protobuf or Flatbuffers). One could then write subscriptions/clustering algorithms in a protocol-neutral library.
- c-cube 3y agoI don't think there's encoding overhead. It's like http, like you said, but just like http body encoding it seems that actual payloads are preceded with their length so that you just read n followed by n bytes. Look at the docs for `pub` for example. Json would be less efficient; gRPC adds tons of complexity and overhead.
- klabb3 3y agoCorrect. Text based is more than fine these days, as long as you use length prefix so you can avoid escaping. But you do still need to parse lines which can be tricky in bare or poorly tooled environments. It’s the best trade off today, because debugging binary formats is a serious obstacle. > Json would be less efficient; gRPC adds tons of complexity and overhead. Indeed. There aren’t many suitable specs around, and this protocol, albeit custom, is very easy to implement. Which is proven by the fact that there are well maintained Nats clients in many different languages.
- jensneuse 3y agoWe've recently built an adapter for NATS KV so you can easily expose it through an HTTP-based GraphQL API. (https://docs.wundergraph.com/docs/databases/kv https://docs.wundergraph.com/docs/databases/kv) We're also looking at supporting streams in the future. What's great is that NATS is written in go and we can easily embed it for testing and dev purposes. Furthermore, Synadia makes it super easy to run NATS across multiple regions.
- throw14082020 3y agoWould it ever make sense for NATS to support UDP or SRTP?
- throw14082020 3y agoAhh, they may work on QUIC this year: https://github.com/nats-io/nats-server/issues/457 https://github.com/nats-io/nats-server/issues/457
- gedw99 3y agoi think they are waiting for QUIC to be merged into golang
- debuggerpk 3y agoBig fan of nats, especially how they are doing tenant isolation and geo-distributed queue.
- ensocode 3y agoWhat are good resources to learn about this new trend of using messaging? We already had this stuff MoM (message oriented middle ware). Pretty interesting.
- evnix 3y agoWe have had pretty bad experience with NATS. Apart from core NATS which is just in memory, you need their sub modules to build any kind of serious business on top of it. We initially built our code around NATSStreaming, they then went ahead and deprecated that. But since we saw so many big companies using NATS, they thought it might be a good idea to stick with it and we did a year long migration to their shiny new NATSJetstream Push based approach, but from what I see now in the conversations they are going to deprecate that too in favour of Pull based approach which is architecturally very different, now we will have to somehow convince management for another rewrite. I am not sure if we should even rewrite or just move to another product at this point. Dear NATS, please stop throwing away and rewriting protocols and products. Or make it such that the end client libraries would handle that upgrade automatically with a library upgrade. We should have just stuck with the more traditional Kafka or RabbitMQ. What I have also learnt is that when companies put big brand logos on their websites, it just means some random Dev from that company is using it for their side project or experimental mini project.
- Kinrany 3y agoFrom what I've seen it's too early to migrate from Push. With the last update they emphasized Pull as the default, but Push is not deprecated yet. And even NATS Streaming still works: it is deprecated, not removed. Unlike databases, a good thing about messaging and streaming solutions is that you don't have to pick one: you can make them talk to each other as long as there are bridges. This also applies to different approaches to messaging/streaming provided by a single platform.
- bzzzt 3y agoJust like you don't want to support tens of different databases you shouldn't build your solution on lots of bridged message busses. If stuff goes down at 2 am you don't want to have to find out which message queue was the problem or what tooling you should use to debug it.
- Kinrany 3y agoSure, but it's an inconvenience, not a total blocker. With databases you lose foreign keys and you have to replace a single query with N. And you don't want to have ten different solutions of course, but two is fine and allows for comfortable migrations.
- jonathanstrange 3y agoCan someone tell me which transport protocols and ports this uses or can be configured to use? Moreover, do clients have NAT traversal capabilities (no pun intended)? This is always so hard to figure out for B2B libraries.
- rcombatwombat 3y agoIt supports clients connecting over TCP (which can be encrypted) and Websockets (can also be encrypted), you can configure it to pick on any port you want and it works through Network Address Translation (see https://docs.nats.io/running-a-nats-service/configuration#connectivity https://docs.nats.io/running-a-nats-service/configuration#co...). Even supports MQTT clients directly (https://docs.nats.io/running-a-nats-service/configuration/mqtt https://docs.nats.io/running-a-nats-service/configuration/mq...)