13 ms·

Honestly, I'm more interested in auth callout, someone making a "how to use this stuff effectively" video and/or article, for dummies, ideally with keycloak. I

Honestly, I'm more interested in auth callout, someone making a "how to use this stuff effectively" video and/or article, for dummies, ideally with keycloak. If you look at [0] how does this explain anything? Seems like I need to write a service utilizing NATS which talks to the OIDC server. NATS auth story is a complicated one, and now with auth callout it's even more complicated. What I also quite didn't succeed in were JS client Jetstream consumers, async polling or w/e. All the examples, async or sync would block execution. The regular request/response with microservices went well and straightforward. No issues at all. But writing middleware is also not very pretty, you have to have dedicated functions which you would call in the request handler (or microservice if you will) instead of being able to define a chain of middleware functions. Potejto potato but organization and structure and naming is half the code. And more now you can't rely on http error codes, even if you can use them. Then debugging is difficult, because messages are sent in binary format, and with no middleware to conditionally plug in, it's even more painful. Yeah you could write a dedicated service listening to >.* and printing deserialized JSON payloads but that's also extra work. However dropping the HTTP overhead leads to a lot more throughput. For now I'll stay with HTTP. The reason I experimented with it was there is a company which required NATS knowledge. [0] https://docs.nats.io/running-a-nats-service/configuration/securing_nats/auth_callout https://docs.nats.io/running-a-nats-service/configuration/se...

wtatum 3y ago
Same. I tried out some toy use-cases using nats.js over websocket a few months ago. The prospect of possibly being able to "directly" consume messaging or key/value store from the browser with only a thin gateway between was really interesting to me but I couldn't square up the NATS-internal JWT cookie thing with how you would handle auth in a traditional web-app (OAuth client on a gateway plus a session cookie). I found some threads saying auth callout in 2.10 would solve this and decided to table the project until 2.10 but it's really really unclear how to work through the details of converting a "traditional" OAuth access token into the NATS-specified access token required by the auth callout contract.
- bruth 3y ago
  Auth callout is post-NATS client authentication, so it would not solve the "auth web flow" for authentication. Instead, the resulting token from that would be set as a cookie that then would be passed into the nats.ws client connection. The auth callout service would use that token to map to the concrete NATS user. The mechanism of doing that is up to the implementation. One option is to manage NATS claims into the OIDC provider (for the user authenticating) and then the auth service would decode that source JWT and extract the NATS claims and generate the NATS user JWT in the response.
  - wtatum 3y ago
    Thanks that observation is extremely helpful. If I have it down then the intended flow looks something like? - Web client is directed to some token vending service. This service implement authn in a manner of its choosing (i.e. OAuth) then sets a NATS client JWT in the cookie per https://docs.nats.io/running-a-nats-service/configuration/securing_nats/auth_intro/jwt https://docs.nats.io/running-a-nats-service/configuration/se... - Nats.ws client connection provides cookie during connection to perform client auth - If further authz/fine-grained control is needed the auth callout mechanism can be used. This would have access to the provided cookie/token so any claims needed for access control could be stapled on during step one and used at this point? For GPs original question -- I'm running a fairly old Keycloak version (v8) but it does appear to set a JWT in KEYCLOAK_IDENTITY and KEYCLOAK_IDENTITY_LEGACY. Am I right in understanding that IFF the token is signed with Ed25519 and both sub and iss are an NKEY value this is sufficient for NATS to accept that cookie as a credential?
    - bruth 3y ago
      Yes that reads correct. The `sub` would a NATS user public nkey, the `iss` would be the NATS account public nkey (either the issuer nkey in config-mode or existing nkey in decentralized auth). As long as it can verify the chain of trust for the user JWT that is returned, it should work. The three schema types are shown here: https://docs.nats.io/running-a-nats-service/configuration/securing_nats/auth_callout#schema https://docs.nats.io/running-a-nats-service/configuration/se... auth request comes in -> generate user jwt, sign + encode -> respond with auth response. As long as the necessary bits of the response and user JWT conform, it will work.
bruth 3y ago
> Seems like I need to write a service utilizing NATS which talks to the OIDC server. Auth callout was designed to be a generic extension point to delegate authentication and generate dynamic a user JWT that NATS understands (permissions, limits, etc). It enables an arbitrary backend to be integrated with, not tied specifically to OIDC. But indeed, this requires implementing a service that does this integration. > NATS auth story is a complicated one, and now with auth callout it's even more complicated. There is a spectrum of auth options, starting with simple config-based, token or user/pass leading up to decentralized auth for use cases that need it. Auth callout is an opt-in thing, so it should only be adopted if it is truly necessary. > you have to have dedicated functions which you would call in the request handler [...] instead of being able to define a chain of middleware functions I don't quite understand this statement. Wrapping a NATS handler is the same approach as wrapping an HTTP handler (within the same client app). The function would take a handler and return a handler. There can be inspection of the message within that function and the choice of calling the next handler, responding early, doing some external call, or doing nothing. If you want to distribute this middleware, then you would need to have separate, explicit subjects that each middleware would subscribe to and then publish to for the next element in the chain (for a choreography approach). There is also the "message slip pattern" where the ingest component sets the path as metadata (e.g. headers) that each middleware component uses that to know (at request time) which subject to publish the result to next.
- lakomen 3y ago
  Regarding middleware, last time I tried there was a lot of information to process. Now I tried again and func loggingMiddleware(next micro.Handler) micro.Handler { return micro.HandlerFunc(func(req micro.Request) { spew.Dump(req.Data()) next.Handle(req) }) } I'll remove the middleware part. EDIT: Seems I can't edit the post anymore. Regarding auth, yes, spectrum of options, but for "production" use it's very complicated and you don't want a shared key, you need the whole operator, account, user thing, and external auth aka auth callout. The documentation is lacking. nkey, xkey what? It's confusing is complicated. Managing auth is a pain, more than once I've been locked out of my test server because of how complicated auth is and had to wipe it. It's so complicated because you have have isolated "namespaces" or operator spaces? Naming an organization operator, or a solution or service group doesn't help. user is synonymous with client. All that you don't get from the documentation, you have to watch Youtube videos to understand it. The documentation is bad. How do I consume a stream from the deno/ws client without the whole client app blocking? No idea. The code snippets don't help. All there is, is a pay for company that offers to host NATS clusters, which you can pay to ask. Slack is not a good medium. I'm guessing that the documentation is bad so one has to pay for support. And that's a no-go for me. I get that you spent time and effort into it, I'm assuming you're one of the NATS people, but what good is it when I don't know how to effectively use the software? The natsbyexample page is also hard to digest. You're in your own little bubble and expect people to be mind readers. If it's too complicated people won't adopt it. I know I won't because it's too complicated. I would've loved to build upon it, because the core functionality is nice and works, easy to grasp. I've not seen a single Jetstream stream and consumer tutorial with the new API and nats.ws and Go. Old API sure, STAN sure. New API no. Some real world stuff, not just code demo snippets. Why not? Because it's complicated. Auth callout, that was the big thing everyone was waiting for? I would work on actually solving problems instead of building components so I can use this extra piece of software who's documentation is so bad. You know how many Kafka tutorials there are? A LOT. I want to like it, it seems very promising, but it suffers from the "coder no likey documentation" illness. Today I saw a Twitch stream of some Netflix dev, talking about DX, developer experience. NATS' DX has a lot of room for improvement, to put it friendly.

wtatum 3y ago

Same. I tried out some toy use-cases using nats.js over websocket a few months ago. The prospect of possibly being able to "directly" consume messaging or key/value store from the browser with only a thin gateway between was really interesting to me but I couldn't square up the NATS-internal JWT cookie thing with how you would handle auth in a traditional web-app (OAuth client on a gateway plus a session cookie). I found some threads saying auth callout in 2.10 would solve this and decided to table the project until 2.10 but it's really really unclear how to work through the details of converting a "traditional" OAuth access token into the NATS-specified access token required by the auth callout contract.

bruth 3y ago
Auth callout is post-NATS client authentication, so it would not solve the "auth web flow" for authentication. Instead, the resulting token from that would be set as a cookie that then would be passed into the nats.ws client connection. The auth callout service would use that token to map to the concrete NATS user. The mechanism of doing that is up to the implementation. One option is to manage NATS claims into the OIDC provider (for the user authenticating) and then the auth service would decode that source JWT and extract the NATS claims and generate the NATS user JWT in the response.
- wtatum 3y ago
  Thanks that observation is extremely helpful. If I have it down then the intended flow looks something like? - Web client is directed to some token vending service. This service implement authn in a manner of its choosing (i.e. OAuth) then sets a NATS client JWT in the cookie per https://docs.nats.io/running-a-nats-service/configuration/securing_nats/auth_intro/jwt https://docs.nats.io/running-a-nats-service/configuration/se... - Nats.ws client connection provides cookie during connection to perform client auth - If further authz/fine-grained control is needed the auth callout mechanism can be used. This would have access to the provided cookie/token so any claims needed for access control could be stapled on during step one and used at this point? For GPs original question -- I'm running a fairly old Keycloak version (v8) but it does appear to set a JWT in KEYCLOAK_IDENTITY and KEYCLOAK_IDENTITY_LEGACY. Am I right in understanding that IFF the token is signed with Ed25519 and both sub and iss are an NKEY value this is sufficient for NATS to accept that cookie as a credential?
  - bruth 3y ago
    Yes that reads correct. The `sub` would a NATS user public nkey, the `iss` would be the NATS account public nkey (either the issuer nkey in config-mode or existing nkey in decentralized auth). As long as it can verify the chain of trust for the user JWT that is returned, it should work. The three schema types are shown here: https://docs.nats.io/running-a-nats-service/configuration/securing_nats/auth_callout#schema https://docs.nats.io/running-a-nats-service/configuration/se... auth request comes in -> generate user jwt, sign + encode -> respond with auth response. As long as the necessary bits of the response and user JWT conform, it will work.

bruth 3y ago

> Seems like I need to write a service utilizing NATS which talks to the OIDC server. Auth callout was designed to be a generic extension point to delegate authentication and generate dynamic a user JWT that NATS understands (permissions, limits, etc). It enables an arbitrary backend to be integrated with, not tied specifically to OIDC. But indeed, this requires implementing a service that does this integration. > NATS auth story is a complicated one, and now with auth callout it's even more complicated. There is a spectrum of auth options, starting with simple config-based, token or user/pass leading up to decentralized auth for use cases that need it. Auth callout is an opt-in thing, so it should only be adopted if it is truly necessary. > you have to have dedicated functions which you would call in the request handler [...] instead of being able to define a chain of middleware functions I don't quite understand this statement. Wrapping a NATS handler is the same approach as wrapping an HTTP handler (within the same client app). The function would take a handler and return a handler. There can be inspection of the message within that function and the choice of calling the next handler, responding early, doing some external call, or doing nothing. If you want to distribute this middleware, then you would need to have separate, explicit subjects that each middleware would subscribe to and then publish to for the next element in the chain (for a choreography approach). There is also the "message slip pattern" where the ingest component sets the path as metadata (e.g. headers) that each middleware component uses that to know (at request time) which subject to publish the result to next.

lakomen 3y ago
Regarding middleware, last time I tried there was a lot of information to process. Now I tried again and func loggingMiddleware(next micro.Handler) micro.Handler { return micro.HandlerFunc(func(req micro.Request) { spew.Dump(req.Data()) next.Handle(req) }) } I'll remove the middleware part. EDIT: Seems I can't edit the post anymore. Regarding auth, yes, spectrum of options, but for "production" use it's very complicated and you don't want a shared key, you need the whole operator, account, user thing, and external auth aka auth callout. The documentation is lacking. nkey, xkey what? It's confusing is complicated. Managing auth is a pain, more than once I've been locked out of my test server because of how complicated auth is and had to wipe it. It's so complicated because you have have isolated "namespaces" or operator spaces? Naming an organization operator, or a solution or service group doesn't help. user is synonymous with client. All that you don't get from the documentation, you have to watch Youtube videos to understand it. The documentation is bad. How do I consume a stream from the deno/ws client without the whole client app blocking? No idea. The code snippets don't help. All there is, is a pay for company that offers to host NATS clusters, which you can pay to ask. Slack is not a good medium. I'm guessing that the documentation is bad so one has to pay for support. And that's a no-go for me. I get that you spent time and effort into it, I'm assuming you're one of the NATS people, but what good is it when I don't know how to effectively use the software? The natsbyexample page is also hard to digest. You're in your own little bubble and expect people to be mind readers. If it's too complicated people won't adopt it. I know I won't because it's too complicated. I would've loved to build upon it, because the core functionality is nice and works, easy to grasp. I've not seen a single Jetstream stream and consumer tutorial with the new API and nats.ws and Go. Old API sure, STAN sure. New API no. Some real world stuff, not just code demo snippets. Why not? Because it's complicated. Auth callout, that was the big thing everyone was waiting for? I would work on actually solving problems instead of building components so I can use this extra piece of software who's documentation is so bad. You know how many Kafka tutorials there are? A LOT. I want to like it, it seems very promising, but it suffers from the "coder no likey documentation" illness. Today I saw a Twitch stream of some Netflix dev, talking about DX, developer experience. NATS' DX has a lot of room for improvement, to put it friendly.