3 ms·

Unless you're all-in on server streaming, the Connect protocol is indistinguishable from RESTful HTTP to this class of L7 proxies - it's just a POST to a Protob

Unless you're all-in on server streaming, the Connect protocol is indistinguishable from RESTful HTTP to this class of L7 proxies - it's just a POST to a Protobuf-looking path, with an `application/json` or `application/proto` Content-Type. Most of the proxies/service mesh things that provide rich gRPC instrumentation are also capable of providing rich instrumentation for this style of HTTP.

rektide 4y ago
I dunno. I'd like to see some comparisons. HTTP is so broad. Observability tools do good with it. But I think it's deceitful & disingenuous to blow this topic off like this. Streams use features like HTTP2+ Push, & there's all manner of semantic meaning implicit in headers & schemas that regular http wont see that these tools pick up on. I want to be sold that the complex multi-directional multi-stream works fine, are fully observable, but I suspect seriously we're overreducing, not being honest, by discarding the difficulties of tuning observability to specific protocols here & saying general http support will buy us everything.
- xyzzy_plugh 4y ago
  I'm honestly confused as to what your gripe is. I've done it all, OpenAPI, graphql, gRPC -- what exactly is difficult observability wise? Can you provide some specifics as to what is problematic? HTTP is broad, and some things are easier to observe than others. Something like streaming or websockets is by definition more complex than the most basic possible HTTP request/response. Your complaint boils down to "using features makes observability difficult", which is obviously true in some sense, but you're dismissive of what those features unlock. You must despise graphql.
  - rektide 4y ago
    > Your complaint boils down to "using features makes observability difficult", which is obviously true in some sense, but you're dismissive of what those features unlock. It's not a complaint, just a fact that gRPC has fantastic integration with very deep telemetry, and that anything off the beaten path (such as this alternate protocol) is probably going to require significant handholding/customization to make visible. Connect-web wont, out of the box, have any of this deep built-in customization, because it's right now a tiny player, and no one is going to do the work for you, unlike the couple big names. It's ironic that you mention GraphQL. Yeah, to be honest, I find graphql to be barberic. Submitting all your requests to /graphql is a dirtbag piece of shit regression that only an idiot too stupid to remember how terrible & shitty SOAP was would ever have suggested. It'd offensively bad a premise, & it makes looking at HTTP basics like the path useless. But I use it at work & dont complain. It's well supported by most observability tools. I have faith Connect-Web hasnt fucked up nearly as bad, but it's still not going to have anywhere near the sweet out of box L7 observability grpc nor graphql has and that's just obvious, uncontestable, clear, & it matters, a lot, & no protest or dislike of what Im saying alters that. Graphql, in spite of it's dis-http piss-around attitude, is also the exact proof of how valuable popularity is; something connect-web protocol doesnt have now. I can search for specific graphql queries or mutations, or even specific arguments passed in to these queries/mutations- in nearly any major observability platform, in spite of this being fairly obfuscated data deep inside the http posted body envelope, because it's popular, because the work has been done to probe in. You talk about respecting the features of new protocols. Im not trying to dis on Connect-Web- I have beef but it's elsewhere- but Connect-Web doesnt actually have new features. It's a new 3rd way to do what we already do. One that that will definitely clearly obviously uncontestably have less good tooling support than the we supported official protocol, & which exists only because the browser fucked shit up & failed to deliver, across 3/4 of a decade. It matters that the solutions we built around this shitshow are not as popular, not as supported, and while I like & respect connect-web protocol, I think it's necessary & truthful to view it as a relatively unsupported ideally-not-needed-at-all upstart; the very need for it is terrible versus just using the real, very well supported, highly observbable thing. I genuinely dont get the objection. Being not-the-mainstream, being an alternate upstart alternative will almost certainly have less awesome support. That's a fact that needs to be faced, not smokescreened.
    - bufbuild 4y ago
      We’d really recommend checking out the Connect protocol in this case - we totally understand that a new custom protocol would induce a few eye rolls, but it’s about as simple as it gets, especially with unary - it’s just HTTP POST at /service/method, switching on Content-Type between application/json and application/proto (for JSON vs binary payloads). All your existing tooling for HTTP observability should mostly “just work”. https://connect.build/docs/protocol https://connect.build/docs/protocol
      - rektide 4y ago
        I love the idea of a more http-native protocol, and I think ya'll are doing saintly work, and I hope observability tools close remaining gaps fast. I still think we're a little short on candor though. Streams are a bitch in grpc. I have a very very hard time believing plain-old http observability is going to be good enough to help me make sense of multiple cross-directional streams stemming from a single request. gRPC just has a lot to it. Being more plain http-centric is applause worthy. But grpc has too much going on for plain http support to be enough, when it comes to really pleasant observability & tooling. I dont want to discourage but I dont see how we could expect tl have near as good an experience as what great real grpc tooling supports. I love these workarounds to browser enable, but they have real tradeoffs & sacrifices. That alas the browser hss forced us all into by it's utter lack of giving a shit, caring, doing anything good with making http useful to us.

rektide 4y ago

I dunno. I'd like to see some comparisons. HTTP is so broad. Observability tools do good with it. But I think it's deceitful & disingenuous to blow this topic off like this. Streams use features like HTTP2+ Push, & there's all manner of semantic meaning implicit in headers & schemas that regular http wont see that these tools pick up on. I want to be sold that the complex multi-directional multi-stream works fine, are fully observable, but I suspect seriously we're overreducing, not being honest, by discarding the difficulties of tuning observability to specific protocols here & saying general http support will buy us everything.

xyzzy_plugh 4y ago
I'm honestly confused as to what your gripe is. I've done it all, OpenAPI, graphql, gRPC -- what exactly is difficult observability wise? Can you provide some specifics as to what is problematic? HTTP is broad, and some things are easier to observe than others. Something like streaming or websockets is by definition more complex than the most basic possible HTTP request/response. Your complaint boils down to "using features makes observability difficult", which is obviously true in some sense, but you're dismissive of what those features unlock. You must despise graphql.
- rektide 4y ago
  > Your complaint boils down to "using features makes observability difficult", which is obviously true in some sense, but you're dismissive of what those features unlock. It's not a complaint, just a fact that gRPC has fantastic integration with very deep telemetry, and that anything off the beaten path (such as this alternate protocol) is probably going to require significant handholding/customization to make visible. Connect-web wont, out of the box, have any of this deep built-in customization, because it's right now a tiny player, and no one is going to do the work for you, unlike the couple big names. It's ironic that you mention GraphQL. Yeah, to be honest, I find graphql to be barberic. Submitting all your requests to /graphql is a dirtbag piece of shit regression that only an idiot too stupid to remember how terrible & shitty SOAP was would ever have suggested. It'd offensively bad a premise, & it makes looking at HTTP basics like the path useless. But I use it at work & dont complain. It's well supported by most observability tools. I have faith Connect-Web hasnt fucked up nearly as bad, but it's still not going to have anywhere near the sweet out of box L7 observability grpc nor graphql has and that's just obvious, uncontestable, clear, & it matters, a lot, & no protest or dislike of what Im saying alters that. Graphql, in spite of it's dis-http piss-around attitude, is also the exact proof of how valuable popularity is; something connect-web protocol doesnt have now. I can search for specific graphql queries or mutations, or even specific arguments passed in to these queries/mutations- in nearly any major observability platform, in spite of this being fairly obfuscated data deep inside the http posted body envelope, because it's popular, because the work has been done to probe in. You talk about respecting the features of new protocols. Im not trying to dis on Connect-Web- I have beef but it's elsewhere- but Connect-Web doesnt actually have new features. It's a new 3rd way to do what we already do. One that that will definitely clearly obviously uncontestably have less good tooling support than the we supported official protocol, & which exists only because the browser fucked shit up & failed to deliver, across 3/4 of a decade. It matters that the solutions we built around this shitshow are not as popular, not as supported, and while I like & respect connect-web protocol, I think it's necessary & truthful to view it as a relatively unsupported ideally-not-needed-at-all upstart; the very need for it is terrible versus just using the real, very well supported, highly observbable thing. I genuinely dont get the objection. Being not-the-mainstream, being an alternate upstart alternative will almost certainly have less awesome support. That's a fact that needs to be faced, not smokescreened.
  - bufbuild 4y ago
    We’d really recommend checking out the Connect protocol in this case - we totally understand that a new custom protocol would induce a few eye rolls, but it’s about as simple as it gets, especially with unary - it’s just HTTP POST at /service/method, switching on Content-Type between application/json and application/proto (for JSON vs binary payloads). All your existing tooling for HTTP observability should mostly “just work”. https://connect.build/docs/protocol https://connect.build/docs/protocol
    - rektide 4y ago
      I love the idea of a more http-native protocol, and I think ya'll are doing saintly work, and I hope observability tools close remaining gaps fast. I still think we're a little short on candor though. Streams are a bitch in grpc. I have a very very hard time believing plain-old http observability is going to be good enough to help me make sense of multiple cross-directional streams stemming from a single request. gRPC just has a lot to it. Being more plain http-centric is applause worthy. But grpc has too much going on for plain http support to be enough, when it comes to really pleasant observability & tooling. I dont want to discourage but I dont see how we could expect tl have near as good an experience as what great real grpc tooling supports. I love these workarounds to browser enable, but they have real tradeoffs & sacrifices. That alas the browser hss forced us all into by it's utter lack of giving a shit, caring, doing anything good with making http useful to us.