6 ms·
Istio Observability with Go, gRPC, and Protocol Buffers-Based Microservices
- xissy 8y agoCan’t read due to medium free access limit policy.
- gstafford 8y agoAlso published on my own blog: https://wp.me/p1RD28-6k1 https://wp.me/p1RD28-6k1
- gstafford 8y agoAlso published (free access) here: https://wp.me/p1RD28-6k1 https://wp.me/p1RD28-6k1
- williamallthing 8y agoHave you considered doing a similar writeup for Linkerd? (https://linkerd.io https://linkerd.io)
- SlowRobotAhead 8y agoThings I don’t get: 1. why are they choosing gRPC over REST if the application is entirely web based. With REST you have a standardized system of verbs and not specific application functions (GET /user/id , 200 OK <data> vs.... MyGetUserFoo(ID) , some return data unique to application). With RPC both sides need very specific knowledge of the functions each other have and their arguments all the time, service discovery seems harder and an update to application seems to nearly always imply and update to web. 2. What is the model supposed to be for typical protobuf schema sharing? I like PBs, but it seems a little harder than JSON or CBOR or MessagePak or other schemaless serializers in that the proto file has the same sync issue among all of your endpoints. 3. Wouldn’t be really swell if serializers were supported by cloud providers and Microservices a little more? I have a project right now that every time a message goes from our services to an endpoint it needs encoding and decoding. This is kind of a header now for every Microservices we have, it would be nice to not have the option of making so many mistakes.
- ec109685 8y agogRPC’s responses are backwards compatible, so you aren’t mandated to update all clients at the same time.
- lugg 8y agoYou're still blocked from using later features potentially even reaching a dependency chain deadlock until you update. Systems like this work far better when you just work to not break backwards compatibility full stop. Similarly, they work better when you don't tangle up dependencies in a bunch of shared code, generated or otherwise.
- joshuamorton 8y agoI'm unclear what problem your describing. Your always blocked from using later features until an update because we'll, you won't have application code that can take advantage of those features until you update.
- ehsankia 8y agoThat's missing the point. Yes, the goal is to get to a state where all the services are running the new code, but the hard part is that transition. You can manually handle it using versioning and such, but gRPC handles that for you.
- nhooyr 8y agoJSON is as well.
- isoskeles 8y agoJSON is agnostic towards anything-compatibility. If you want your service to be backwards compatible, and it is serving JSON, I think you're only going to achieve your goal through rigorous 'integration' tests.
- nine_k 8y agoREST is seriously more expensive traffic-wise. Traffic may be cheap, but latency of receiving a large message can't be helped. Protobufs can be and in normal practice are made backwards-compatible. For serious breaking changes, you want to version your endpoints anyway.
- eeZah7Ux 8y agoThe complexity and amount of code involved is staggering. I'm not sure how we can possibly justify the Kubernetes, Istio, Docker, Envoy, Prometheus, Grafana, Jaeger, Kiali, Helm stack. What problem are we solving that we were not solving 15 years ago, again? How much time and effort are being saved (by organizations smaller than google)?
- closeparen 8y agoHow about 5,000 engineers each committing and deploying daily?
- lugg 8y agoIf that's the real problem this is the wrong solution. I don't care how good your hiring practices are, there is zero chance you get 5000 devs using this stack correctly. Less is more at this scale.
- closeparen 8y agoDevs by and large aren't doing this stuff, it's abstracted by infrastructure teams behind application frameworks and a "Deploy" button UI. Much of this stuff was developed by such infrastructure teams to solve their own problems; people realized they were common across the industry and started collaborating on open source. The alternative is typically half-baked, homegrown deployment automation, service discovery, etc. not a fundamentally simpler architecture.
- deleted 8y ago[deleted]
- spyspy 8y agothis is not true. the promise of k8s led many companies to believe that individual dev teams can now manage their own infra and it’s been a giant mess. almost no one had the expertise or patience to properly manage their clusters. my company had pivoted back to a central cluster teams can run on that’s managed by real infra people, and dev teams leaning more heavily on PaaS/serverless systems like that they can actually wrap their heads around.
- nevi-me 8y agoIs Istio/Envoy supported on a non-k8s deployment? We're deploying an application that's in containers but not running on k8s, and has: * an Angular front-end, using grpc-web * a gRPC-web proxy (improbable-eng; golang) * an nginx proxy (for Angular, and routing grpc traffic) * an authentication manager (grpc+rust) * an audit logging service (grpc+rust) * a database service (grpc+kotlin) * a 'logic' engine thing (grpc+kotlin) * jaeger for distributed tracing It's been a joy to develop, but I wouldn't mind reducing the 2 proxies to 1 with maybe Envoy. I tried rewriting grpc-web proxy in Rust, but gave up after struggling and not having enough time to complete it.
- styles 8y agoIt should be. I was at a talk where they said Envoy was and I believe that Istio is coming soon. From what I remember, Envoy is what you need to plugin to bridge your infrastructure.
- owaislone 8y agoEnvoy assumes nothing about k8s AFAIK. It should be able to run just fine outside it but if you were using k8s, then using ambassador instead of pure Envoy would be a better choice. http://getambassador.io/ http://getambassador.io/
- twblalock 8y agoIt is a goal of Istio to be portable and not tied to Kubernetes, but I don't think it's there yet. I have looked for people who have successfully run Istio in production outside Kubernetes and I cannot find any. Most of the documentation and examples you will find online are for Kubernetes.
- Thaxll 8y agoHow is Rust support for gRPC last time I checked it was very slow / broken compare to supported languages? Also I quiet don't understand using 3 different languages for a tech stack, why don't you stick to one language?
- steveklabnik 8y ago
- sklivvz1971 8y agoI found this interesting and an excellent general overview of today's monitoring practices while using microservices. I have to say, though, that I am always perplexed by these toy microservice architectures because they are not solving a problem but creating it. For example, we would not need sophisticated distributed tracing if we used less moving parts; we would not need to use highly optimized (but hard to debug) protocols like grpc and protobuf if we did not have to rely on a massive graph of calls for each service, and so on. Of course, there are a bunch of lovely use cases for microservices, mostly around "how do we make 1k developers collaborate over our codebase?" (answer: enlarge the codebase!), which always sounded like a self-fulfilling prophecy to me, but then again, I loathe working for huge companies. If that is your case, then these tools will make your developer life a little more bearable. If you are playing with microservices because they are fresh and new, please consider a more conservative architecture first.
- chessturk 8y agoAs a developer who uses go microservice architecture and is in the process of containerizing our services at work, this is very valuable. Thank you! As an elixir/erlang enthusiast, haven't we just reinvented BEAM and observer?
- pdimitar 8y agoYep. No small amount of Erlang/Elixir programmers are kind of amused how K8s is trying to invent OTP for nodes.
- mercer 8y agoI wish we could somehow convince the undoubtedly very smart people working on K8s to just get on the BEAM and go from there? You know, work on Dialyzer, fix my code highlighting issues in VSCode, make Elixir faster, somehow make it easy to create cross-platform binaries, and give me a platform-native Observer ;).
- pdimitar 8y agoI suppose it's not easy to part with the illusion that containers should not be ephemeral. I mean, that's probably easy to do for microservices; you fire off a function, it does something useful and returns back quickly. Having K8s do a home-grown lambda hosting for you is useful there. But the moment you introduce any state, the whole idea of ephemeral containers is much more of a hindrance than something actually useful. A singular BEAM VM might run (realistically) tens of thousands of processes that do a lot of useful work. It's just not practical to kill off such a container and hand-wave away the huge costs of spinning it back up. IMO K8s and Erlang/Elixir are mutually exclusive at this point, sadly. There are projects in progress that attempt to solve the distributed supervisors ideas -- Swarm, Firenest, Horde -- but even without that, any Erlang/Elixir app can go a long way before needing distributed coordination.