5 ms·
Protobuf has LSP support
- echelon 2mo agoBuf's offering of protobuf registries and codegen SDKs for microservices seems less necessary in the LLM era. I'm starting to question many of protobuf's advantages (perhaps not the wire format). Add to that monorepos and other fads of the 2010s given the rise of LLMs. I used to be a big believer in this stuff, but I'm quickly having my core assumptions change out from under me.
- sudorandom 2mo agoI feel exactly that way about REST. The assumption that LLMs make schemas obsolete misses how structured outputs actually work in production. When you have probabilistic models generating code, strict contracts become more critical, not less. It is no coincidence that several major LLM platforms rely on ConnectRPC and Protobuf for their own APIs.
- echelon 2mo agoYou're right, but the adeptness of models to spin up clients and behaviors on the fly is remarkable. They're capturing the semantics of behavior at a deeper level. If we do strict schemas, I'd like to see less ceremony around them. Tool calls instead of brittle build steps and protocol registries. Perhaps we need new tools for this going forward.
- sudorandom 2mo agoHm... Maybe. In my view, an IDL is part of the input that you absolutely want humans to author or carefully review at least. In my experience, the ceremony around generating code is also performed very well by LLMs. But I do agree, there's definitely some changes that are needed to integrate Protobufs better. Some languages have built-in tooling to make it seamless, but it's definitely not universal.
- someothherguyy 2mo agowhy? are humans going to stop using services?
- giancarlostoro 2mo agoThe real argument shouldn't be about protocols becoming obsolete, but programming languages that are "less efficient" could eventually become obsolete in favor of highly scalable and performant languages due to LLMs when the main gap becomes knowing how the tech works at a high level, and not the syntax, why code in one language over another if you don't need to worry about messing up on syntax, only about reviewing logic for sanity and correctness against business rules as well as validating that it is stable code.
- cyberax 2mo agoAnybody who thinks that you can just chuck unstructured data into LLM and YOLO the app is an idiot. This works up to a point, and then it doesn't. And you're left with tons of inconsistently formatted data. My company is built on protobufs from ground up :) We use it in the database, for remote calls, on the frontend, etc. The protobuf language is not great, but it's about the right balance between too expressive and too restricting. And the best thing is that it's compact, compared to OpenAPI.
- kristjansson 2mo agoA big differentiator is whether one imagines an LLM in-band with most/all future software. If there is, and we’re deferring until very late parts of a program that world have been load bearing, and we’re able to programmatically ands reliably squint and say “eh i know what you meant” … then yeah formalizations seem superfluous-to-counterproductive. OTOH if LLMs are to write, but not supplant, much of software, then boundaries, delegation to deterministic layers, good compilers to bonk miscreant models on the head with error message seem essential. At one point it would have been shocking to assert that the compiler would live in-band with the program too. and yet JS eats the world. It seems shocking today that we could have a universal prior over the world operating in the ms/us nJ/pJ range required. And yet … ?
- ragall 2mo agoYou're wrong on every count: given the small context window of LLMs, the need for interfaces and scehmas is even greater.
- deleted 2mo ago[deleted]
- deleted 2mo ago[deleted]
- ltbarcly3 2mo agoWatch as I don't use protobuf because it is horrible. .... Tada! If you patch clients to google services in Python to use json instead of grpc they get faster and more reliable. A lot faster. Benchmark it! def get_json_client() -> CloudLoggingQueryClient: """Client for log queries (JSON transport, avoids gRPC overhead).""" client = google.cloud.logging.Client.from_service_account_info(...) client._use_grpc = False return client For me that is how I know something like protobuf is good. It is a nuisance to manage and distribute the definitions, adds a build step even to languages with no build step normally, is slower than almost every alternative, and artificially restricts you from doing lots of common things. It's so good! And look at the code quality of the implementation! It's like a team of interns wrote it while drunk. It is a complete spaghetti mess, but has tons of super convoluted micro optimizations that are slower than just doing the most obvious thing, but make the implementation confusing and indirect. It's trash code.
- itsthecourier 2mo agoso how do you save data over the cable when it's needed?
- ltbarcly3 2mo agozstd?? Obviously?
- anematode 2mo agoJSON. /s
- throw1234567891 2mo agoEventually it’s all bytes. Where do you want to save them?
- pastel8739 2mo agoIs this because load is lighter on their JSON endpoints that their gRPC ones?
- 2mo ago
- gafferongames 2mo agoWhile not a direct competitor to protobufs, if you are working in the video game space where struct versioning is not needed, there is an alternative language called "schema" that supports C, C++, C#, Golang, Rust and JavaScript. https://github.com/mas-bandwidth/schema https://github.com/mas-bandwidth/schema
- satvikpendem 2mo agoOf all names, they pick schema? That's like calling a programming language "language."
- max-privatevoid 2mo agoMy favorite programming language is called "A Programming Language".
- andai 2mo agoReminds me of xkcd tattoo that says in Chinese, "It's what my tattoo says."
- jayd16 2mo ago> video game space where struct versioning is not needed Save files? Looser than exact version multiplayer?
- gafferongames 2mo agoMultiplayer games typically deploy both client and server at the same time, and refuse to connect a client if it doesn't speak the exact same protocol as the server. Thus all the versioning overhead of protobufs is not needed for this wire protocol. (Yes, games still use versioning everywhere else where it makes sense: save games, asset data, config etc...)
- npstr 2mo ago
- eterm 2mo agoLots of naysayers here, but an advantage of protobuf is that proto files are hand-writeable, and therefore having an LSP for that could be useful. That said, proto itself dissuades or forbids the kind of common things you might do with a LSP, such as renaming. Renaming fields is a big no-no [edit: this isn't true, see corrections below], as is doing things like re-ordering fields. A core idea of proto is that versions are strictly compatible with with previous versions. This itself has limitations and challenges for migrations, but encourages good practice about compatibility that usually gets ignored or hand-waved away in most ecosystems. I accept however that it's often easy to offload both the re-structuring and the checking of version compatibility to an LLM and let them go at it.
- wazzaps 2mo agoActually renaming and reordering fields is completely fine, as long as the field ID and type stay the same
- eterm 2mo agoShit, you're right, I'm trying to remember the one that used to trip us up all the time, maybe it was removing fields? There was definitely one that kept tripping up the checks and it was something people like to do.
- dieortin 2mo agoDepends, if you use textproto then renaming is problematic too
- ventana 2mo agoConsidered breaking change because JSON serialization will change, and JSON is used quite often. Reordering fields without changing their ID is indeed a non-breaking change from what I understand, albeit quite a useless one :)
- imoverclocked 2mo ago> a useless one If you have a (long) list of fields and you want to keep them lexicographically sorted, being able to reorder is quite useful if you rename a field.
- williamcotton 2mo agoI looked at the dependencies and noticed that it wasn’t using an existing Protobuf parser which means they reimplemented the parser from scratch. Perhaps due to a lack of error recovery in the existing implementations? I don’t have the energy right now for further inspection. It is definitely best to reuse the parser for the runtime when implementing an LSP but to do so properly means implementing the parser itself as a standalone library. Even better is shipping the semantic analysis as well! Implementation drift is definitely an issue. But great project anyways, just wanted to put my thoughts on the matter into the conversation!
- arccy 2mo agobuf is a parser / compiler as an alternative to protoc https://buf.build/docs/migration-guides/migrate-from-protoc/ https://buf.build/docs/migration-guides/migrate-from-protoc/
- williamcotton 2mo agoAh, so does the LSP just call out to the Buf binary?
- arccy 2mo agoThe lsp is a subcommand of buf, from the page: `buf lsp serve`
- hxtk 2mo agoThe LSP is one of the subcommands provided by the buf binary. My LSP configuration for protobuf is `buf lsp serve`.
- williamcotton 2mo agoMakes sense, thanks for the clarification.
- adastra22 2mo agoIt is my understanding that basically all LSP implementations use tree-sitter because of its incremental rebuild capability, and this requires re-implementing the parser.
- PufPufPuf 2mo agoBuf does great work at fixing Protobuf to the point of being just barely usable. A godsend if you're stuck with Protobuf/gRPC on a legacy project.
- qweqwe14 2mo agoWhat world are you living in where Protobuf/gRPC are considered "legacy"? ..what?
- antonvs 2mo agoIn the web dev world, I’ve noticed there are a lot of people who think that JSON is the ultimate modern data format. Typically those same people have barely heard of JSON Schema.
- cdelsolar 2mo agoI use Go + Buf + protobuf-es + Connect on my web projects. There is no better stack, I'll die on this hill.
- dewey 2mo agoThis is the way! It's great to just point frontend developers to the Buf documentation for new features and they easily have all the types and generated clients available in their projects too.
- sudorandom 2mo agoI pushed for us to adopt this setup at my previous company and it was wildly successful. I thought frontend devs would resist but they absolutely loved the generated types and clients. It quickly seemed silly to do anything else.
- weitendorf 2mo agoThis has been driving me crazy ever since I started using protobuf/grpc and realized more major tech companies (in the cloud/infra/data world at least, and a lot of other SAAS) were using it or something similar (eg capn proto) internally than not. It feels like we’re in some sort of deadlock where each of them think “proto/grpc are too niche to support for external users, better just use JSON”, keeping it unfamiliar for an Average Web Developer. But if every company using it just exposed it to third parties/added it to their public APIs, it would immediately be common (and trendy) enough for every web developer to learn it and start using it. If Google added the missing HTTP/2 streaming support to browsers (blocking native bidi grpc streaming) it would have an instant killer app in making it easier to implement websocket-like client/server applications. It makes absolutely no sense that full duplex bidi was added to the HTTP/2 but remains unimplemented in browsers. The role MCP, OpenAPI, and JSON schema fill all would be a million times simpler if they were based on protobuf instead of JSON. I can forgive OpenAPI/JSON but it honestly pisses me off that we ended up with MCP and JSON-RPC + JSON schema, and people think these are cool/good tools, and actively adopting them. Just piling on the slop
- lacoolj 2mo ago"You're welcome" is absolutely hilarious to read from a company post.
- trial3 2mo agoYour HN comment has a reply. You're welcome.
- bmitc 2mo agoIt's gotten out of hand. Companies making money think they do everyone a favor.
- tredre3 2mo agoThey are doing us a favor, there's no denying that. Just like dozen of other providers did them a favor by providing them with the free software pieces they relied on to build their product. Of course it's poor taste to add the "You're welcome". But poor taste is par for the course for such startup, unfortunately.
- bmitc 2mo agoI meant it in a broader sense. Even for paid products, corporations view themselves as doing customers a favor. That's the sort of antagonistic relationship they define.
- imoverclocked 2mo agoDespite interesting legal definitions, companies are just groups of people. I also read the title as having an air of self-importance or maybe passive-aggression. If you read it a little more dispassionately then it's just an awkward way of stating that this is a gift. Why is this a gift? The company does not need to give it to you, though it arguably benefits from doing so. It did so with some definition of "free."
- hinkley 2mo agoWhat can I say, except you're welcome.
- unprovable 2mo agoFrom protobuf to quantum computing, Google just keeps solving problems nobody has...
- r_lee 2mo agoso you're saying protobuf has no purpose?
- booi 2mo agoi mean...
- ventana 2mo agoThat's not Google. Buf.build is, hilariously enough, a completely separate company that tries to make sense of Google's protobuf, and it's doing a decent job from what I can see.
- teaearlgraycold 2mo agoProtobuf should at least get credit for enforcing fields be optional.
- debugnik 2mo agoThey reluctantly acknowledged that as a mistake by adding an actual "optional" modifier for field presence. proto3's previously so-called optional were really all present with a zero default value; proto2 could have supported that for required fields at the code level if they had wanted. In the end, all they've changed is how they frame the feature.
- alecthomas 2mo agoWhat an oddly arrogant post, there's been a Protobuf LSP available for years: https://github.com/lasorda/protobuf-language-server https://github.com/lasorda/protobuf-language-server
- llm_nerd 2mo agoThe bizarre "you're welcome" in the title is so obnoxious that I was sure it would be related to some sort of funny twist in the blog entry. Nope, they really thought that was appropriate.
- colechristensen 2mo agosurely we're not a community so unfamiliar with lacking social skills and this aberration isn't at all important to the topic at hand
- Dylan16807 2mo agoIt sounds like you're having a bad day or something. Putting "you're welcome" in the title of your own blog, on a free software release, is fine. It's not pushed into anyone's face. It loses some points for being a corporate ad but that would happen regardless of wording.
- otterley 2mo agoI think it makes the author sound like a dick. Test for yourself: next time you take out the garbage, loudly tell your partner or roommate, “I took out the garbage. You’re welcome.”
- Dylan16807 2mo agoThat's a shared chore. Do you not see the obvious differences in the situation? If they said "you're welcome" about an obligation like paying taxes and also made it very personal, then it would be close in feeling to your scenario. But it's not. If you offer a genuine gift, in a situation where there is no expectation of payback or debt, you can say you're welcome.
- aussieguy1234 2mo agoI've always thought of protobuf as a Java thing. It worked great in Java, but not so well in JS, especially with Kafka.
- antonvs 2mo agoProtobuf was never particularly a Java thing. You could more accurately say it was a C++ thing. It was initially developed in C++, and used mainly in C++ services at Google. The implementation and early ecosystem were C++-centric. Later, Go became one of the major Protobuf ecosystems, and today it would be understandable to think it was a Go thing.
- gjvc 2mo ago"You're welcome." is such a smug verbal tic.
- jvolkman 2mo ago> Protobuf now has modern IDE support for the first time Weird post. I built IntelliJ protobuf support [1] while at Google like 10 years ago, and it started shipping by default with IntelliJ in ~2021. Maybe that's not considered "modern". [1] https://github.com/jvolkman/intellij-protobuf-editor https://github.com/jvolkman/intellij-protobuf-editor
- brunoborges 2mo ago"IntelliJ is for Java, and Java is legacy!" Probably the author
- throwaway27448 2mo agoI wouldn't consider an IDE-specific integration very modern (in IDE terms) in the age of LSP.
- mzl 2mo agoConversely, LSP is quite limited in comparison to what can be done with a real IDE integration in JetBrains product suite, so it is not so clear cut. See this post for some details: https://matklad.github.io/2023/10/12/lsp-could-have-been-better.html https://matklad.github.io/2023/10/12/lsp-could-have-been-bet...
- dewey 2mo agoSure, but having a generic LSP that works with all editors vs. an editor specific integration is still a very different thing. It's a bit odd for the parent commenter to say "Weird post" and implying that having an editor specific feature would somehow make having an LSP that works for everyone be any less interesting.
- saghm 2mo agoYeah, it would be like someone complaining for a new language getting invented because clang implemented a frontend for it years ago
- blackbriar75 2mo ago[dead]
- nullbio 2mo agoProtobuf getting LSP support before Codex implements LSPs is wild.
- theowaway213456 2mo agoWhat would it mean for codex to "implement LSPs"
- nullbio 2mo agoLSP server support, like Claude Code does.
- deepsun 2mo agoUnpopular opinion -- LSPs are bad because they introduce latency into coding. For every keystroke, my IDE needs to make request/response with LSP, instead of using its own internal parser/colorer/autocomplete.
- atiedebee 2mo agoYou can just not use an LSP? My understanding is that LSPs aren't made to be the fastest (code completion|syntax highlighter|code analysis), but easily integrated into all sorts of IDEs/text editors.
- deepsun 2mo agoAs far as I understand, IDEs ara way less eager to spend resources onto their own integration if there's an LSP. E.g. last time I tried to code in Zig there was either LSP or nothing.
- chlorion 2mo agoMaybe if you have an extremely slow computer? Even my little chromebook can provide completions and errors effectively instantly to me, even in medium sized rust projects. Or are you saying (probably maximum) ~20ms delay is blocking your ability to code?
- deepsun 2mo agoIt's not 20ms, 1-2 seconds for sure, for Zig LSP. 96GB RAM, 12 vCPU, Linux, desktop.
- ecnahc515 2mo agoLSP isn't used for syntax highlights usually, that's still the job of the editor. This is where something like treesitter usually comes in. LSP is only used in this case for errors/annotations/etc.
- Storewide 2mo ago[flagged]
- yodsanklai 2mo agoAren't LSP dying (and eventually IDE, at least in their current form) as everybody use LLM to code. I know some big tech companies redirected teams supporting them to new efforts (ie. tool integration with AI).
- loicalleyne 2mo agobuf also has an interesting dynamic protobuf parser that's a drop-in for dynamicpb https://github.com/bufbuild/hyperpb-go https://github.com/bufbuild/hyperpb-go