4 ms·

I need to get around to playing with tree-sitter. The approach in this article is neat. Here's another approach. The AST of a .proto file is itself a protobuf.

I need to get around to playing with tree-sitter. The approach in this article is neat. Here's another approach. The AST of a .proto file is itself a protobuf. That's how the codegen plugins work. Protobuf also has a canonical mapping to JSON, so... What you can do is use protoc to parse the .proto file, spit it out as JSON, and then process that data using your favorite pattern matching language. I wrote a [tool][1] that helps with that. For example, here's some [js code][2] that translates protobuf message definitions into "types" for use in an ORM. [1]: https://github.com/dgoffredo/protojson https://github.com/dgoffredo/protojson [2]: https://github.com/dgoffredo/okra/blob/master/lib/proto2types.js https://github.com/dgoffredo/okra/blob/master/lib/proto2type...

superb_dev 2y ago
Oh my god… this might’ve made some tools I’m developing a lot easier
fizx 2y ago
Writing a protoc plugin would have been 5x easier, but its harder to get a blog article out of it. Also, this reads like they might not have seen the newer proto3 optional keyword, or know about the well-known wrapper types.
danielvaughn 2y ago
I highly recommend it. The docs describe it as a zen-like experience, and I fully agree. Once you get the hang of it, it makes it so easy to tweak the syntax of whatever language you’re building. I love it.
lelandbatey 2y ago
I found that some parts of a protobuf aren't captured well by protoc; specifically annotations were not well exposed to Go libraries for writing protoc plugins in 2016. I ended up having to write my own basic protobuf parser to reliably extract annotations and comments for code and documentation generation: https://github.com/metaverse/truss/blame/master/svcdef/svcdef.go https://github.com/metaverse/truss/blame/master/svcdef/svcde...
- chipdart 2y ago
  > I ended up having to write my own basic protobuf parser (...) Wouldn't it have been far more efficient to contribute back to protoc? Certainly posting a patch to a parser takes far fewer work than writing an alternative implementation of the same parser.
  - duggan 2y ago
    Not speaking for Leland, and this is not a universal principle, but if you know what you're doing, you can write a solution and have it out of your way faster than going through the bureaucracy of trying to get some code merged upstream.
    - chipdart 2y ago
      Your comment makes no sense at all. You do not need to get a PR merged before you can use your changes. You work out a patch, use it locally to meet your needs, post a PR, and you move on with your life regardless of whether your PR is merged or not. In fact, that's the whole premise of GitHub: you fork a repo, you work on a feature branch, you post a PR, anf your fork stays there. To repeat my point: it's easier to add small features to a parser than it is to reimplement everything, and posting a PR with your small change takes no work at all.
      - duggan 2y ago
        > Your comment makes no sense at all. You do not need to get a PR merged before you can use your changes. What you said was: > Wouldn't it have been far more efficient to contribute back to protoc? My point was about the bureaucracy of contributing back upstream. Nonetheless, your new point, that patching and using your own fork is preferable makes sense on the surface given the ease of forking in go projects. Be interested to hear the author's reasoning.
        chipdart 2y ago
        > My point was about the bureaucracy of contributing back upstream. There is no bureaucracy. Once you have your patch, you can post a PR. Click on a button, write a description, and you're done. That's it. Posting a PR is hardly the hard part of the process.

superb_dev 2y ago

Oh my god… this might’ve made some tools I’m developing a lot easier

fizx 2y ago

Writing a protoc plugin would have been 5x easier, but its harder to get a blog article out of it. Also, this reads like they might not have seen the newer proto3 optional keyword, or know about the well-known wrapper types.

danielvaughn 2y ago

I highly recommend it. The docs describe it as a zen-like experience, and I fully agree. Once you get the hang of it, it makes it so easy to tweak the syntax of whatever language you’re building. I love it.

lelandbatey 2y ago

I found that some parts of a protobuf aren't captured well by protoc; specifically annotations were not well exposed to Go libraries for writing protoc plugins in 2016. I ended up having to write my own basic protobuf parser to reliably extract annotations and comments for code and documentation generation: https://github.com/metaverse/truss/blame/master/svcdef/svcdef.go https://github.com/metaverse/truss/blame/master/svcdef/svcde...

chipdart 2y ago
> I ended up having to write my own basic protobuf parser (...) Wouldn't it have been far more efficient to contribute back to protoc? Certainly posting a patch to a parser takes far fewer work than writing an alternative implementation of the same parser.
- duggan 2y ago
  Not speaking for Leland, and this is not a universal principle, but if you know what you're doing, you can write a solution and have it out of your way faster than going through the bureaucracy of trying to get some code merged upstream.
  - chipdart 2y ago
    Your comment makes no sense at all. You do not need to get a PR merged before you can use your changes. You work out a patch, use it locally to meet your needs, post a PR, and you move on with your life regardless of whether your PR is merged or not. In fact, that's the whole premise of GitHub: you fork a repo, you work on a feature branch, you post a PR, anf your fork stays there. To repeat my point: it's easier to add small features to a parser than it is to reimplement everything, and posting a PR with your small change takes no work at all.
    - duggan 2y ago
      > Your comment makes no sense at all. You do not need to get a PR merged before you can use your changes. What you said was: > Wouldn't it have been far more efficient to contribute back to protoc? My point was about the bureaucracy of contributing back upstream. Nonetheless, your new point, that patching and using your own fork is preferable makes sense on the surface given the ease of forking in go projects. Be interested to hear the author's reasoning.
      - chipdart 2y ago
        > My point was about the bureaucracy of contributing back upstream. There is no bureaucracy. Once you have your patch, you can post a PR. Click on a button, write a description, and you're done. That's it. Posting a PR is hardly the hard part of the process.