4 ms·

One place protoc doesn't align well is the descriptor object. https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.descriptor https

by kyrra 4y ago

One place protoc doesn't align well is the descriptor object. https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.descriptor https://developers.google.com/protocol-buffers/docs/referenc... Comment placement is basically allowed anywhere by protoc, but how to get those comments within a Descriptor object for a proto is not well defined (there are places where you can put comments that are not available within Descriptor). It provides leading/trailing comments, but there are many other cases that are missed today (like comments embedded within a list of items in an array). Maybe this is a mismatch between what protoc allows and what Descriptor presents, but it's definitely annoying.

jvolkman 4y ago
I totally forgot about .proto comments. I spent a while re-implementing that [1] for the IntelliJ editor and even longer testing it [2]. Apparently I opened a bug while doing so; is b/33539835 still open? 1: https://github.com/jvolkman/intellij-protobuf-editor/blob/master/core/src/main/java/idea/plugin/protoeditor/lang/psi/util/PbCommentUtil.java https://github.com/jvolkman/intellij-protobuf-editor/blob/ma... 2: https://github.com/jvolkman/intellij-protobuf-editor/blob/master/core/src/itest/java/idea/plugin/protoeditor/lang/comments/PbCommentUtilTest.java https://github.com/jvolkman/intellij-protobuf-editor/blob/ma...
jhumphries131 4y ago
I agree! Protoc allows comments anywhere, but it doesn't bother preserving them all in the descriptor. At one point I thought this was a bug, since comments _could_ be preserved in far greater contexts (though definitely not all). But then I realized that the descriptor.proto comments do actually state the places where comments are retained: > If this SourceCodeInfo represents a complete declaration, these are any > comments appearing before and after the declaration which appear to be > attached to the declaration. https://github.com/protocolbuffers/protobuf/blob/v21.5/src/google/protobuf/descriptor.proto#L844 https://github.com/protocolbuffers/protobuf/blob/v21.5/src/g... After re-reading with this caveat in mind -- that comments are only preserved before and after complete declarations -- I realized that it does retain the comments that are expected. Long story short: the descriptor is bad as an AST if you care about recovering the original source. The descriptor is lossy. I'd recommend using a real AST for use cases that want something non-lossy: https://pkg.go.dev/github.com/jhump/protoreflect/desc/protoparse#Parser.ParseToAST https://pkg.go.dev/github.com/jhump/protoreflect/desc/protop...

by kyrra 4y ago

jvolkman 4y ago

I totally forgot about .proto comments. I spent a while re-implementing that [1] for the IntelliJ editor and even longer testing it [2]. Apparently I opened a bug while doing so; is b/33539835 still open? 1: https://github.com/jvolkman/intellij-protobuf-editor/blob/master/core/src/main/java/idea/plugin/protoeditor/lang/psi/util/PbCommentUtil.java https://github.com/jvolkman/intellij-protobuf-editor/blob/ma... 2: https://github.com/jvolkman/intellij-protobuf-editor/blob/master/core/src/itest/java/idea/plugin/protoeditor/lang/comments/PbCommentUtilTest.java https://github.com/jvolkman/intellij-protobuf-editor/blob/ma...

jhumphries131 4y ago

I agree! Protoc allows comments anywhere, but it doesn't bother preserving them all in the descriptor. At one point I thought this was a bug, since comments _could_ be preserved in far greater contexts (though definitely not all). But then I realized that the descriptor.proto comments do actually state the places where comments are retained: > If this SourceCodeInfo represents a complete declaration, these are any > comments appearing before and after the declaration which appear to be > attached to the declaration. https://github.com/protocolbuffers/protobuf/blob/v21.5/src/google/protobuf/descriptor.proto#L844 https://github.com/protocolbuffers/protobuf/blob/v21.5/src/g... After re-reading with this caveat in mind -- that comments are only preserved before and after complete declarations -- I realized that it does retain the comments that are expected. Long story short: the descriptor is bad as an AST if you care about recovering the original source. The descriptor is lossy. I'd recommend using a real AST for use cases that want something non-lossy: https://pkg.go.dev/github.com/jhump/protoreflect/desc/protoparse#Parser.ParseToAST https://pkg.go.dev/github.com/jhump/protoreflect/desc/protop...