4 ms·
Hopefully this is good news. The old Protobuf package had ossified, with lots of ergonomic issues that the maintainers seemed uninterested in addressing. One p
by atombender 7y ago
Hopefully this is good news. The old Protobuf package had ossified, with lots of ergonomic issues that the maintainers seemed uninterested in addressing.
One perennial annoyance is how awkwardly some Protobuf stuff ends up being represented in Go. For example, "oneof" types:
message Event {
oneof payload {
Create create = 1;
Delete delete = 2;
}
}
message Create {
string id = 1;
}
message Delete {
string version = 1;
}
You get these messy types (I've elided most of the yucky marshaling-related stuff):
type Event struct {
Payload isEvent_Payload `protobuf_oneof:"item"`
}
type isEvent_Payload interface {
isEvent_Payload()
MarshalTo([]byte) (int, error)
Size() int
}
type Event_Create struct {
Create *Create `protobuf:"bytes,1,opt,name=create,proto3,oneof"`
}
type Event_Delete struct {
Delete *Delete `protobuf:"bytes,1,opt,name=delete,proto3,oneof"`
}
There are multiple problems here. One: Why the extra struct? For example, to create a single event, you have to do:
Event{
Payload: &Event_Create{
Create: &Create{ID: "123"},
},
},
...instead of just:
Event{
Payload: &Create{ID: "123"},
},
Secondly, isEvent_Payload is private! So given a Create or an Delete, you can't store it in a single variable:
func nextEvent() isEvent_Payload { // <-- not possible
if something {
return &Create{...}
} else {
return &Delete{...}
}
}
The reason for this appears to be so that the generated serialization can be very dumb and not rely on type switches. But I don't buy that this is how it has to be.
There are other issues. Overall, the whole package seems designed from the bottom up for machines (mumbles possibly also by machines), not humans.The upshot is that using Protobuf types as "first-class" types — meaning the types you actually use internally in the meat of your app, as opposed to in the controller glue that lives in your API and mediates between the API and the internals — feels super messy.
As an aside, anyone know what issues this paragraph refers to?
The google.golang.org/protobuf/encoding/protojson package
converts protocol buffer messages to and from JSON using the
canonical JSON mapping, and fixes a number of issues with the
old jsonpb package that were difficult to change without
causing problems for existing users.
Did they finally fix the zero value problem with jsonpb.go [1]?
[1] https://github.com/gogo/protobuf/issues/218 https://github.com/gogo/protobuf/issues/218
- dana321 7y agoIts horrible, i know. Try mixing that with grpc too and it gets even worse.
- lelandbatey 7y agoTo quote one of the maintainers of the golang/protobuf implementation: > The jsonpb package is intended to the be faithful implementation of the proto<->JSON specification, for which the C++ implementation is considered the canonical "reference" implementation. And that C++ implementation does some kinda dumb things. For example, it marshals the protobuf type of int64 into a string when marshaling from a protobuf struct into JSON. https://github.com/golang/protobuf/pull/916 https://github.com/golang/protobuf/pull/916 I believe that the package they mention there is meant to be a more "canonical" protobuf <-> json marshaller than the existing jsonpb package.
- al2o3cr 7y agoIMO marshaling int64 to string is required for interoperability; RFC7159 recommends not assuming that more than IEEE754's 53 bits of integer precision are available: https://tools.ietf.org/html/rfc7159#section-6 https://tools.ietf.org/html/rfc7159#section-6
- bhawks 7y agoThe original golang codegen omitted oneof support completely on the grounds that the concept didn't apply to golang. Eventually they realized that oneof isnt an optional feature of proto - but the solution always felt rushed and bolted on because it was. I hope the second try at proto api addresses these warts. I'm generally a golang fan - but not in this area. Citations: https://github.com/golang/protobuf/issues/29 https://github.com/golang/protobuf/issues/29
- DelightOne 7y agoDo you know whether they fixed oneof with APIv2?
- atombender 7y ago