5 ms·

I would love to hear how you are managing protofiles across repositories. Do you have a central repository for protofiles? It's something we're trying to figu

I would love to hear how you are managing protofiles across repositories. Do you have a central repository for protofiles? It's something we're trying to figure out with our GRPC implementation.

jpdb 8y ago
This is something I've pondered myself, but haven't got a chance to play around with. My idea was simply to integrate it with make (or whatever build tool/script you prefer) where it pulls the .proto file from master in git unless you specify a branch. The downside is that you aren't versioning the protoc file with the client. It's probably best to separate the proto file for server and client and treat it as a dependency. If you're using a mono-repo the solution is possibly a little simpler.
- be_erik 8y ago
  I recently built something like this: https://github.com/reverbdotcom/protopkg https://github.com/reverbdotcom/protopkg
asdkhadsj 8y ago
We opted to store proto files in a singular location. In my view, this has three benefits: 1. Protobufs importing each other becomes effortless. You don't need to download multiple repositories, if you have the proto repo, you have all the dependencies. 2. Decoupling "ownership" allows for a bit more freedom when creating proto files that describe shared data structures. 3. You get a singular location to browse infra. It doesn't describe relationships of course, but it's handy to be able to see what services you have and what their API looks like. Disclaimer: I don't use gRPC, but I do use Protobufs & an RPC, Twirp.
nzoschke 8y ago
OP here. We keep them all in one repo. Right now the entire API is a monorepo so it’s easy. But I plan to split it up soon... All the protos in one repo. Ideally public so Segment users can use it as a reference too! Then all the generated Go clients, servers, mocks, typescript, swagger, etc in another repo. This is what programs can import. Then we can build gRPC services in their own repos if we want to. And it’s easy to produce and consume messages for all the other services.

I would love to hear how you are managing protofiles across repositories. Do you have a central repository for protofiles? It's something we're trying to figure out with our GRPC implementation.

jpdb 8y ago

This is something I've pondered myself, but haven't got a chance to play around with. My idea was simply to integrate it with make (or whatever build tool/script you prefer) where it pulls the .proto file from master in git unless you specify a branch. The downside is that you aren't versioning the protoc file with the client. It's probably best to separate the proto file for server and client and treat it as a dependency. If you're using a mono-repo the solution is possibly a little simpler.

be_erik 8y ago
I recently built something like this: https://github.com/reverbdotcom/protopkg https://github.com/reverbdotcom/protopkg

asdkhadsj 8y ago

We opted to store proto files in a singular location. In my view, this has three benefits: 1. Protobufs importing each other becomes effortless. You don't need to download multiple repositories, if you have the proto repo, you have all the dependencies. 2. Decoupling "ownership" allows for a bit more freedom when creating proto files that describe shared data structures. 3. You get a singular location to browse infra. It doesn't describe relationships of course, but it's handy to be able to see what services you have and what their API looks like. Disclaimer: I don't use gRPC, but I do use Protobufs & an RPC, Twirp.

nzoschke 8y ago

OP here. We keep them all in one repo. Right now the entire API is a monorepo so it’s easy. But I plan to split it up soon... All the protos in one repo. Ideally public so Segment users can use it as a reference too! Then all the generated Go clients, servers, mocks, typescript, swagger, etc in another repo. This is what programs can import. Then we can build gRPC services in their own repos if we want to. And it’s easy to produce and consume messages for all the other services.