3 ms·

Please correct me if I'm wrong, but I've always had the impression that protobufs introduce more coupling than JSON. Example: when you add a new element to the

Please correct me if I'm wrong, but I've always had the impression that protobufs introduce more coupling than JSON. Example: when you add a new element to the API response, the recipient needs to update their schema. Is that correct? If yes, how do you deal with that in a microservice architecture where you have little control over consumers of your API?

vkjv 9y ago
Yes, this is correct, but, it's also exactly what you want in microservices. You are free to make forward compatible API changes without a deployment dance. E.g., you can add a field in the service and the caller won't see it until it is prepared to handle it properly.
weego 9y ago
I don't understand the issue. The contract between producer and consumer should never be fragile around additional values existing.
- zbentley 9y ago
  > The contract between producer and consumer should never be fragile around additional values existing. ... it should never be fragile where fragility might cost more than it gains you. The open/closed principle is the right thing to follow in most cases, but not all. For example, choking on unrecognized additional fields by default in development is often a big time/bug-saver, since it gives you an early "hint" that things might not be speaking the same dialect to each other. Similarly, if you have a message convention that is heavy on "modifiers" which control irreversible things, sometimes it's better to be fragile rather than count on orchestration systems to work perfectly 100% of the time. Suppose you have a system for buying bicycles with a required field of "bike model", and modifiers of "color", "wheel size", etc. Now let's say your servers all got patched to expect a new modifier of "frame size" . . . except for one server which didn't restart due to a deployment bug, and is still serving traffic. If a client sends that unexpected additional modifier to the "old" server, and it gets ignored, this could result in sending a production order for an expensive unit with the wrong specifications. If not caught, that cost could be compounded by sending the wrong unit to the customer and pissing them off. Now, this is clearly not a problem with the message format, but rather with the orchestration/release system. But bugs in orchestration layers are incredibly common, and baking a bit of fragility into the messaging layer at the right places can help mitigate those bugs' impact.
schrodinger 9y ago
It’s true, but it’s also backwards compatible. That way you add the new field and consumers can update whenever they need to utilize the new field.
icebraining 9y ago
"You can add new fields to your message formats without breaking backwards-compatibility; old binaries simply ignore the new field when parsing. So if you have a communications protocol that uses protocol buffers as its data format, you can extend your protocol without having to worry about breaking existing code."
maltalex 9y ago
> Example: when you add a new element to the API response, the recipient needs to update their schema. That's not the case with protobufs. The sender can add new fields, and they will simply be invisible to receivers using the previous version of the message. You'd need to update the receiver only if you want it to access the new fields, which is what you'd need to regardless of serialization format.

vkjv 9y ago

Yes, this is correct, but, it's also exactly what you want in microservices. You are free to make forward compatible API changes without a deployment dance. E.g., you can add a field in the service and the caller won't see it until it is prepared to handle it properly.

weego 9y ago

I don't understand the issue. The contract between producer and consumer should never be fragile around additional values existing.

zbentley 9y ago
> The contract between producer and consumer should never be fragile around additional values existing. ... it should never be fragile where fragility might cost more than it gains you. The open/closed principle is the right thing to follow in most cases, but not all. For example, choking on unrecognized additional fields by default in development is often a big time/bug-saver, since it gives you an early "hint" that things might not be speaking the same dialect to each other. Similarly, if you have a message convention that is heavy on "modifiers" which control irreversible things, sometimes it's better to be fragile rather than count on orchestration systems to work perfectly 100% of the time. Suppose you have a system for buying bicycles with a required field of "bike model", and modifiers of "color", "wheel size", etc. Now let's say your servers all got patched to expect a new modifier of "frame size" . . . except for one server which didn't restart due to a deployment bug, and is still serving traffic. If a client sends that unexpected additional modifier to the "old" server, and it gets ignored, this could result in sending a production order for an expensive unit with the wrong specifications. If not caught, that cost could be compounded by sending the wrong unit to the customer and pissing them off. Now, this is clearly not a problem with the message format, but rather with the orchestration/release system. But bugs in orchestration layers are incredibly common, and baking a bit of fragility into the messaging layer at the right places can help mitigate those bugs' impact.

schrodinger 9y ago

It’s true, but it’s also backwards compatible. That way you add the new field and consumers can update whenever they need to utilize the new field.

icebraining 9y ago

"You can add new fields to your message formats without breaking backwards-compatibility; old binaries simply ignore the new field when parsing. So if you have a communications protocol that uses protocol buffers as its data format, you can extend your protocol without having to worry about breaking existing code."

maltalex 9y ago

> Example: when you add a new element to the API response, the recipient needs to update their schema. That's not the case with protobufs. The sender can add new fields, and they will simply be invisible to receivers using the previous version of the message. You'd need to update the receiver only if you want it to access the new fields, which is what you'd need to regardless of serialization format.