5 ms·

This is exactly why a lot of people choose to use the protobuf-generated data types as their data model internally within their app in addition to for serializa

This is exactly why a lot of people choose to use the protobuf-generated data types as their data model internally within their app in addition to for serialization, and many people even derive their database schemas and validation from their protobuf schemas. (When I left Google in 2012, Spanner table schemas were literally defined using protobuf IDL. I haven't followed it since so I have no idea if that's still the case.) Admittedly, protobuf wasn't explicitly designed for these purposes, it just of grew into them because they were convenient, and as a result there are certainly warts. But it's close enough that it works pretty well.

finder83 6y ago
Fascinating, I didn't know it could be used for all of that. Last time I tried it it felt more like static typing than true validation (e.g. validating email addresses, date formats, dependent fields, etc) I'll take a look at it again, thanks.
p1necone 6y ago
I've had a lot of fun hacking together graph/tree types in protobuf.
fizx 6y ago
You can take it all the way into the db with OSS if you like https://github.com/FoundationDB/fdb-record-layer/blob/master/docs/GettingStarted.md https://github.com/FoundationDB/fdb-record-layer/blob/master...
hedora 6y ago
The last time I checked, protobuf didn’t help with any sort of validation. In fact, the code I’ve had to maintain (written by others) is more verbose than a hand rolled, validating serializer would be in C++, even though it didn’t actually validate. My only explanation for its success is that people often cargo-cult Google technology, and that it can be used for cross language communication by people that can’t hand-serialize a struct.
- gravypod 6y ago
  You can make use of features like annotations to magic away the validation of APIs [0]. > people that can’t hand-serialize a struct If you have 1 device type and 1 software version then this might work. If you have multiple CPU architectures and multiple versions of your software this will likely not work. [0] - https://github.com/envoyproxy/protoc-gen-validate https://github.com/envoyproxy/protoc-gen-validate
- deleted 6y ago
  [deleted]
gravypod 6y ago
Very much this. We do that at my current job. We use protos to: 1. Define configuration options for all services/devices 2. Define data models 3. Persist them into our DB: https://github.com/CaperAi/pronto https://github.com/CaperAi/pronto 4. Define our APIs It's extremely convenient to have a single source of truth.
bvrmn 6y ago
Sounds like a bad idea. If your storage follows HTTP API 1:1 then why you need backend at all? Take firebase all other db-as-service solutions. It's like developing without public interfaces. Implementation changes will break clients.

finder83 6y ago

Fascinating, I didn't know it could be used for all of that. Last time I tried it it felt more like static typing than true validation (e.g. validating email addresses, date formats, dependent fields, etc) I'll take a look at it again, thanks.

p1necone 6y ago

I've had a lot of fun hacking together graph/tree types in protobuf.

fizx 6y ago

You can take it all the way into the db with OSS if you like https://github.com/FoundationDB/fdb-record-layer/blob/master/docs/GettingStarted.md https://github.com/FoundationDB/fdb-record-layer/blob/master...

hedora 6y ago

The last time I checked, protobuf didn’t help with any sort of validation. In fact, the code I’ve had to maintain (written by others) is more verbose than a hand rolled, validating serializer would be in C++, even though it didn’t actually validate. My only explanation for its success is that people often cargo-cult Google technology, and that it can be used for cross language communication by people that can’t hand-serialize a struct.

gravypod 6y ago
You can make use of features like annotations to magic away the validation of APIs [0]. > people that can’t hand-serialize a struct If you have 1 device type and 1 software version then this might work. If you have multiple CPU architectures and multiple versions of your software this will likely not work. [0] - https://github.com/envoyproxy/protoc-gen-validate https://github.com/envoyproxy/protoc-gen-validate
deleted 6y ago
[deleted]

gravypod 6y ago

Very much this. We do that at my current job. We use protos to: 1. Define configuration options for all services/devices 2. Define data models 3. Persist them into our DB: https://github.com/CaperAi/pronto https://github.com/CaperAi/pronto 4. Define our APIs It's extremely convenient to have a single source of truth.

bvrmn 6y ago

Sounds like a bad idea. If your storage follows HTTP API 1:1 then why you need backend at all? Take firebase all other db-as-service solutions. It's like developing without public interfaces. Implementation changes will break clients.