3 ms·

Protobufs are designed to support schema evolution without explicit versioning. (All fields are optional so they can be added or dropped, provided field numbers

Protobufs are designed to support schema evolution without explicit versioning. (All fields are optional so they can be added or dropped, provided field numbers aren’t reused.) It looks like Preserves just uses version numbers in its schemas. On the other hand, you can read the data without a schema, similar to JSON.

tonyg 2y ago
The version number is the schema language version, not the version of the collection of types described in the file. The schema language is extensible/evolvable in that pattern matching ignores extra entries in a sequence and extra key/value pairs in a dictionary. So you could have a "version 1" of a schema with Person = <person @name String> . and a "version 2" with Person = @v2 <person @name String @address Address> / @v1 <person @name String> . Then, Person.v2 from "version 2" would be parseable by Person from "version 1", and Person from "version 1" would parse using "version 2" as a Person.v1. The schema language is in production but the design is still a work in progress and I expect more changes before a 1.0 release of the schema language. (The schema language is completely separate from the preserves data model, by the way -- one could imagine other schema languages being used instead/as well)
- skybrian 2y ago
  Thanks for the clarification! That sounds about as evolvable as JSON or any system that uses string keys (like HTTP headers). Protobufs have an extra level of indirection built in: code refers to fields using names, but numbers are sent on the wire. Without convenient access to field numbers, they can’t as easily be hard-coded. This also strongly encourages using the schema file for most tasks. With protobufs (or similar), any user-friendly editor will need a schema to make sense of the data. JSON-like systems and protobufs have opposite design goals: encouraging versus discouraging schemaless data access.
  - tonyg 2y ago
    There are no string keys in the Person example above. You could add some, though, or use numbers instead with the same host-language API: Person = <person @name String @address Address> as above, or Person = <person { @name "name": String @address "address": Address }> or Person = { @name 1: String @address 2: Address } etc. all produce the same host-language record, e.g. in TypeScript export type Person = { name: String, address: Address, };

tonyg 2y ago

The version number is the schema language version, not the version of the collection of types described in the file. The schema language is extensible/evolvable in that pattern matching ignores extra entries in a sequence and extra key/value pairs in a dictionary. So you could have a "version 1" of a schema with Person = <person @name String> . and a "version 2" with Person = @v2 <person @name String @address Address> / @v1 <person @name String> . Then, Person.v2 from "version 2" would be parseable by Person from "version 1", and Person from "version 1" would parse using "version 2" as a Person.v1. The schema language is in production but the design is still a work in progress and I expect more changes before a 1.0 release of the schema language. (The schema language is completely separate from the preserves data model, by the way -- one could imagine other schema languages being used instead/as well)

skybrian 2y ago
Thanks for the clarification! That sounds about as evolvable as JSON or any system that uses string keys (like HTTP headers). Protobufs have an extra level of indirection built in: code refers to fields using names, but numbers are sent on the wire. Without convenient access to field numbers, they can’t as easily be hard-coded. This also strongly encourages using the schema file for most tasks. With protobufs (or similar), any user-friendly editor will need a schema to make sense of the data. JSON-like systems and protobufs have opposite design goals: encouraging versus discouraging schemaless data access.
- tonyg 2y ago
  There are no string keys in the Person example above. You could add some, though, or use numbers instead with the same host-language API: Person = <person @name String @address Address> as above, or Person = <person { @name "name": String @address "address": Address }> or Person = { @name 1: String @address 2: Address } etc. all produce the same host-language record, e.g. in TypeScript export type Person = { name: String, address: Address, };