8 ms·
The Last Breaking Change
- beachy 4y agoAdmiring the chutzpah of declaring this will be the "last breaking change". We'll see! And perhaps json schema should borrow from http header conventions of old and allow unknown elements if prefixed in some way e.g: X-my-funky-element: true
- crabbone 4y agoIt's probably "last" as in "latest".
- Supermancho 4y agoSMH > A couple years later, they want to upgrade the validator now that supports the 2025 specification. However in that 2025 specification, we added database-field-id as a keyword, and its value is expected to be a string. Suddenly the user's schema is no longer valid. We've broken a user by adding a new keyword. No you haven't. They chose to change. Same as if they change to another schema format altogether (eg XML). Is this implying a user is going commit to a schema (pragmatic laziness, notwithstanding) that they haven't read? Regardless, their following solution baffles... > Therefore, no keyword addition can be considered safe, making forward compatibility impossible to guarantee. Correct. > Using the vocabulary requires writing a custom meta-schema that adds the vocabulary URI and references the associated meta-schema, and referencing that custom meta-schema in the schema. I just can't fathom this ceaseless drive toward cruft. Now a meta-vocabulary (a sub-schema). That's supposed to be better? Change will come, in 10 years, 50 years, etc. Semantic versioning normalizes it in the json-schema itself. It's as good a system as any and definitely better than adding another versioned dependency.
- salmonellaeater 4y ago> But many people also said they would be less bothered if there was a defined migration path and tooling to help. If things were left as-is (allowing unknown keywords), it would be straightforward to write a tool that checks "upgrade compatibility" for a schema. If it's using keywords that were unknown under the old JSON Schema version but are now supported, the schema must be fixed before the JSON Schema version can be upgraded.
- WirelessGigabit 4y agoDo it like OpenAPI does. Separate structure from properties and type. And if you want a fixed value your type is a value. That way you don't mix the JSONSchema's schema is separate from the thing you're describing.
- EdwardDiego 4y agoDoes OpenAPI support union types yet? More importantly, if it does, does Swagger support the OpenAPI version that does?
- andreareina 4y agoWe have union types defined in fastapi/pydantic and they do show up in the api explorer. Tagged unions don't work but I think that's more to do with us/python than openapi.
- EdwardDiego 4y agoAh, that's nice. I think a year or so ago I was dealing with api docs that'd been run through Swagger and it just stopped emitting fields when it hit union types.
- junon 4y agoUnderscore prefixes like C. Dedicated property to act as kitchen sinks, e.g. "custom", that says "anything in here is implementation specific". Simply not allowing custom keywords at all, also a good option. Have users write higher level schemas that compile down to jsonschema. All viable options.
- Karellen 4y agoThe thing that confuses me is: > A keyword is "known" if it is defined by a vocabulary listed in the schema's meta-schema (identified by the value in $schema). { "$schema": "https://json-schema.org/draft/2020-12/schema", So, if a schema identifies as conforming to the 2020-12 meta-schema, why would a new "database-field-id" keyword in the 2025 upgrade be an issue? The 2025 parsers should know that it's not a keyword in the 2020-12 meta-schema, and treat it the same as they always did, shouldn't they? But also - as an alternate to underscore prefixes, could new keywords just reuse the "$" prefix as in "$schema"?
- mst 4y agoI've always been fond of _x_ as a prefix for such things.
- superb-owl 4y agoI prefer letting people use arbitrary keys, with a _caveat emptor_. It's much cleaner and easier to use. And it the vast majority of cases there will never be a collision. You can also pin your schema version with `$schema`, so breaking changes should be opt-in only.
- deleted 4y ago[deleted]
- mihaic 4y agoCan anyone mention some interesting projects/use-case for JSON Schema? I have terrible WSDL flashbacks from a past project, where the abstraction layer was always either too high (something simpler was better) or too low (and written documentation was better).
- relequestual 4y agoHere's a YouTube playlist of 6 case studies with companies of various sizes: https://www.youtube.com/watch?v=fkziMQD7pqQ&list=PLHVhS4Tj1YZOrrvl7_a9LaBAtst7BWH8a https://www.youtube.com/watch?v=fkziMQD7pqQ&list=PLHVhS4Tj1Y... There are also two written case studies on the JSON Schema blog: https://json-schema.org/blog/ https://json-schema.org/blog/
- pramodbiligiri 4y agoHere is one: OpenMetadata uses it extensively to help you catalog the various types of (software-related) entities you have in your organization. Each entity (tables, alerts, api endpoints..) has been given a set of fields that you can populate (https://docs.open-metadata.org/main-concepts/metadata-standard/schemas https://docs.open-metadata.org/main-concepts/metadata-standa...). The schema language used for this is JSON Schema.
- EdwardDiego 4y agoIt's much the same as an XML schema, or K8s CRDs that have a schema embedded into the definition - schema are very useful for ensuring your data structure is valid. You can use the schema to validate that data is structured correctly, then apply business rule validations afterwards. And most importantly, people creating data you're ingesting have a thorough schema to validate against. The JSON Schema documentation was rather terrible for a long time, but has significantly improved. Now the fun part is making sure that the version of JSON Schema you're using is supported by the library you're using. But JSON Schema is far better than OpenAPI because the latter is mostly focused on API description, which adds a lot of noise when you're trying to simply describe a data structure. And OpenAPI is also hampered by the tight binding to Swagger. For a long time OpenAPI disallowed union types because it caused issues with Swagger's codegen. I think that latest versions of OpenAPI allow union types, but I don't think Swagger supports those versions yet. At least they didn't the last time I looked.
- rektide 4y agoInstead of making a brutal narrow spec that snubs ease of use & extensibility, just to create a "last breaking change" forever spec, the spec should definitely just keep dropping new major versions when it needs to change. This is an ugly & rough conceitedness propsed here: no one can do anythint, because we reserve the right to do anything, and we dont want to have to drop a new major version to do it. Craven attitude. Building ultra-strict narrow protocols like this should be highly highly discouraged. This is such a mean & awful tradeoff to foist upon people; I cant see any advantage to making a brittle, narrow, but forever "safe" little path like this. Be braver.
- vincnetas 4y agoAre you saying that exact and precise specifications are bad? Can you elaborate, because i always thought that the opposite is bad.
- deleted 4y ago[deleted]
- EdwardDiego 4y ago> brutal > ugly & rough conceitedness > Craven attitude > mean & awful Why do you phrase it in such emotive terms?
- rektide 4y agoIt's a terrible & self-interested & small-minded design philosophy making awful trade-offs. It's an actively dangerous way of building, to the exclusion of allowing possibility, and for false gain ("no new major versions but it will totally change maybe & no one else has freedom"). I use strong terms because this breaks so many internet ethos, logos, & pathos. Picking the narrowest path should be warded & hazarded against in strong terms. This reaction is emotional because it's terrible terrible choices. Postel's Law[1] has come under fire from some pretty conservative-minded viewsets (and frankly I'be rarely been persuaded by the angst against permit-if-you-can), but this is a next level assault on loose-coupling, a stark new frontier in reserving almost all possibility for yourself. My description while strong is accurate in many ways: this is brutally narrow. This is an extremeist perspective that json schema is embarking towards, snubbing basic protocol design & rejecting a non-harmful extensibility that cost them nothing (besides needing to tick a major version to make a major change). [1] https://en.m.wikipedia.org/wiki/Robustness_principle https://en.m.wikipedia.org/wiki/Robustness_principle
- smallstepforman 4y agoFor a format designed to be human readable, JSON not supporting comments is a major let down. And not supporting trailing comma.
- speedgoose 4y agoYou have the superset JSON5 https://json5.org/ https://json5.org/ but it’s quite rare compared to JSON.
- singularity2001 4y agoAdding comments to json is the one breaking change the world needs. As a parser if you stumble over a comment it's your fault! nothing in the world could stop you from just sanely ignoring comments.
- alpaca128 4y agoJSON5 exists and allows both comments and trailing commas. It's basically like defining objects in plain JavaScript syntax.
- jgalt212 4y agowhat's the difference between JSON5 and the config language that vs code uses ,JSON + comments (as best I can tell).
- nicoburns 4y agoJSON5 also allows extra things like single-quoting strings and multiline strings. VS Code uses JSONC which is strictly trailing commas and comments on top of JSON
- grey-area 4y agoWhat’s wrong with using a ‘comment’ key? I’d say not supporting large ints is worse. https://stackoverflow.com/questions/209869/what-is-the-accepted-way-to-send-64-bit-values-over-json https://stackoverflow.com/questions/209869/what-is-the-accep...
- joeblubaugh 4y agoThis takes JSON schema from a fairly lightweight, flexible definition to one of the most rigid. As much as I have a hairs time with dynamic languages, I can’t get with this. Feels contrary to the spirit of the JS ecosystem in the first place. Makes CUE a much more attractive way to directly validate JSON - you can declare with struct a are open or closed as part of the definition.
- IshKebab 4y agoMy issue with Cue is that there's no actual way to use it as a schema. At least I couldn't find a way to have one cue document link to another cue document so that e.g. IDEs can use the information. Also the only language Cue supports is Go at the moment which makes it a bit of a non starter IMO. It does have a quite elegant design, but I'm not sure it really solves enough to be worth the hassle, especially given that it doesn't attempt to solve the "make config less tedious using functions" problem.
- crabbone 4y agoI could never bring myself to take JSON and family seriously. This is just another indication. I mean they didn't consider that it's not possible to have things that escape validation being made to validate in the future version? And these are the people which are making the validator software?
- beeforpork 4y agoIf the JSON file had a version number for the spec, then could could support future versions without breaking it, ever. Just define that all future versions will include the previous versions. E.g., specify that if a file is meant to be compliant with the next JSON version, then it needs to start with #JSON1 And the next one with #JSON2. And if a JSON2 parser finds a JSON1 file, it should interpret it as JSON1. If a JSON1 parser finds a JSON2 file, it should try its best to parse it anyway, because your changes are not going to arbitrarily break stuff, right? This is a breaking change, too, but it may indeed be the last one...
- ChrisMarshallNY 4y agoWell, time will tell. If they get it right, people will use it. If they don’t, it will wither on the vine. Pretty darwinian, but This Is The Way. XML Schema is a fiendishly complex, awkward, and verbose standard, but it is very much in use. Personally, I hate it, but I have used it fairly often. I would like it to succeed, and will use it, if it does; even if it’s ugly. If it doesn’t work, I won’t use it. My experience tells me that it can be next to impossible to predict what will and won’t work, and there’s probably no replacement for actually putting it out there and trying. My experience also tells me that the most certain way to ensure failure, is to try predicting the future, and engineering for a very specific one. Personally, I have had good luck, with an “heuristic” approach, where my designs act as a “lattice” for growth, and I evolve it, as I see the direction the project takes. The main thing, in my experience, is to never have to go back and change stuff that has already been done. I may do new stuff, in the future, that breaks the API, but I should always honor The Old Ways.
- ChrisMarshallNY 4y ago> The main thing, in my experience, is to never have to go back and change stuff that has already been done. I may do new stuff, in the future, that breaks the API, but I should always honor The Old Ways. I just realized that sentence doesn't make sense. I meant that people will build on an API, and make assumptions to "fill in the gaps." As long as there's nothing in the current API that "codifies" these assumptions, I consider it OK, to add stuff that breaks the assumptions, but I won't go and make changes to stuff I've codified. An example is some work I was doing, with an Apple UIKit API. I was calling a set of functions in a particular order. Apple released a system update that reset one of the settings, so I needed to move the call that affected that setting, to the end of the chain. They never specified the order (nor did they do so, after that, even though they probably should have), so I had no room to complain.