5 ms·

Can anyone mention some interesting projects/use-case for JSON Schema? I have terrible WSDL flashbacks from a past project, where the abstraction layer was alw

by mihaic 4y ago

Can anyone mention some interesting projects/use-case for JSON Schema? I have terrible WSDL flashbacks from a past project, where the abstraction layer was always either too high (something simpler was better) or too low (and written documentation was better).

relequestual 4y ago
Here's a YouTube playlist of 6 case studies with companies of various sizes: https://www.youtube.com/watch?v=fkziMQD7pqQ&list=PLHVhS4Tj1YZOrrvl7_a9LaBAtst7BWH8a https://www.youtube.com/watch?v=fkziMQD7pqQ&list=PLHVhS4Tj1Y... There are also two written case studies on the JSON Schema blog: https://json-schema.org/blog/ https://json-schema.org/blog/
pramodbiligiri 4y ago
Here is one: OpenMetadata uses it extensively to help you catalog the various types of (software-related) entities you have in your organization. Each entity (tables, alerts, api endpoints..) has been given a set of fields that you can populate (https://docs.open-metadata.org/main-concepts/metadata-standard/schemas https://docs.open-metadata.org/main-concepts/metadata-standa...). The schema language used for this is JSON Schema.
EdwardDiego 4y ago
It's much the same as an XML schema, or K8s CRDs that have a schema embedded into the definition - schema are very useful for ensuring your data structure is valid. You can use the schema to validate that data is structured correctly, then apply business rule validations afterwards. And most importantly, people creating data you're ingesting have a thorough schema to validate against. The JSON Schema documentation was rather terrible for a long time, but has significantly improved. Now the fun part is making sure that the version of JSON Schema you're using is supported by the library you're using. But JSON Schema is far better than OpenAPI because the latter is mostly focused on API description, which adds a lot of noise when you're trying to simply describe a data structure. And OpenAPI is also hampered by the tight binding to Swagger. For a long time OpenAPI disallowed union types because it caused issues with Swagger's codegen. I think that latest versions of OpenAPI allow union types, but I don't think Swagger supports those versions yet. At least they didn't the last time I looked.
- jspdown 4y ago
  Open API schema objects are define using JSON Schema. The latest version even let you specify the JSON Schema dialect.
- crabbone 4y ago
  > It's much the same as an XML schema, That's wishful thinking. I can find a lot of differences, but want to point out this one to underscore the difference in approach and level of sophistication. So, in XSL (which is one of the XML schema languages), there's a sequence element. This element describes how tags are supposed to appear in certain order. The analogous structure in JSON would be the dictionary keys, which are also allowed to repeat... but the schema doesn't even cover that option! To make this more concrete, below, is a valid JSON: { "x": 1, "x": "2" } But there's no way of describing in the schema what happens here. Not only this, XSL can restrict the number of times a certain element may repeat... but JSON schema cannot still figure out what to do about this. There are plenty of things that can be implemented in various XML schema languages, but have nothing analogous to them in JSON schema. Another example would be ensuring uniqueness of some particular type of values across the entire document.
  - 9dev 4y ago
    Just because you can doesn’t mean you should. If someone told me tags must appear in a specific sequence for their markup to work, I call the implementation flawed. That’s been the downfall of XML - doing way too much for it’s own good.
    - crabbone 4y ago
      > Just because you can doesn’t mean you should. This is where you are wrong. If you are making a validator, if the format can do something, the validator should deal with that possibility. If validator is incomplete wrt' validated features, it cannot be used to guarantee the document validity, which is... well, it's most important function. It has nothing to do with your emotional attachments to tags and sequences.
9dev 4y ago
We use Elasticsearch to store a vast amount of JSON documents in a specific format. Elasticsearch uses a strict schema to validate new items. Those documents are generated and used by several downstream projects. We have a single JSON schema installable as a package in all of our projects which is used in unit tests, and converted to the Elasticsearch schema format for reindexing. Effectively, this ensures we only have compatible documents in the index; all applications running in prod are verified to be able to handle them. Thus, we use JSON schema as an universally valid way to exchange the data schema.
ttiurani 4y ago
A team of 10 devs I was a part of got lots of everyday value from JSON schemas (w/ code generation) being the single source of truth of the domain model for a Typescript frontend and Java backend. I would absolutely choose JSON schemas again for any complex enough project that chooses typed languages for both backend and frontend. (And JSON as the protocol, naturally.)
_ZeD_ 4y ago
> Can anyone mention some interesting projects/use-case for JSON Schema? finally come back to WSDL and SOAP. sorry, you may have "terrible flashbacks", but I enjoyed those days, where I could have clients and server code auto generated, with automatic validation and so on, and I felt all these "unstructured" data sent over json as "dark ages". I'm tired of writing again yet another rest client payload mapping by hand...
- mihaic 4y ago
  I actually did miss the feeling of being safe in the serialization/deserialization of my data when JSON became widespread, you're right. These flashback were specifically about WSDL maintenance by hand (it was not very human readable), and how the existence of a schema was used as an excuse to not actually document the meaning of certain fields. I guess it depends a lot on people, but it seems like most systems need to be robust to idiots to some degree the more widespread they become.
- wolpoli 4y ago
  I agree, WSDL were great. There was no need to keep checking the API doc because the client codes were auto-generated. If we can't have JSON Schema, perhaps we could have a large language model auto generate our client codes by ingesting the API doc.
IshKebab 4y ago
It makes editing JSON in IDEs way way nicer. Worth it for that alone IMO.

by mihaic 4y ago

relequestual 4y ago

Here's a YouTube playlist of 6 case studies with companies of various sizes: https://www.youtube.com/watch?v=fkziMQD7pqQ&list=PLHVhS4Tj1YZOrrvl7_a9LaBAtst7BWH8a https://www.youtube.com/watch?v=fkziMQD7pqQ&list=PLHVhS4Tj1Y... There are also two written case studies on the JSON Schema blog: https://json-schema.org/blog/ https://json-schema.org/blog/

pramodbiligiri 4y ago

Here is one: OpenMetadata uses it extensively to help you catalog the various types of (software-related) entities you have in your organization. Each entity (tables, alerts, api endpoints..) has been given a set of fields that you can populate (https://docs.open-metadata.org/main-concepts/metadata-standard/schemas https://docs.open-metadata.org/main-concepts/metadata-standa...). The schema language used for this is JSON Schema.

EdwardDiego 4y ago

It's much the same as an XML schema, or K8s CRDs that have a schema embedded into the definition - schema are very useful for ensuring your data structure is valid. You can use the schema to validate that data is structured correctly, then apply business rule validations afterwards. And most importantly, people creating data you're ingesting have a thorough schema to validate against. The JSON Schema documentation was rather terrible for a long time, but has significantly improved. Now the fun part is making sure that the version of JSON Schema you're using is supported by the library you're using. But JSON Schema is far better than OpenAPI because the latter is mostly focused on API description, which adds a lot of noise when you're trying to simply describe a data structure. And OpenAPI is also hampered by the tight binding to Swagger. For a long time OpenAPI disallowed union types because it caused issues with Swagger's codegen. I think that latest versions of OpenAPI allow union types, but I don't think Swagger supports those versions yet. At least they didn't the last time I looked.

jspdown 4y ago
Open API schema objects are define using JSON Schema. The latest version even let you specify the JSON Schema dialect.
crabbone 4y ago
> It's much the same as an XML schema, That's wishful thinking. I can find a lot of differences, but want to point out this one to underscore the difference in approach and level of sophistication. So, in XSL (which is one of the XML schema languages), there's a sequence element. This element describes how tags are supposed to appear in certain order. The analogous structure in JSON would be the dictionary keys, which are also allowed to repeat... but the schema doesn't even cover that option! To make this more concrete, below, is a valid JSON: { "x": 1, "x": "2" } But there's no way of describing in the schema what happens here. Not only this, XSL can restrict the number of times a certain element may repeat... but JSON schema cannot still figure out what to do about this. There are plenty of things that can be implemented in various XML schema languages, but have nothing analogous to them in JSON schema. Another example would be ensuring uniqueness of some particular type of values across the entire document.
- 9dev 4y ago
  Just because you can doesn’t mean you should. If someone told me tags must appear in a specific sequence for their markup to work, I call the implementation flawed. That’s been the downfall of XML - doing way too much for it’s own good.
  - crabbone 4y ago
    > Just because you can doesn’t mean you should. This is where you are wrong. If you are making a validator, if the format can do something, the validator should deal with that possibility. If validator is incomplete wrt' validated features, it cannot be used to guarantee the document validity, which is... well, it's most important function. It has nothing to do with your emotional attachments to tags and sequences.

9dev 4y ago

We use Elasticsearch to store a vast amount of JSON documents in a specific format. Elasticsearch uses a strict schema to validate new items. Those documents are generated and used by several downstream projects. We have a single JSON schema installable as a package in all of our projects which is used in unit tests, and converted to the Elasticsearch schema format for reindexing. Effectively, this ensures we only have compatible documents in the index; all applications running in prod are verified to be able to handle them. Thus, we use JSON schema as an universally valid way to exchange the data schema.

ttiurani 4y ago

A team of 10 devs I was a part of got lots of everyday value from JSON schemas (w/ code generation) being the single source of truth of the domain model for a Typescript frontend and Java backend. I would absolutely choose JSON schemas again for any complex enough project that chooses typed languages for both backend and frontend. (And JSON as the protocol, naturally.)

_ZeD_ 4y ago

> Can anyone mention some interesting projects/use-case for JSON Schema? finally come back to WSDL and SOAP. sorry, you may have "terrible flashbacks", but I enjoyed those days, where I could have clients and server code auto generated, with automatic validation and so on, and I felt all these "unstructured" data sent over json as "dark ages". I'm tired of writing again yet another rest client payload mapping by hand...

mihaic 4y ago
I actually did miss the feeling of being safe in the serialization/deserialization of my data when JSON became widespread, you're right. These flashback were specifically about WSDL maintenance by hand (it was not very human readable), and how the existence of a schema was used as an excuse to not actually document the meaning of certain fields. I guess it depends a lot on people, but it seems like most systems need to be robust to idiots to some degree the more widespread they become.
wolpoli 4y ago
I agree, WSDL were great. There was no need to keep checking the API doc because the client codes were auto-generated. If we can't have JSON Schema, perhaps we could have a large language model auto generate our client codes by ingesting the API doc.

IshKebab 4y ago

It makes editing JSON in IDEs way way nicer. Worth it for that alone IMO.