5 ms·
JSON for Linking Data
- encryptluks2 4y agoI used JSON-LD spec because it is preferred by Google, but it requires a lot of duplicate effort of the actual page content. Its almost as if it is designed for sites that use heavy JavaScript frameworks and not for HTML/CSS sites.
- SquareWheel 4y agoUsually the dynamic data would be generated by the server. eg. if you have an eCommerce site, it might update prices, stock remaining, applicable regions, etc. That would then be pulled into Google Shopping or whatever other service might consume it.
- turnsout 4y agoJSON-LD is the reason ActivityPub hasn’t taken off even more. It’s just such a pain to deal with variables that could be URLs or entire object trees. Especially for strongly typed languages, it’s a nightmare.
- berkes 4y agoIt isn't. It isn't hard: I've built several Serde impls for Rust that handle JSON-api and JSON-LD setups (for a.o. ActivityPub). It's just an Enum and a type with some logic to parse one or the other. An if/else isn't hard. And it certainly is not the reason AP hasn't taken off. For one, because AP has taken off, and secondly because of all the reasons it would potentially not have taken off, the underlying dialect of JSON is the least of all concerns.
- mariusor 4y agoYou're right about serializing/de-serializing the data not being difficult, but the problem lies more in how you handle that data down the pipleine after you deserialized it. Maybe rust has a better typing system that can handle the union of MaybeSliceOfStrings|MaybeSliceOfObjects|MaybeString|MaybeObject|MaybeNull[1] types better than Go does, but I can tell you - after about 4-5 years of trying to implement a clean Go API for processing ActivityPub payloads that the complexity is quite high, at least if you want to build something that can work with more than one client/service. [1] I believe that a good implementation should handle even mixed cases, where you have a slice of both objects and strings.
- talideon 4y agoThat's one of my root annoyances with Go's type system: it lacks sum types. Rust has sum types in the form of enums, and thus unions. People don't appreciate how useful sum types are until they need them, and then they discover how much easier they make things.
- berkes 4y agoIt sounds like the issue is not Json-* formats but Go's type system. And I'm not familiar enough with large go setups, but generally -in any language- anything that must handle data "down the pipeline" should ideally not be bothered about whether this link (or meta, or paginator, or whatever) is there, is of Type-T and so on: that's for middleware/controllers etc to ensure, and to transform. In other words: your business-logic should ideally never have to bother about a link being either a string or a struct or a null etc: it should be able to assume it's a MyLink.
- mariusor 4y agoI'm not sure I understand correctly what you mean by assuming everything to be a "MyLink" but that's not always possible. It depends first on how deep the rabbit hole you want to go with dereferencing URLs - what logic will you employ to decide what level of dereference to full objects you stop at, and, more importantly on being authorized to access the objects pointed at by said URL. Sometimes ActivityPub activities come with properties that are IRIs that when dereferenced are giving you 404 or 401. What does your middleware do then: Retry, Abort, Fail? Also, I'm not sure I agree with your thesis that business logic needs to worry only about a single type of object, because the ActivityPub specification recognizes all of the types I mentioned in my first reply as being "valid" and if you consider their superset as being the "canonical type" you'll be wasting quite a lot of storage (memory and disk wise) when all you have access to is the plain URL.
- UncleEntity 4y agoCouldn’t you just have a RDF database do the heavy lifting and use go for the application logic? I did some deep diving into RDF stuff not too long ago and all the FLOSS databases I found didn’t do json-ld but it also didn’t look all that hard to add. Just adapt the existing xml code methinks.
- lcrz 4y agoIt's only a pain if you try to look at it as a JSON tree instead of a knowledge graph.
- CharlesW 4y agoFor those more familiar with it, didn't this decade-old standard die with other efforts the Semantic Web?
- morelisp 4y agoWith industry bullshit starting to pare down, time for the academia bullshit to ramp up again!
- x-complexity 4y agoIt's reductively the same thing as the Semantic Web, just for JSON. It still suffers from the link rot issue, as well as the left-pad issue. https://news.ycombinator.com/item?id=35322592 https://news.ycombinator.com/item?id=35322592
- endtime 4y agoThe canonical JSON mapping of protocol buffers uses a JSON-LD @type field for google.protobuf.Any. Other than that, the only use I've ever run into is for Schema.org, which is as you say a "semantic web"-era initiative.
- dunham 4y agoBecause of Google, json-ld is very common on recipe web sites on the web. (Recipes for cooking.) It's typically found in a script tag and usually contains the full recipe, even if the page itself is paywalled. This can neatly solve the problem of extracting the recipe to save, and the complaints about having to dig through a page of text to read a recipe. I wrote a little browser plugin that pops up the recipe in a window and offers to save it to Obsidian (using a custom url and hammerspoon). I don't know if the schema.org stuff is in wide use outside of that. It would perhaps be useful to enable a browser to pull things like appointments and contacts out of a web page, but I suspect the world will just go the way of data detection instead.
- karmahunter1234 4y agoGoogle also uses them for other types of structured information to show what they call "rich results": https://developers.google.com/search/docs/appearance/structured-data/search-gallery https://developers.google.com/search/docs/appearance/structu...
- lfciv 4y agoBit of a tangent, but it'll be interesting to see how standards designed to make the web more machine-readable are viewed/treated as more and more content on the web is ultimately consumed and provided to users higher up in the funnel via LLMs
- x-complexity 4y agoI'll be the first to admit this: All I'm seeing here is the JSON version of the semantic web. https://en.wikipedia.org/wiki/Semantic_Web#Example https://en.wikipedia.org/wiki/Semantic_Web#Example Fundamentally, it suffers from the same problems that the semantic web suffers from as well, primarily the top of my concerns being link rot & scenarios involving at least one link in those examples being inaccessible/unrecoverable. For the Semantic Web / JSON-LD to work, the infrastructure behind it needs to permanently keep all of the necessary sources that a file relies on (i.e. permaweb). Otherwise, the entire knowledge graph suffers from a left-pad style situation, wherein the removal of a critical piece of definition(s) will cause the unparsability of all documents reliant on it for definitions.
- hobofan 4y agoYes, JSON-LD is just a different serialization format for RDF. For anyone looking for a solution to the link rot problem (as well as versioning and dependency management), they might be interested in Plow[0]. We built it due to frustration with exactly those drawbacks of the semantic web/data space. At the core it builds on a model very similar to Cargo, with a public index and checksummed artifacts, that allows for easy mirroring for use-cases where all transitive links always need to be resolvable. We are also looking to build out some tooling in the future that makes keeping up those properties easier, such as a linter that ensures that e.g. all referenced concepts in the dependency tree have a definition. (We already have a primitive version of that, but that's just on a single package level.) [0]: https://plow.pm/ https://plow.pm/
- geysersam 4y agoWhat's the advantages of plow over ipfs?
- hobofan 4y agoIPFS alone only gives you content addressing. So you'll need at the minimum some higher-level IPLD structure that allow you to express structures such as an ontology, etc.. That can be based on the Plow model, if you want. If you want a workable ecosystem that enables many decentralized parties to collaborate, the model of not-too-big-not-too-small packages that are interconnected via abstract dependencies (embodied by SemVer ranges), is still the gold standard to achieve this in my opinion. And breaking the mold of traditional monolithic slow-moving ontologies with high technical barriers to publish is one of the main motivating factors for building Plow (together with building a stable ontological layer that you can build other software on). I actually have worked on bridging the gap between semantic data tech and IPFS in the past[0] (both on a fine-granular per-concept level and a more coarse-grained ontology level), and I can just say that there are a ton of additional challenges if you want to do it right (and semtech is already challenging enough as it stands today). Conceptually, the infrastructure parts that make up Plow (the index and artifact store) are also flexible enough that you could distribute them via IPFS. [0]: https://github.com/rlay-project https://github.com/rlay-project
- ec109685 4y agoIt’s a standard in search of anyone who wants to implement it. Meanwhile this already works today: https://www.wikidata.org/wiki/Wikidata:Main_Page https://www.wikidata.org/wiki/Wikidata:Main_Page
- hobofan 4y agoWikidata builds on RDF at its core (the data model behind JSON-LD), and also supports queries with responses in JSON-LD format.
- speedgoose 4y agoIt’s always wikidata.
- KrugerDunnings 4y agoI want to live in an alternative timeline where RDF was never adopted by wikidata but instead created something that solved its specific problems in a human friendly manner. People always point to wikidata as a succesful semantic web project but fail to imagine how much more awesome it could have been. First off, wikidata has little use for ontologies outside of its own domain because all the types are modeled as dynamic second order concepts. Meaning, people organise knowledge using web pages and those webpages are used to structure other knowledge. A simple Sparql query for wikidate would look something like this select * where { ?item wdt:Qe23oieke wd:Ert923Ee3451 . service wikibase:label { bd:serviceParam wikibase:language "en" } } This incomprehensible nonsense is the result of two choices: 1) It had to be language independent because you know the world is bigger than just the anglosphere. 2) It had to be "machine readable", because then the computer can work together in harmony. Putting the cart before the horse, or in this case the machine before the man.
- davidy123 4y agoWikidata was consciously designed without consideration for the semantic web/rdf. I remember being dismayed by this, but they added some facilities later. It is designed around a purpose built data model. https://www.mediawiki.org/wiki/Wikibase/DataModel https://www.mediawiki.org/wiki/Wikibase/DataModel
- vaughan 4y agoWhy isn’t semantic web more popular inside companies? This expandable graph seems like the purest representation of all data in a company. Instead we have several different databases each with their own schemas. All with entities related to one another, but without these relations defined anywhere, and impossible to query. RDBMS are implementation details of this graph that involve a heap of manual work. Wouldn’t it be great if the rdbms was generated from this graph. There are so many interconnections between the data involved in the design and implementation of a product and the product database itself. Imagine hovering over a UI element and seeing who implemented it and when, what project it was part of, why the project was initiated, and what kpis and goals it contributes to.
- dym_sh 4y agosame reason wikis arent, it takes too much effort to maintain and keep it semantic instead of copypasted pile of text you'd need CMS, CRM, knowledge base, documentation, source control, chat/forum, inventory, point of sale, customer support channels, issues ticketing system, and every other thing interconnected to with each other do you know of anyone offering this as a complex turn-key integrated solution at a reasonable price?
- WJW 4y agoI know of several ERP vendors offering such products at a price that is reasonable for such complexity. "Reasonable" is not the same as "cheap" though.
- dym_sh 4y agowell, i would like a couple of links if that's not too much to ask
- vaughan 4y agoSounds like an opportunity. In every category you mentioned there are new products released and widely adopted all the time. Look at the rise of ClickUp for example in such a crowded space as project management. I don't think its too farfetched to one day see a new entrant offering as a key selling point that their api is just a big interoperable graph that you can easily plug your company into...and that's not just GraphQL.
- dym_sh 4y agonext step n the json path is efficiency: https://bsonspec.org/ https://bsonspec.org/
- berkes 4y agoEvery time I get to work with JSON-LD (and json-schema and so on) I keep asking myself: how is this better than XML? (I am convinced it isn't).
- lcrz 4y agojson-schema has nothing to do with JSON-LD. If you want shape constraints on your JSON-LD data, you'll need SHACL: https://www.w3.org/TR/shacl/ https://www.w3.org/TR/shacl/ JSON itself has a much simpler specification than XML, and feels much more lightweight for a lot of devs. Under the hood though, linked data is all RDF.
- HelloNurse 4y ago"Over the hood" JSON is less complicated but less correct than XML, not a good positioning for long-term data storage.
- seabass-labrax 4y agoAre you claiming that A: plain (non-LD) JSON is less correct than plain XML, or that B: JSON-LD is less correct than RDF/XML?
- berkes 4y agoI wasn't implying they are related in that way. Just that both make me think "this thing has long been solved in XML, why re-invent it in JSON?"
- matja 4y agoI was just thinking this looks similar to a DDI (Data Documentation Initiative) citation element, but the advantage of the DDI XML schema is that one can annotate any existing element in an XML document by importing the DDI namespace, which shouldn't change how the document is otherwise interpreted by applications, but this JSON schema as far as I can tell is a stand-alone document?
- 4y ago
- zeven7 4y agoIs it not strange that the first example lists Cynthia Lennon as John Lennon's spouse instead of Yoko?
- tgv 4y agoIt's just an example, but it does highlight another problem. If your model has a single spouse, it's going to be wrong. It needs a list, with time periods. But then, why stop at spouses? Surely you want to include relationships that were not sanctified. But why stop at single partnership? Some people have multiple partners. And people have more than romantic/partner-type relationships: parents, step-parents, children, friends, bosses, colleagues. People are going to extend the model, and not update the records, because that's a hassle. And there will be multiple versions of a record. What you'd really want is an entity that takes care of all of this, in one place. And not machine readable, because who is going to program a machine to find out the marriages of John Lennon? Something like an encyclopedia, perhaps.
- jonnycomputer 4y agoIt's actually a problem with the semantic web generally: it tends to assume that the world is compartmentalizable into neat categories. Instead, we live in a world where (a) membership in categories is partial (b) membership in categories is probabilistic (c) membership in categories is contested (see Pluto for an innocuous example) (d) definition, legitimacy, etc. of categories is contested (e) category boundaries are vague, e.g. Sorites paradox: Base step: A one day old human being is a child. Induction step: If an n day old human being is a child, then that human being is also a child when it is n+1 days old. Conclusion: Therefore, a 36,500 day old human being is a child. (f) membership in categories changes with time etc... There are hot things, and there are cold things, and then there are things that are neither hot nor cold.
- singularity2001 4y agoThere could be semantic conflict: "spouse": "http://dbpedia.org/resource/Cynthia_Lennon" Entity "spouse": "Cynthia Lennon" String value "spouse": "http://my_url.com/Cynthia_Lennon.mpg" Url Also please use Json5 to allow comments, trailing commas and get rid of superfluous '"'
- seabass-labrax 4y agoThe first line of the example on the JSON-LD website is: "@context": "https://json-ld.org/contexts/person.jsonld", This is a 'context map', which is what links the short name "spouse" for the JSON key to a particular class defined in a RDF (Resource Description Framework) ontology. Here, that's <https://schema.org/spouse https://schema.org/spouse>. These ontologies are what make it unambiguous to the parser that the content is a IRI/URL rather than a literal string. Alternatively, one can write JSON-LD without a context map by using the full IRI of the class as the key: "https://schema.org/spouse": "https://json-ld.org/contexts/person.jsonld",
- thinkingemote 4y agoAnyone know if JSON-LD is being used more in very recent years for AI? Saw a couple of comments elsewhere that many large companies are using it now. If so, how is is (and any knowledge graph / linked data) used in these situations?
- jillesvangurp 4y agoHaven't come across this in the wild. My impression is that most of the semantic web people that were obsessing about ontologies twenty years ago have moved on. As for AI, we now have deep learning approaches that don't require an investment in a lot of machine readable, disambiguated data but is as good or better than we are of making sense of unstructured data. That probably explains the low interest in semantic web type stuff at this point.
- seabass-labrax 4y agoThe ones that have moved on are being replaced by newcomers though - as evidenced by my multiplicity of my comments on this post, I'm one of them ;) I would agree with your appraisal of AI - at the moment, there is a much greater return-on-investment for processing vast quantities of unstructured data than there is for meticulously curating knowledge graphs. However, I think that, in time, the balance will shift to vindicate the semantic web as people desire more trustworthiness from automated systems.
- jillesvangurp 4y agoWhy? What's changed to switch the balance from machine learned back in favor of painstakingly manually curated? Sounds a bit like wishful thinking to me. I don't think that cat will jump back into the bag and zip the bag up behind itself. Machines are only going to continue to outpace humans when it comes to pattern recognition, classifying things, or making sense of unstructured data. I don't see that turning around. Maybe there will be a market for "artisanal ontologies". But I wouldn't get my hopes up for that one.
- UncleEntity 4y ago
- dathinab 4y agoyay good ol' jsonld. I love and hate it. Having semantic metadata weather in from of jsonld or others is pretty nice. But then in my experiences most systems having it either: - don't really use it, could simply not have had it, likely for problems with the metadata to not be detected - use it but end up with a horrible UX - use it, decent UX but extremely complex And that is under the assumption that for projects which can use existing tooling, but in a lot of contexts there is way too little tooling for this.
- seabass-labrax 4y agoJSON-LD is an open standard for expressing RDF data as JSON. RDF is the most fundamental part of the W3C's Semantic Web and Linked Data projects, which began at the end of the 1990s to make the Web more machine-readable and continue to make steady progress. If you're not familiar with RDF, I would suggest starting by reading the RDF 1.1 Primer[1], using the RDF 1.1 Concepts and Abstract Syntax as a reference[2] if something is confusing. I don't think you'll regret spending the time; RDF is a fascinating field! [1]: https://www.w3.org/TR/rdf11-primer/ https://www.w3.org/TR/rdf11-primer/ [2]: https://www.w3.org/TR/rdf11-concepts/ https://www.w3.org/TR/rdf11-concepts/
- whartung 4y ago> I don't think you'll regret spending the time; RDF is a fascinating field! I love RDF. I'm using it for a project of my own, which has nothing to do with semantic web. I'm not using any of the standard ontologies, or OWL, or anything like that. I just wanted an extensible, schemaless data model, because I have no idea how folks will want to correlate data with each other. Being able to hang anything off of anything is important. I'm also not processing trillions of triples for my work (if you ever look at triple stores, they like to taut how fast they can import vast troves of data). Mind, I'm no doubt reinventing some stuff that I could probably be using a standard vocabulary for, but since global interoperability is not a goal, I'm not that concerned about it. I also have some concept of structure that I'm capturing, and encoding, and I perhaps could be using something like OWL for, but this is ad hoc doing things as I go and while there may well be gems within those spaces I can leverage, I'd rather make progress on my own path at the moment than fall into those large maws of times trying to suss them out to see if I could adapt them to my project. SPARQL is an odd duck I'm still wrapping my head around (30 years of SQL warps ones point of view), but at least it exists, and I can use it.