10 ms·
The semantic web is now widely adopted
- tossandthrow 2y agoIn all honesty, llms are probably going to make all this entirely redundant. As such semantic web was not a natural follower to what we had before, and not web 3.0.
- asymmetric 2y agoHave you read the article? It addresses this point towards the end.
- tannhaeuser 2y agoAnd it fails to address why SemWeb failed in its heyday: that there's no business case for releasing open data of any kind "on the web" (unless you're wikidata or otherwise financed via public money) the only consequence being that 1. you get less clicks 2. you make it easier for your competitors (including Google) to aggregate your data. And that hasn't changed with LLMs, quite the opposite. To think a turd such as JSON-LD can save the "SemWeb" (which doesn't really exist), and even add CSV as yet another RDF format to appease "JSON scientists" lol seems beyond absurd. Also, Facebook's Open Graph annotations in HTML meta-links are/were probably the most widespread (trivial) implementation of SemWeb. SemWeb isn't terrible but is entirely driven by TBL's long-standing enthusiasm for edge-labelled graph-like databases (predating even his WWW efforts eg [1]), plus academia's need for topics to produce papers on. It's a good thing to let it go in the last decade and re-focus on other/classic logic apps such as Prolog and SAT solvers. [1]: https://en.wikipedia.org/wiki/ENQUIRE https://en.wikipedia.org/wiki/ENQUIRE
- tossandthrow 2y agoyes
- peterlk 2y agoThe article addresses this point with the following: > It would of course be possible to sic Chatty-Jeeps on the raw markup and have it extract all of this stuff automatically. But there are some good reasons why not. > > The first is that large language models (LLMs) routinely get stuff wrong. If you want bots to get it right, provide the metadata to ensure that they do. > > The second is that requiring an LLM to read the web is throughly disproportionate and exclusionary. Everyone parsing the web would need to be paying for pricy GPU time to parse out the meaning of the web. It would feel bizarre if "technological progress" meant that fat GPUs were required for computers to read web pages.
- tsimionescu 2y agoThe first point is moot, because human annotation would also have some amount of error, either through mistakes (interns being paid nothing to add it) or maliciously (SEO). Plus, human annotation would be multi-lingual, which leads to a host of other problems that LLMs don't have to the same extent. The second point is silly, because there is no reason for everyone to train their own LLMs on the raw web. You'd have a few companies or projects that handle the LLM training, and everyone else uses those LLMs. I'm not a big fan of LLMs, and not even a big believer in their future, but I still think they have a much better chance of being useful for these types of tasks than the semantic web. Semantic web is a dead idea, people should really allow it to rest.
- tossandthrow 2y agoWhile both of these points a valid today they are likely going to be invalidated going forward - assume that what you can conceive is technically possible will become technically possible. In 5 years resource price is likely negligible and accuracy is high enough that you just trust it.
- null_investor 2y agoIt's HN, most people don't read the article and jump into whatever conclusion they have at the moment despite not being an expert in the field.
- xoac 2y agoHe had it summarized by chatgpt
- tossandthrow 2y agoAs I already pointed out, none of the arguments the author brings up are really relevant. Resources and accuracy will not be a concern in 5 years. What makes you think that I am not an expert btw? It indeed seems like you appear to believ that what's written on the internet is true. So if someone writes that LLMs are not a contester to semantic web - then it might be true. Could it be, that I merely challenge that author of the blog article and don't take his predictions for granted?
- aaron695 2y ago[dead]
- mg 2y agoThe author gives two reasons why AI won't replace the need for metadata: 1: LLMs "routinely get stuff wrong" 2: "pricy GPU time" 1: I make a lot of tests on how well LLMs get categorization and data extraction right or wrong for my Product Chart (https://www.productchart.com https://www.productchart.com) project. And they get pretty hard stuff right 99% of the time already. This will only improve. 2: Loading the frontpage of Reddit takes hundreds of http requests, parses megabytes of text, image and JavaScript code. In the past, this would have been seen as an impossible task to just show some links to articles. In the near future, nobody will see passing a text through an LLM as a noteworthy amount of compute anymore.
- monero-xmr 2y agoLLMs have no soul, so I like content and curation from real people
- doe_eyes 2y agoThe main problem is that the incentive for well-intentioned people to add detailed and accurate metadata is much lower than the incentive for SEO dudes to abuse the system if the metadata is used for anything of consequence. There's a reason why search engines that trusted website metadata went extinct. That's the whole benefit of using LLMs for categorization: they work for you, not for the SEO guy... well, prompt injection tricks aside.
- monero-xmr 2y agoThere is value-add if you can prove whatever content you are producing is from an authentic human, because I dislike LLM produced garbage
- usrusr 2y agoThe point is that metadata lies. Intentionally, instead of just being coincidentally wrong. For example everybody who wants to spew LLM produced garbage in your face will go out of their way to attach metadata claiming the opposite. The value proposition of LLM categorization would be that the LLM looks at the same content as the eventual human (if, in fact, it does - which is a related but different problem)
- Devasta 2y ago> Before JSON-LD there was a nest of other, more XMLy, standards emitted by the various web steering groups. These actually have very, very deep support in many places (for example in library and archival systems) but on the open web they are not a goer. If archival systems and library's are using XML, wouldn't it be preferable to follow their lead and whatever standards they are using? Since they are the ones who are going to use this stuff most, most likely. If nothing else, you can add a processing instruction to the document they use to convert it to HTML.
- whartung 2y agoThe format really isn’t much of an issue. From an information point of view, the content of the different formats are identical, and translation among them is straightforward. Promoting JSON-LD potentially makes it more palatable to the modern web creators, perhaps increasing adoption. The bots have already adapted.
- cess11 2y agoYou're aware of straightforward translations to and from E-ARK SIP and CSIP? Between what formats? As far as I can tell archivists don't care about "modern web creators", and they likely shouldn't, since archiving is a long term project. I know I don't, and I'm only building software for digital archiving.
- tannhaeuser 2y agoIf by that the author means JSON-LD has replaced MarcXML, BibTex records, and other bibliographic information systems, then that's very much not the case.
- AlecSchueler 2y agoThey recognise that in the quoted paragraph. The JSON-LD thing was only about the open web: > [MarcXML, BibTex etc] actually have very, very deep support in many places (for example in library and archival systems) but on the open web they are not a goer.
- npunt 2y agoThe argument about LLMs is wrong, not because of reasons stated but because semantic meaning shouldn't solely be defined by the publisher. The real question is whether the average publisher is better than an LLM at accurately classifying their content. My guess is, when it comes to categorization and summarization, an LLM is going to handily win. An easy test is: are publishers experts on topics they talk about? The truth of the internet is no, they're not usually. The entire world of SEO hacks, blogspam, etc exists because publishers were the only source of truth that the search engine used to determine meaning and quality, which has created all the sorts of misaligned incentives that we've lived with for the past 25 years. At best there are some things publishers can provide as guidance for an LLM, social card, etc, but it can't be the only truth of the content. Perhaps we will only really reach the promise of 'the semantic web' when we've adequately overcome the principal-agent problem of who gets to define the meaning of things on the web. My sense is that requires classifiers that are controlled by users.
- atoav 2y agoYet LLMS fail to make these simple but sometimes meaningful differentiation. See for example this case in which a court reporter is described as being all the things he reported about by Copilot: a child molester, a psychatric escapee, a widow cheat. Presumably because his name was in a lot of articles about said things and LLMS simply associate his name with the crimes without making the connection that he could in fact be simply the messenger and not the criminal. If LLMS had the semantic understanding that the name on top/bottom of a news article is the author, it would not have made that mistake. https://www.heise.de/en/news/Copilot-turns-a-court-reporter-into-a-child-molester-9840612.html https://www.heise.de/en/news/Copilot-turns-a-court-reporter-...
- npunt 2y agoAbsolutely! Today's LLMs can sometimes(/often?) enormously suck and should not be relied upon for critical information. There's a long way to go to make them better, and I'm happy that a lot of people are working on that. Finding meaning in a sea of information is a highly imperfect enterprise regardless of the tech we use. My point though was that the core problem we should be trying to solve is overcoming the fundamental misalignment of incentives between publisher and reader, not whether we can put a better schema together that we hope people adopt intelligently & non-adversarially, because we know that won't happen in practice. I liked what the author wrote but they also didn't really consider this perspective and as such I think they haven't hit upon a fundamental understanding of the problem.
- hmottestad 2y agoMetadata in PDFs is also typically based on semantic web standards. https://www.meridiandiscovery.com/articles/pdf-forensic-analysis-xmp-metadata/ https://www.meridiandiscovery.com/articles/pdf-forensic-anal... Instead of using JSON-LD it uses RDF written as XML. Still uses the same concept of common vocabularies, but instead of schema.org it uses a collection of various vocabularies including Dublin Core.
- kkfx 2y agoEhm... The semantic web as an idea was/is a totally different thing: the idea is the old libraries of Babel/Bibliotheca Universalis by Conrad Gessner (~1545) [1] or the ability to "narrow"|"select"|"find" just "the small bit of information I want". Observing that a book it's excellent to develop and share a specific topic, it have some indexes to help directly find specific information but that's not enough, a library of books can't be traversed quick enough to find a very specific bit of information like when John Smith was born and where. The semantic web original idea was the interconnection of every bit of information in a format a machine can travel for a human, so the human can find any specific bit ever written with little to no effort without having to humanly scan pages of moderately related stuff. We never achieve such goal. Some have tried to be more on the machine side, like WikiData, some have pushed to the extreme the library science SGML idea of universal classification not ended to JSON but all are failures because they are not universal nor easy to "select and assemble specific bit of information" on human queries. LLMs are a, failed, tentative of achieve such result from another way, their hallucinations and slow formation of a model prove their substantial failure, they SEEMS to succeed for a distracted eye perceiving just the wow effect, but they practically fails. Aside the issue with ALL test done on the metadata side of the spectrum so far is simple: in theory we can all be good citizens and carefully label anything, even classify following Dublin Core at al any single page, in practice very few do so, all the rest do not care, or ignoring the classification at all or badly implemented it, and as a result is like an archive with some missing documents, you'll always have holes in information breaking the credibility/practical usefulness of the tool. Essentially that's why we keep using search engines every day, with classic keyword based matches and some extras around. Words are the common denominator for textual information and the larger slice of our information is textual. [1] https://en.wikipedia.org/wiki/Bibliotheca_universalis https://en.wikipedia.org/wiki/Bibliotheca_universalis
- DrScientist 2y agoThe problem I find with semantic search is first I have to read and understand somebody elses definitions before I can search within the confines of the ontology. The problem I have with ML guided search is the ML takes web average view of what I mean, which sometimes I need to understand and then try and work around if that's wrong. It can become impossible to find stuff off the beaten track. The nice thing about keyword and exact text searching with fast iteration is it's my mental model that is driving the results. However if it's an area I don't know much about there is a chicken and egg problem of knowing which words to use.
- gostsamo 2y agoSo much jumping to defend llms as the future. I'd like to point that llms hallucinate, could be injected, and often lack context which well structured metadata can provide. At least, I don't want for an llm to hollucinate the author's picture and bio based on hints in the article, thank you very much. I don't think that one is necessarily better than the other, but imagining that llms are a silver bullet when another trending story in the front pages is about prompt injection used against the slack ai bots sounds a bit over optimistic.
- IshKebab 2y agoSure but do hallucinations matter then much just for categorisation? Hardly the end of the world if they make up a published date occasionally. And prompt injection is irrelevant because the alternative we're considering is letting publishers directly choose the metadata.
- gostsamo 2y agoPrompt injection is highly relevant because you end up achieving the same as the publisher choosing the metadata, but on a much higher price for the user. Price which needs to be paid by each user separately instead of using one already generated. LLMs are much better when the user adapts the categories to their needs or crunches the text to pull only the info relevant to them. Communicating those categories and the cutoff criteria would be an issue in some contexts, but still better if communication is not the goal. Domain knowledge is also important, because nitch topics are not represented in the llm datasets and their abilities fail in such scenarios. As I said above, one is not necessarily better than the other and it depends on the use cases.
- IshKebab 2y ago> Prompt injection is highly relevant because you end up achieving the same as the publisher choosing the metadata, but on a much higher price for the user. How does price affect the relevance of prompt injection? That doesn't make sense. > nitch Niche. Pronounced neesh.
- tsimionescu 2y agoIf even the semantic web people are declaring victory based on a post title and a picture for better integration with Facebook, then it's clear that Semantic Web as it was envisioned is fully 100% dead and buried. The concept of OWL and the other standards was to annotate the content of pages, that's where the real values lie. Each paragraph the author wrote should have had some metadata about its topic. At the very least, the article metadata was supposed to have included information about the categories of information included in the article. Having a bit of info on the author, title (redundant, as HTML already has a tag for that), picture, and publication date is almost completely irrelevant for the kinds of things Web 3.0 was supposed to be.
- lynx23 2y agoI had pretty much the same reacon while reading the article. "BlogPosting" isn't particularily informative. The rest of the metadata looked like it could/should be put in <meta> tags, done. A very bad example if the intention was to demonstrate how cool and useful semweb is :-)
- oneeyedpigeon 2y agoThe schema.org data is much more rich than meta tags, though. Using the latter, an author is just a string of text containing who-knows-what. The former lets you specify a name, email address, and url. And that's just for the Person type—you can specify an Organization too.
- tsimionescu 2y agoThat's still just tangential Metadata. The point of a semantic web would be to annotate the semantic content of text. The vision was always that you can run a query like, say, "physics:particles: proton-mass", over the entire web, and it would retrieve parts of web pages that talk about the proton mass.
- rakoo 2y agoWhich was already possible with RDF. It is hard to not see JSON-LD as anything other than "RDF but in JSON because we don't like XML".
- trainyperson 2y agoAre there any tools that employ LLMs to fill out the Semantic Web data? I can see that being a high-impact use case: people don’t generally like manually filling out all the fields in a schema (it is indeed “a bother”), but an LLM could fill it out for you – and then you could tweak for correctness / editorializing. Voila, bother reduced! This would also address the two reasons why the author thinks AI is not suited to this task: 1. human stays in the loop by (ideally) checking the JSON-LD before publishing; so fewer hallucination errors 2. LLM compute is limited to one time per published content and it’s done by the publisher. The bots can continue to be low-GPU crawlers just as they are now, since they can traverse the neat and tidy JSON-LD. —————— The author makes a good case for The Semantic Web and I’ll be keeping it in mind for the next time I publish something, and in general this will add some nice color to how I think about the web.
- safety1st 2y agoBringing an LLM into the picture is just silly. There's zero need. The author (and much of HN?) seems to be unaware that it's not just thousands of websites using JSON-LD, it's millions. For example: install WordPress, install an SEO plugin like Yoast, and boom you're done. Basic JSON-LD will be generated expressing semantic information about all your blog posts, videos etc. It only takes a few lines of code to extend what shows up by default, and other CMSes support this took. SEOs know all about this topic because Google looks for JSON-LD in your document and it makes a significant difference to how your site is presented in search results as well as all those other fancy UI modules that show up on Google. Anyone who wants to understand how this is working massively, at scale, across millions of websites today, implemented consciously by thousands of businesses, should start here: https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data https://developers.google.com/search/docs/appearance/structu... https://search.google.com/test/rich-results https://search.google.com/test/rich-results Is this the "Semantic Web" that was dreamed of in yesteryear? Well it hasn't gone as far and as fast as the academics hoped, but does anything? The rudimentary semantic expression is already out there on the Web, deployed at scale today. Someone creative with market pull could easily expand on this e.g. maybe someday a competitor to Google or another Big Tech expands the set of semantic information a bit if it's relevant to their business scenarios. It's all happening, it's just happening in the way that commercial markets make things happen.
- nox101 2y agoNo ... because the incentives to lie in metadata are too high
- swiftcoder 2y agoAs much as I like the ideas behind the semantic web, JSON-LD feels like the least friendly of all semantic markup options (compared to something like, say, microformats)
- MrVandemar 2y agoMicroformats feel like they're ugly retrofitted kludges, where it would have been way more elegant if in among all the crazy helter-skelter competing development of HTML, someone thought to invent a <person> tag, maybe a <organisation> tag. That would have solved a few problems that <blink> certainly didn't.
- fabianholzer 2y agoThey certainly are retrofitted, but the existing semantic tags are largely abandoned for div soups that are beaten into shape and submission by lavish amounts of JS and a few sprinkles of CSS (and the latter often as CSS-in-JS). For microformats there is at least a little ecosystem already, and the vendor-driven committees don't need to be involved.
- swiftcoder 2y agoI mean, is anything actually stopping one from adding something like those tags today? Web components use custom tags all the time
- MrVandemar 2y agoNothing at all. I believe you don't even need to use web components. You can just throw in <my-person>Joe Bloggs</my-person> and that's valid HTML-whatever. But it's not a standard that is recognised, and so is no kind of metadata format.
- swiftcoder 2y agoNeither were microformats, up until a handful of aggregators started parsing them. We're still kind of stuck in the infancy of semantic markup it seems.
- deleted 2y ago[deleted]
- renegat0x0 2y agoI think that if you want your page to be well discoverable, to be well asvertised, positioned in search engines and social media you have to support standards. Like open graph protocol, or json ld. Be nice to bots. This is advertisment after all. Support standards even if Google does not. Other bots might not be as sofisticated. For me, yes, it is worth the bother
- jillesvangurp 2y agoDid json-ld get a lot of traction for link previews? I haven't really encountered it much. I actually implemented a simple link preview system a while ago. It uses opengraph and twitter cards meta data that is commonly added to web pages for SEO. That works pretty well. Ironically, I did use chat gpt for helping me implement this stuff. It did a pretty good job too. It suggested some libraries I could use and then added some logic to extract titles, descriptions, icons, images, etc. with some fallbacks between various fields people use for those things. It did not suggest me to add logic for json-ld.
- knallfrosch 2y agoHere I was, thinking the machines would make our lives easier. Now we have to make our websites Reader-Mode friendly, ARIA[1]-labelled, rendered server-side and now semantic web on top, just so that bots and non-visitors can crawl around? [1] This is also something the screen assist software should do, not the publisher.
- MrVandemar 2y agoARIA is something that really shouldn't have been necessary, but today it is absolutely crucial that content publishers make sure is right. Because the screen assist software can't do it. Why? Because a significant percentage of people working on web development think a webpage is composed as many <spans> and <divs> as you like, styled with CSS and the content is injected into it with JavaScript. These people don't know what an <img> tag is, let alone alt-text, or semantic heading hierarchy. And yet, those are exactly the things that Screen Reader software understands.
- conzept 2y agoI think the future holds a synthesis of LLM functions with semantic entities and logic from knowledge graphs (this is called "neuro-symbolic AI"), so each topic/object can have a clear context, upon which you can start prompting the AI for the preferred action/intention. Already implemented in part on my Conzept Encyclopedia project (using OpenAI): https://conze.pt/explore/%22Neuro-symbolic%20AI%22?l=en&ds=reference&t=string&batchsize=5&s=true# https://conze.pt/explore/%22Neuro-symbolic%20AI%22?l=en&ds=r... Something like this is much easier done using the semantic web (3D interactive occurence map for an organism): https://conze.pt/explore/Trogon?l=en&ds=reference&t=link&batchsize=5&i=Q191469&u=%2Fapp%2Fmap%2Findex.html%3Fl%3Den%26title%3DTrogon%26gbif%3D9356&s=true# https://conze.pt/explore/Trogon?l=en&ds=reference&t=link&bat... On Conzept one or more bookmarks you create, can be used in various LLM functions. One of the next steps is to integrate a local WebGPU-based frontend LLM, and see what 'free' prompting can unlock. JSON-LD is also created dynamically for each topic, based on Wikidata data, to set the page metadata.
- Vinnl 2y agoThe question is: does this bring any of the purported benefits of the Semantic Web? Does it suddenly allow "agents" to understand the meaning of your web pages, or are we just complying with a set of pre-defined schemas that predefined software (or more specifically, Google, in practice) understands and knows how to render. In other words, was all the SemWeb rigmarole actually necessary, or could the same results have been achieved using any of the mentioned simpler alternatives (microdata, OpenGraph tags, or even just JSON schemas)?
- sebstefan 2y agoIs that really what Discord, Whatsapp & co are using to display the embed widgets they have or is it just <meta> tags like I would expect...?
- johneth 2y agoThere are several methods they may use: - OpenGraph (by Facebook, probably used by Whatsapp) – https://ogp.me/ https://ogp.me/ - Schema.org markup (the main point of this blog) – https://schema.org/ https://schema.org/ - oEmbed (used to embed media in another page, e.g. YouTube videos on a WordPress blog) – https://oembed.com/ https://oembed.com/
- vouaobrasil 2y ago> The first is that large language models (LLMs) routinely get stuff wrong. If you want bots to get it right, provide the metadata to ensure that they do. Yet another reason NOT to use the semantic web. I don't want to help any LLMs.
- bigiain 2y agoI laughed at this bit: "Googlers, if you're reading this, JSON-LD could have the same level of public awareness as RSS if only you could release, and then shut down, some kind of app or service in this area. Please, for the good of the web: consider it."
- nujsii 2y ago[dead]
- druskacik 2y agoThere's a project [0] that parses Commoncrawl data for various schemas, it contains some interesting datasets. [0] http://webdatacommons.org/ http://webdatacommons.org/
- undefinedblog 2y agoThat’s a really useful link, thanks for sharing. We’re building a scrapping service and only parsing rely on native html tags and open graph metadata, based on this link we should definitely take a step forward to parse JSON-LD as well.
- peter_retief 2y agoNot totally sure if it is needed, nice to have? RSS feeds are great but seen less and less.
- openrisk 2y agoThe semantic web standards are sorely lacking (for decades now) a killer application. Not in a theoretical universe of decentralized philosopher-computer-scientists but in the dumbed down, swipe-the-next-30sec-video, adtech oligopolized digital landscape of walled gardens. Providing better search metadata is hardly that killer app. Not in 2024. The lack of adoption has, imho, two components. 1. bad luck: the Web got worse, a lot worse. There hasn't been a Wikipedia-like event for many decades. This was not pre-ordained. Bad stuff happens to societies when they don't pay attention. In a parallel universe where the good Web won, the semantic path would have been much more traveled and developed. 2. incompleteness of vision: if you dig to their nuclear core, semantic apps offer things like SPARQL queries and reasoners. Great, these functionalities are both unique and have definite utility but there is a reason (pun) that the excellent Protege project [1] is not the new spreadsheet. The calculus of cognitive cost versus tangible benefit to the average user is not favorable. One thing that is missing are abstractions that will help bridge that divide. Still, if we aspire to a better Web, the semantic web direction (if not current state) is our friend. The original visionaries of the semantic web where not out of their mind, they just did not account for the complex socio-economics of digital technology adoption. [1] https://protege.stanford.edu/ https://protege.stanford.edu/
- austin-cheney 2y agoA killer app is still not enough. People can’t get HTML right for basic accessibility, so something like the semantic web would be super science that people will out of their way to intentionally ignore any profit upon so long as they can raise their laziness and class-action lawsuit liability.
- PaulHoule 2y agoI see RDF as a basis to build on. If I think RDF is pretty good but needs a way to keep track of provenance or temporality or something I can probably build something augmented that does that. If it really works for my company and it is a competitive advantage I would keep quiet about it and I know of more than one company that's done exactly that. The standards process is so exhausting and you have to fight with so many systems programmers who never wrote an application that it's just suicide to go down that road. BTW, RSS is an RDF application that nobody knows about https://web.resource.org/rss/1.0/spec https://web.resource.org/rss/1.0/spec you can totally parse RSS feeds with a RDF-XML parser and do SPARQL and other things with them.
- 627467 2y ago> The Semantic Web is the old Web 3.0. Before "Web 3.0" meant crypto-whatnot, it meant "machine-readable websites". Using contemporary AI models aren't all websites machine-readable? - or potentially even more readable than semantic web unless an ai model actually does the semantic classification while reading it?
- CaptArmchair 2y agoI'm a bit surprised that the author doesn't mention key concepts such as linked data, RDF, federation and web querying. Or even the five stars of linked open data. [1] Sure, JSON-LD is part of it, but it's just a serialization format. The really neat part is when you start considering universal ontologies and linking to resources published on other domains. This is where your data becomes interoperable and reusable. Even better, through linking you can contextualize and enrich your data. Since linked data is all about creating graphs, creating a link in your data, or publishing data under a specific domain are acts that involves concepts like trust, authority, authenticity and so on. All those murky social concepts that define what we consider more or less objective truths. LLM's won't replace the semantic web, nor vice versa. They are complementary to each other. Linked data technologies allow humans to cooperate and evolve domain models with a salience and flexibility which wasn't previously possible behind the walls and moats of discrete digital servers or physical buildings. LLM's work because they are based on large sets of ground truths, but those sets are always limited which makes inferring new knowledge and asserting its truthiness independent from human intervention next to impossible. LLM's may help us to expand linked data graphs, and linked data graphs fashioned by humans may help improve LLM's. Creating a juxtaposition between both? Well, that's basically comparing apples against pears. They are two different things. [1] https://5stardata.info/en/ https://5stardata.info/en/
- ThinkBeat 2y agoI dont like the use of a Json "script" inside an HTML page. I understand the flexibility it grants but markup tags is what HTL is based on and the design would be more consistent by using HTML tags as we have had them for decades to also handle this extra meta data.
- M2Ys4U 2y agoJSON-LD isn't the only way one can embed these metadata (though I think most tooling prefers it now). For example, Microdata[0] is one in-line way to do it, and RDFa[1] is another. [0] https://en.wikipedia.org/wiki/Microdata_(HTML) https://en.wikipedia.org/wiki/Microdata_(HTML) [1] https://en.wikipedia.org/wiki/RDFa https://en.wikipedia.org/wiki/RDFa
- est 2y agoI've playing with RSS feeds recently, suddently it occured to me, XML can be transformed into anything with XSL, for static hosting personal blogs, I can save articles into the feeds directly, then serve frontend single-page application with some static XSLT+js. This is content-presentation separation at best. Is JSON-LD just reinventation of this?
- martin_a 2y agoThat is exactly the thought behind SGML/XML and its derivatives. XSL is kind of clumsy but very powerful and the most direct way to transform documents. JSON-LD to me looks more like trying to glue different documents together, its not about the transformation itself.
- rakoo 2y ago> This is content-presentation separation at best. The idea is the best, but arguably the implementation is lacking. > Is JSON-LD just reinventation of this? Yup. It's "RDF/XML but we don't like XML"
- ttepasse 2y agoBack in the optimistic 2000s there was the brief idea of GRDDL – using XSLT stylesheets and XPath selectors for extracting stuff from HTML, e.g. microformats, HTML meta, FOAF, etc, and then transforming it into RDF or other things: https://www.w3.org/TR/grddl/ https://www.w3.org/TR/grddl/
- mcswell 2y agoBut why? Isn't most of the information you can extract from those tags stuff that's pretty obvious, like title and author (the examples the linked page uses)? How do you extract really useful information using that methodology, supporting searches that answer queries like "110 volt socket accepting grounding plugs"? Of course search engines can (and do) get such info, but afaik it doesn't require or use XSLT beyond extracting the plain text.
- anonymous344 2y agoWell worth, for whom? as a blogger, these things are 99% for the companies making profit by scraping my content, maybe 1% of the users will need them. Or am I wrong?
- _heimdall 2y agoThis has been my hang up as well. Providing metadata seems extremely useful and powerful, but coming into web development in the mid 10s rather than mid 00s made it more clear that the metadata would largely just help a handful of massive corporations. I will still include JSON-LD when it make financial sense for a site. In practice that usually just means business metadata for search results and product data for any ecommerce pages.
- Lutger 2y agoEveryone is optimizing for their own local use-case. Even open-source. Standards get adopted sometimes, but only if they solve a specific problem. There is an additional cost to making or using ontologies, making them available and publishing open data on the semantic web. The cost is quite high, the returns aren't immediate, obvious or guaranteed at all. The vision of the semantic web is still valid. The incentives to get there are just not in place.
- BiteCode_dev 2y agoThe article talks about JSON-LD, but there is also shema.org and open graph. What which one should you use, and why? Should you use several? How does that impact the site?
- dangoodmanUT 2y agoJSON-LD uses schema.org schema
- giantrobot 2y agoBut very helpfully Google supports...mostly schema.org except when they don't when they feel like it.
- codelion 2y agoI started this thread on the w3c list almost 20 years ago - https://lists.w3.org/Archives/Public/semantic-web/2005Dec/0066.html https://lists.w3.org/Archives/Public/semantic-web/2005Dec/00... Unfortunately, it is unlikely we will ever get something like a Semantic web. It seemed like a good idea in the beginning of 2000s but now there is honestly no need for it as it is quite cheap and easy to attach meaning to text due to the progress in LLMs and NLP.
- mcswell 2y agoExactly. Afaik, there are certain corners of the Web that benefit from some kind of markup. I think real estate is one, where you can generate searches of the MLS on sites like Redfin or Zillow (or any realtor's site, really) such that you can set parameters: between 1000 and 1500 square feet (or meters in Europe), with a garage and no basement. That's very helpful (although I don't know whether that searching is done over indexed web pages, or on the MLS itself). But most of the Web, afaict, have nothing like that---and don't need it, because NLP can distinguish different senses of 'bank' (financial vs. river), etc.
- kvgr 2y agoI was doing bachelor thesis 10 years ago on some semantic file conversions, we had a lot of projects at school. And looks like there is not much progress for end user…
- grumbel 2y agoI don't see how one can have any hope in a Semantic Web ever succeeding when we haven't even managed to get HTML tags for extremely common Internet things: pricetags, comments, units, avatars, usernames, advertisement and so on. Even things like pagination are generally just a bunch of links, not any kind of semantic thing holding multiple documents together (<link rel> exists, but I haven't seen browsers doing anything with it). Take your average website and look at all the <div>s and <span>s and there is a whole lot more low hanging fruit one could turn semantic, but there seems little interest in even trying to.
- rakoo 2y agoI don't think we necessarily need new tags: they narrow down the list of possible into an immutable set and require changing the structure of your already existing content. What exists instead are microformats (http://microformats.org/wiki/microformats2 http://microformats.org/wiki/microformats2), a bunch of classes you sprinkle in your current HTML to "augment" it.
- _heimdall 2y agoI include microformats on blog sites, but at scale the challenge with microformats is that most existing tooling doesn't consider class names at all for semantics. Browsers, for example, completely ignore classes when building the accessibility tree for a web page. Only the HTML structure and a handful of CSS properties have an impact on accessibility. Class names were always meant as an ease of use feature for styling, overloading them with semantic meaning could break a number of sites built over the last few decades.
- ttepasse 2y agoThere is also RDFa and even more obscure Microdata to augment HTML elements. Google’s schema.org vocabulary originally used these before switching to JSON-LD. The trick, as always, is to get people to use it.
- dsmurrell 2y ago"Googlers, if you're reading this, JSON-LD could have the same level of public awareness as RSS if only you could release, and then shut down, some kind of app or service in this area. Please, for the good of the web: consider it." - lol
- dgellow 2y agoCompanies use open-graph because it gives them something in return (nice integration in other products when linking to your site). That’s nice and all but outside of this niche use case there is no incentives for a semantic web from the point of view of publishers. You just make it simpler to crawl your website (something you cannot really monetize) instead of offering a strict API you can monetize to access structured data.
- deleted 2y ago[deleted]
- 1f60c 2y agoThis has been invented a number of times. Facebook's version is called Open Graph. https://ogp.me/ https://ogp.me/
- ttepasse 2y agoBack then Facebook said their Open Graph Protocol was only an application of RDFa – and syntax wise it seemed so.
- patagnome 2y agoworth the bother. "preview" on the capitalocenic web without any mention of the Link Relation Types does not a semantic web adoption make. no mention of the economic analysis and impact of monopoly, no intersectional analysis with #a11y. if the "preview" link relation type is worth mentioning it's worth substantiating the claims about adoption. when did the big players adopt? why? what of the rest of the types and their relation to would-be "a.i." claims? how would we write html differently and what capabilities would we expose more readily to driving by links, like carousels only if written with a11y in mind? how would our world-wild web look different if we wrote html like we know it? than only give big players a pass when we view source?
- hoosieree 2y ago> If Web 3.0 is already here, where is it, then? Mostly, it's hidden in the markup. I feel like this is so obvious to point out that I must be missing something, but the whole article goes to heroic lengths to avoid... HTML. Is it because HTML is difficult and scary? Why invent a custom JSON format and a custom JSON-to-HTML compiler toolchain than just write HTML? The semantics aren't hidden in the markup. The semantics are the markup.
- patagnome 2y ago[dead]
- wepple 2y agoI think that’s what we’re doing today, and it’s a phenomenal mess. The typical HTML page these days is horrifically bloated, and whilst it’s machine parsable, it’s often complicated to actually understand what’s what. It’s random nested divs and unified everything. All the way down. But I do wonder if adding context to existing HTML might be better than a whole other JSON blob that’ll get out of sync fast.
- hoosieree 2y agoI'm just not convinced that swapping out "<ol></ol>" for "[]" actually addresses any of the problems.
- deleted 2y ago[deleted]
- Lutger 2y agoI must have missed your point, isn't the answer obviously that HTML is very, very limited and intended as a way to markup text? Semantic data is a way to go further and make machine-readable what actually is inside that text: recipes, places, people, posts, animals, etc, etc and all their various attributes and how they relate to each other. Basically, what you are saying is already rdf/xml, except that devs don't like xml so json-ld came along as a man-machine-friendlier way to do rdf/xml. There are also various microdata formats that allow you to annotate html in a way the machines can parse it as rdf. But that can be limited in some cases if you want to convery more metadata.
- matheusmoreira 2y agoI wish there was a better alternative to JSON-LD. I want to avoid duplication by reusing the data that's already in the page by marking them up with appropriate tags and properties. Stuff like RDF exists but is extremely complex and verbose.
- ttepasse 2y agoOriginally you could use the schema.org vocabulary with RDFa or Microdata which embed the structured data right at the element. But than can be brittle: Markup structures change, get copy-and-pasted and editing attributes is not really great in CMS. I may not like it aesthetically but embedded JSON-LD makes some sense. See also this comment above: https://news.ycombinator.com/item?id=41309555 https://news.ycombinator.com/item?id=41309555
- makkes 2y agoSemantic Web technology (RDF, RDFS, OWL, SHACL) is widely used in the European electricity industry to exchange grid models: https://www.entsoe.eu/data/cim/cim-for-grid-models-exchange/ https://www.entsoe.eu/data/cim/cim-for-grid-models-exchange/
- etimberg 2y agoI have experience using this back when I worked for a startup that did distribution grid optimization. The specs are unfortunately useless in practice because while the terminology is standardized the actual use of each object and how to relate them is not. Thus, every tool makes CIM documents slightly differently and there are no guarantees that a document created in one tool will be usable in another
- makkes 2y agoThat's why ENTSO-E has just completed a software vendor interoperability workshop. :-) And import/export/validation worked just fine for all participants.
- ubertaco 2y agoWell, the immediate initial test failed for me: I thought, "why not apply this on one of my own sites, where I have a sort of journal of poetry I've written?"...and there's no category for "Poem", and the request to add Poem as a type [1] is at least 9 years old, links to an even older issue in an unreadable issue tracker without any resolution (and seemingly without much effort to resolve it), and then dies off without having accomplished anything. [1] https://github.com/schemaorg/suggestions-questions-brainstorming/issues/194 https://github.com/schemaorg/suggestions-questions-brainstor...
- tossandthrow 2y agoHaving worked in this field for a bit, this uncovers an even more fundamental flaw: The idea that we can have a single static ontology.
- lambdaba 2y agoWhat kind of work do you do?
- tossandthrow 2y agovarious. Notable I, some years ago, had a project that considered automatic consolidation of ontologies based on meta-ontologies and heuristics. The idea being that everyone have their own ontology for the data they release and the system would make a consolidated ontology that could be used to automatic integration of data from different datasources. regardless, that project did not get traction, so now it sits.
- lambdaba 2y agoThanks! Is it possible for you to share more details about the domain and problem?
- codewithcheese 2y agoDomain driven design is well aware that is not feasible to have a single schema for everything, they use bounded contexts. Is there something similar for the semantic web?
- renonce 2y agoLooks like a perfect use case for LLM: generate that JSON-LD metadata from HTML via LLM, either by the website owner or by the crawler. If crawlers, website owners doesn’t need to do anything to enter Semantic Web and crawlers specify their own metadata format they want to extract. This promises an appealing future of Web 3.0, not by crypto, defined not by metadata but by LLMs.
- eadmund 2y agoEmbedding data as JSON as program text inside a <script> tag inside a tagged data format just seems like such a terrible hack. Among other things, it stutters: it repeats information already in the document. The microdata approach seems much less insane. I don’t know if it is recognised nearly as often. TFA mentions it at the end: ‘There is also “microdata.” It’s very simple but I think quite hard to parse out.’ I disagree: it’s no harder to parse than HTML, and one already must parse HTML in order to correctly extract JSON-LD from a script tag (yes, one can incorrectly parse HTML, and it will work most of the time).
- ryukoposting 2y agoPardon my naivetée, but what exactly is JSON-LD doing that the HTML meta tags don't do already? My blog doesn't implement JSON-LD but if you link to my blog on popular social media sites, you still get a fancy link.
- ttepasse 2y agoJSON-LD / RDFa and such can use the full type hierarchy of schema.org (and other languages) and can build a tree or even a graph of data. Meta elements are limited to property/value pairs.
- ryukoposting 2y agoInteresting. How is this implemented by the server? Just periodically refresh a cached value for each page, or are you updating the tree in real-time for all pages, and having them traverse it on each request? Either way it sounds awfully expensive for data that probably isn't used by the client most of the time. Do you have to explicitly ask for it? Is there some ad-hoc way to tell the server "hey I don't need the JSON-LD data?"
- _heimdall 2y agoMonetization is the elephant in the room in my opinion. IMDB could easily be a service entirely dedicated to hosting movie metadata as RDF or JSON-LD. They need to fund it though, and the go to seems to be advertising and API access. Advertising means needing human readable UI, not metadata, and if they put data behind an API its a tough sell to use a standardized and potentially limiting format.
- deleted 2y ago[deleted]
- jrochkind1 2y ago> Semantic Web information on websites is a bit of a "living document". You tend publish something, then have a look to see what people have parsed (or failed to parse) it and then you try to improve it a bit. Hm.
- physicsguy 2y agoSemantic web suffers from organisational capture. If there's a big org they get to define the standard at the expense over everyone else use cases.
- gdegani 2y agoThere is a lot of value on Enterprise Knowledge Graphs, applying the semantic web standards into the "self-contained" world of enterprise data, there are many large enterprises doing it, and there is an interesting video from UBS on how they consider it a competitive advantage
- bawolff 2y agoIf this counts as the "semantic web", then <meta name="description"... should to. In which case we have all been on it since the mid 90s.
- PaulHoule 2y agoIt's real RDF. You can process this with RDF tools. Certainly do SPARQL queries. Probably add a schema and have valid OWL DL and do OWL inference if the data is squeaky clean. Certainly use SPIN or Jena rules. It leans too hard on text and doesn't have enough concepts defined as resources but what do you expect, Python didn't have a good package manager for decades because 2 + 2 = 3.9 with good vibes beats 2 + 2 = 4 with honest work and rigor for too many people. The big trouble I have with RDF tooling is inadequate handling of ordered lists. Funny enough 90% of the time or so when you have a list you don't care about the order of the items and frequently people use a list for things that should have set semantics. On the other hand, you have to get the names of the authors of a paper in the right order or they'll get mad. There's a reasonable way to turn native JSON lists into RDF lists https://www.w3.org/TR/json-ld11/#lists https://www.w3.org/TR/json-ld11/#lists although unfortunately this uses the slow LISP lists with O(N) item access and not the fast RDF Collections that have O(1) access. (What do you expect from M.I.T.?) The trouble is that SPARQL doesn't support the list operations that are widespread in document-based query languages like https://www.couchbase.com/products/n1ql/ https://www.couchbase.com/products/n1ql/ https://docs.arangodb.com/3.11/aql/ https://docs.arangodb.com/3.11/aql/ or even Postgresql. There is a SPARQL 1.2 which has some nice additions like https://www.w3.org/TR/sparql12-query/#func-triple https://www.w3.org/TR/sparql12-query/#func-triple but the community badly needs a SPARQL 2 that catches up to today's query languages but the semantic web community has been so burned by pathological standards processes that anyone who can think rigorously or code their way out of a paper bag won't go near it. A substantial advantage of RDF is that properties live in namespaces so if you want to add a new property you can do it and never stomp on anybody else's property. Tools that don't know about those properties can just ignore them, but SPARQL, RDFS and all that ought to "just work" though OWL takes some luck. That's got a downside too which is that adding namespaces to a system seems to reduce adoption by 80% in many cases because too many people think it's useless and too hard to understand.
- ChrisMarshallNY 2y agoI suspect that AI training data standards will make this much more prevalent. Just today, I am working on an experimental training/consuming app pair. The training part will leverage JSON data from a backend I designed.
- taeric 2y agoIt is hilarious to see namespaces trying to creep into json. I do wonder how any of this is better than using the meta tags of the html, though? Especially for such use cases as the preview. Seems the only thing that isn't really there for the preview is the image? (Well, title would come from the title tag, but still...)
- esbranson 2y agoArguing against standard vocabularies (part of the Semantic Web) is like arguing against standard libraries. "Cool story bro." But it is true, if you can't make sense of your data, then the Semantic Web probably isn't for you. (It's the least of your problems.)
- rchaud 2y ago> Googlers, if you're reading this, JSON-LD could have the same level of public awareness as RSS if only you could release, and then shut down, some kind of app or service in this area. Please, for the good of the web: consider it. Google has been pushing JSON-LD to webmasters for better SEO for at least 5 years, if not more: https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data https://developers.google.com/search/docs/appearance/structu... There really isn't a need to do it as most of the relevant page metadata is already captured as part of the Open Graph protocol[0] that Twitter and Facebook popularized 10+ years ago as webmasters were attempting to set up rich link previews for URLs posted to those networks. Markup like this: <meta property="og:type" content="video.movie" /> is common on most sites now, so what benefit is there for doing additional work to generate JSON-LD with the same data? [0]https://ogp.me/ https://ogp.me/
- weego 2y ago'it makes social sharing look a bit nicer' being the only benefit that can scraped from the barrel as a benefit undermines the entire premise. It's not widely adopted, it's used as an attempted growth hack in a few locations that may or may not be of use (with value being relative to how US centric your and your audiences Internet use is)
- pablomendes 2y agoThat statement is both kind of true and, well, revisionist. Originally there was a strong focus on logics, clean comprehensive modeling of the world through large complicated ontologies, and the adoption of super impractical representation languages, etc. It wasn't until rebellious sub-communities went rogue and pushed for pragmatic simplifications that things got any widespread impact at all. So here's to the crazy ones, I guess.
- jgalt212 2y agoMy fear around JSON-LD is too much of our content will end up on a SERP, and we'll attract less traffic.
- NoelDeMartin 2y agoI found about The Semantic Web about 5 years ago, and since then I've been pulling the thread and I realized it's a lot more than just "machine readable metadata". In particular, it's also about data ownership and control. You can see what I mean learning about the Solid Protocol, I gave a talk about it a couple of years ago: https://m.youtube.com/watch?v=kPzhykRVDuI https://m.youtube.com/watch?v=kPzhykRVDuI