7 ms·

Developing for the Semantic Web

86 points by headalgorithm 6y ago

redleggedfrog 6y ago
The problem isn't the technology, it's the people. People don't want to share their data. You have to get people to actually accept that their competitors might be able to easily scrape data out of their website. They don't want that, and they'll pay money to keep it from happening.
- egfx 6y ago
  > People don't want to share their data Where did you get that notion? Just take a look at the internet this last decade. What people crave now is a semantic web that is connected to a community of computers able to very quickly verify and distinguish what’s real and what’s disconnected from the truth.
  - Uehreka 6y ago
    I think they meant that companies don’t want to share their data. If you look at things like the Twitter API and how it’s been slowly locked down over the years, it’s clear that this is the case.
- astrobe_ 6y ago
  This, and the economic model of the web. You can't insert ads in APIs.
- Vinnl 6y ago
  That really depends on the type of data, I think. For example, I'm quite sure restaurant owners love it when their restaurant address is shared far and wide, so people can find the restaurant. When the data is your USP, sure. But there's a ton of data that's not.
bobosha 6y ago
Former semantic web researcher here. The problem with semantic web is that it's way too complicated for the average user (unlike html), with no clear payoff/ I see the need for Machine learning to automate several of these tasks. It is a laudable effort, but I'm not sure if it'd be anything more than some marginal improvements in data interop on a web-scale.
- echelon 6y ago
  You're right. It's hard to publish data with semantic markup and understand the schemas (or even the benefit of using them). Nobody is writing FOAF or any other RDF/OWL by hand. In order to bootstrap the process, authorship tools need to produce semantic markup automatically on the behalf of users. Unfortunately the content silos do not want to do that. They're happy to keep the data to themselves and run Snowflake / big data queries on them all day long. They just don't want you doing the same. It's no benefit to them if your friend or interest graph escapes their hold. This is a difficult chicken and egg problem. A good angle of attack might be for the distributed web folks to take an interest. ActivityPub, Mastodon, and Matrix/Riot/(whatever they rebranded to) would be beneficiaries of schema ingestion. The more they can mark up the data they produce, the more useful it becomes. A tooling ecosystem can grow around semantic data. Technically open standard APIs in a distributed web are already semantic, it may just be that their schemas don't map well to queryable triple stores. But that's just a transformation away. RDF is just a format for expressing your ontology in an open and compatible way others can reference and build upon.
  - Karrot_Kream 6y ago
    The ActivityPub standard is built atop JSON-LD, which is a Semantic Web standard. I don't think anyone has really tried to parse the data exposed by the Fediverse for much though.
- dumb1224 6y ago
  I was working in RDF/ontology space. IMO the additional domain knowledge required to model data correctly and publish them in a way an average user can comprehend and navigate properly is disproportional to the benefit of such data model. By the time everything is properly modelled the knowledge itself might be a bit obsolete (speaking from bio-tech related fields).
rshnotsecure 6y ago
The semantic web was at one time "the next big thing". It was one of those technologies that topped the Gartner Hype Cycle sometime during the first Web 2.0 years. Interesting to see it back again. It had many problems but was always an interesting concept.
simongray 6y ago
I've had to dive into this recently as part of my work (I'm in academia). While the original vision seems to have failed from a lack of incentives, the idea behind RDF as way to model connected data and its many serialisation formats is actually quite helpful. It seems to have found a niche in academia when it comes to sharing and integrating large graph datasets. I just consider RDF and most of the other semantic web standards as a framework for modelling and serialising graph data. It's pretty good at that.
- tasogare 6y ago
  No, it’s not even good for that. The fact you need yo reify properties (relationships) to annotate them is a big failure, which is not present in more general graph models.
  - Karrot_Kream 6y ago
    I think this is a matter of opinion. I'm personally a fan of reified properties, but this is the source of differences between property graph databases and RDF triple stores.
  - simongray 6y ago
    Works fine for my use cases (e.g. representing multiple WordNets as labeled property graphs in Neo4j) and I quite like the simplicity of data modelling based on triplets of data literals. What do you suggest as an alternative to RDF?
  - emmanueloga_ 6y ago
    Not true. See "reification is a red herring" [1]. Also, there's now a more natural way to add properties to relationships if really needed, it is call RDF* [2], and it is already implemented of a lot of triplestores, including RDF4J, Jena, StarDog and more. 1: http://www.snee.com/bobdc.blog/2018/04/reification-is-a-red-herring.html http://www.snee.com/bobdc.blog/2018/04/reification-is-a-red-... 2: https://blog.liu.se/olafhartig/2019/01/10/position-statement-rdf-star-and-sparql-star/ https://blog.liu.se/olafhartig/2019/01/10/position-statement...
hmottestad 6y ago
I'm a committer on Eclipse RDF4J and currently working on our implementation of SHACL. I've worked with semantic web now for 7-8 years I think. Seen quite a few projects struggle or fail mainly due to performance issues. That is why I'm doing my best with optimising RDF4J.
- emmanueloga_ 6y ago
  Hey there! I actually saw this on my own experiments! I was experimenting with validating some data through the SHACL version of schema.org [1]. To be fair my findings are not much more than "anecdata" for now, since a proper benchmark should use a bigger dataset, allow the JVM to warm up, etc... But from a quick comparison, the RDF4j implementation was on my tests from 2 to 3 times faster than the "reference" one [2] when validating some small in-memory data. 1: http://datashapes.org/schema http://datashapes.org/schema 2: https://github.com/topquadrant/shacl https://github.com/topquadrant/shacl
  - hmottestad 6y ago
    You will notice a much bigger difference once you have a larger database and want to make changes to your data. RDF4J is able to analyse your changes and validate only the smallest amount of data to make sure that everything is still valid.
    - emmanueloga_ 6y ago
      Great to know!
enumjorge 6y ago
I appreciate the spirit of the semantic web, but I've never really been clear on how it would accomplish its stated goals. Let's imagine the web has lots of annotated, machine-readable data. You still need a client to explore that data, right? The browser is not going to know how to make sense of the relationships between your data points. How is the end user going to consume the information, a web app? As the dominance of companies like Facebook, Google and Twitter have shown, the average user doesn't care about de-centralization. They care about a good user experience. Let's take the recipe scenario presented in the article. What's to stop Google from ingesting all of this semantic data, and then using their army of software devs to integrate the results into Google's search results? Google has invested in natural language processing that can allow their web crawlers to make semantic sense of written text. I'm sure they would love it if a swath of volunteers did this for them instead. Sure, some small, indie shop could use the data too. But Google can integrate with their translation engine, their home speakers and Android. They'll make the experience nice enough that your aunt looking for a scone recipe won't want to leave their ecosystem, even if a more free, de-centralized alternative is available.
- jcelerier 6y ago
  > What's to stop Google from ingesting all of this semantic data, and then using their army of software devs to integrate the results into Google's search results? that's pretty much exactly what they were doing with freebase (https://developers.google.com/freebase/guide/basic_concepts https://developers.google.com/freebase/guide/basic_concepts)
- elliottinvent 6y ago
  Yes, I think the semantic web is suited to big companies with large teams and huge processing capabilities – they hoover the data up and add it to their own graph. As you've pointed out, the trend is for Google to pull it out of the website and display it on the results page, cutting the content creator out entirely. Building a tool or service on top of the semantic web is out of reach for most developers because of inconsistencies and complexities, unless they use the tools provided by the giants – and that goes directly against the principles of the semantic web. The exception here is niche use cases and academia working with huge datasets, where the semantic web seems to be incredibly useful.
emmanueloga_ 6y ago
The problem with the "semantic web" projects is that they are always proposing "pie in the sky" projects that require a big leap of faith to get invested on. I see solidproject.org as an example of this. I've been exploring semantic technologies from the cold and incredible practical angle of "my own needs" :-). The RDF model offers a lot even when keeping data for myself: I can use RDF to infer new things from existing data and visualize relationships. An RDF triplestore can be used to store pretty much anything, and works well for CRUD. RDF is about graphs, so a triple store can be used to model documents too, since trees are a subset of all graphs. RDF4J has really nice documentation and supports a lot of modern features. The getting started guide is great [1], and there's also a super friendly gitter room to ask questions [2]. 1: https://rdf4j.org/documentation/tutorials/getting-started/ https://rdf4j.org/documentation/tutorials/getting-started/ 2: https://gitter.im/eclipse/rdf4j https://gitter.im/eclipse/rdf4j
- darkhorse13 6y ago
  >The problem with the "semantic web" projects is that they are always proposing "pie in the sky" projects I have read this sentiment countless of times, but I don't quite understand it. Maybe because I am not familiar with the semantic web. Can you please suggest some reading list for a person who is unfamiliar with semantic web to better understand these issues?
  - emmanueloga_ 6y ago
    The first thing you should read is the RDF primer [1]. It explains the basic data model that the semantic web uses as building block for anything else. The gist of it is to provide a way to model data as a graph, and make it almost trivial to exchange data between parties. A simple example is, if you and I want to share, say, user data, and you have a "people" table and I have "Users", and my fields are "first_name" "last_name" and yours "given" and "proper", and so on, how do we match the same user across tables? RDF provides a solution for that. The "pie in the sky" part is that, from this basic building block, a whole host of other technologies can be built, some of them dealing with artificial intelligence, machines being able to _infer_ more information from what you have already said [2]. Inferences can work best when there's more data, that's were the ability to mix your data with third party databases comes handy. Solid adds even more stuff to the table. In their words: derived from "social linked data", is a proposed set of conventions and tools for building decentralized social applications based on Linked Data principles. I think "decentralized" is the keyword here. I haven't even started looking at it :-) 1: https://www.w3.org/TR/rdf11-primer/ https://www.w3.org/TR/rdf11-primer/ 2: https://www.w3.org/standards/semanticweb/inference https://www.w3.org/standards/semanticweb/inference
elliottinvent 6y ago
The semantic web makes sense for large datasets provided by tech-enabled companies, but the overwhelming majority of entities represented on the web don't have large datasets or tech teams. They have a small amount of standard data they want to get out to customers – contact data, company data (tax registration or whatever), GPS locations for stores, etc. They don't need 99.9% of the complexities of Schema.org. They only consider a website from their viewpoint – a human viewing the website through browser. The HTML (including microdata, JSON-LD, etc) is terrifying for most of them. So at best, their web designer builds in some semantic web standards but the data can be inaccurate or become out of data quickly because the website owner doesn't even know it's there. Even big companies have adopted semantic web standards in such a confusing and inconsistent way that it's difficult to rely on the data being available in an expected format, so that a developer can build something on top of it. Unless semantic web standards can be adopted by small businesses in a simple way (point, click and type) I don't think they will ever be widely adopted. Finally, I don't think it's efficient to hide the 1kb of machine usable data inside 99kb of HTML designed for browser rendering. Shameless plug: A small team and I have built an alternative to the semantic web, to store machine readable data in DNS. It can be populated by simple user friendly online forms. Explainer and summary: https://www.num.uk https://www.num.uk Best way to play around with it: https://app.numserver.com/tools/editor/add https://app.numserver.com/tools/editor/add
Vinnl 6y ago
Interesting that the article doesn't touch on Tim Berners-Lee's current focus: https://solidproject.org https://solidproject.org (Disclosure: I work for his startup, Inrupt, which he founded to give that project a boost. Opinions here are my own though.) Although it's primarily known for allowing apps to store their users' data in a place under their users' control, for better or worse, it's built on many SemWeb technologies. And even though I'm still somewhat disillusioned about the Semantic Web (like Aaron Schwarz, I used to be enthusiastic about it way back when), but there certainly is a chance it will be interesting again when your personal (potentially private) data can be part of the Semantic Web as well. For example, if they'd have written their reviews in a Solid-based website, linking them to MusicBrainz would be a relatively small step.
- laurencerowe 6y ago
  The universal login aspect of Solid sounds rather like Mozilla’s Persona. What about Solid makes it more likely to succeed against the incumbents than Persona?
  - Vinnl 6y ago
    There are several things that make it different, making the reason Persona failed not apply (though obviously it might still fail for different reasons): - There are several organisations behind Solid that whose raison d'etre is Solid. I'd argue that Mozilla possibly pulled the plug for Persona too early, specifically, without first adding support for it in Firefox, thereby creating massive incentive for website owners to support it. But then, it wasn't their core focus, and there was no return on investment for them, so I can see why they cancelled it. - Ironically, a wider focus. The incentive for website owners to support Persona would have been that it would be easier for potential users to join - although they already got most of that benefit by allowing sign-in with Facebook, and most people did not actually use Persona. Solid, on the other hand, has several incentives to support it. It can give organisations access to data that the user is actively curating, rather than having to provide it to different organisations individually, with it getting outdated the moment they shared it. Additionally, given how, for many organisations, data is increasingly (post-GDPR and all that) a liability and not their primary competency, outsourcing management of user data to the user themselves makes sense. And finally, there will hopefully be more people with a Solid Pod, especially in specific markets - for example, IIRC the government of Flanders was looking at giving every citizen a Pod. - It can lean on the marketing benefits of it being TimBL's main focus, which may or may not be stronger than being one of Mozilla's projects. - Depending on your definition of success, it can be as successful as Mastodon/the Fediverse is. As opposed to Persona, people can and do write apps specifically to support Solid, because it's more than just a login. Just off the top of my head, and again, personal views. But hopefully that gives some insight.
kangnkodos 6y ago
The problem is the economic incentives. The people who have to do extra work are the people providing the data. The people who get the benefit are the people consuming the data.
jandrewrogers 6y ago
The semantic web currently scales poorly in two different dimensions. As long as this remains the case, it will see little adoption. First, the software technology scales very poorly unless the size of your data model is trivial. The semantic web is a type of graph search problem with all of the hard algorithm and data structure challenges that entails. Most interesting semantic web applications require data models operating at scales where existing graph platforms have pathological performance characteristics. The gap between capability and requirements is several orders of magnitude. You can't throw hardware at it; when I was first hired to do CS research on scaling the semantic web, I was working on literal supercomputers. Second, the human element of the semantic web doesn't scale. Semantics are contextual and subjective. As the number of people contributing to the data model increases, the consistency and quality of the data model decreases. This becomes an extremely expensive global coordination problem at scale that may be unsolvable. It is broadly recognized that you need a single, global arbiter of semantics in any scalable system just to maintain semantic consistency, which basically means strong AI. If we have the technology to solve these two problems, I suspect the Semantic Web will no longer be interesting.
- pfraze 6y ago
  I think you said it pretty well, but I'll re-iterate some of your second point. The human element defies any solution other than social coordination such as a specs registry or org. At some point the software needs to understand each other and that requires people to work together. A technical solution around schemas can assist the coordination work, but it can't replace it. EDIT to add: Since we can't remove human coordination, I question the value of arbitrary schema composition when it incurs the huge DX cost of using a URL to represent every attribute.
  - infinite8s 6y ago
    Alan Kay had a comment on here from a few years ago on how this could scale: https://news.ycombinator.com/item?id=15320789 https://news.ycombinator.com/item?id=15320789
  - bigmanwalter 6y ago
ilaksh 6y ago
I think that AI is getting close to the point that it is going to be a significant factor in the story of the semantic web from here on. What comes to mind is people training AI to parse web pages and mark them up to indicate different types of data inside of them. Such as the recipe example. So someone who authors a WordPress plugin for recipes could incorporate it. Or, and this is more scalable but obviously very difficult, it may be possible to create plugins for web frameworks that automatically inject the semantic markup using AI. Maybe. Then there could be another AI program that converts natural language queries into semantic knowledge queries. But I think that AI could play a big part in rolling it out. I mean, if you had the AI systems without the semantic web, for open systems you would still want a way for them to share data unambiguously.
gumby 6y ago
The semantic web can only work via automation -- pretty much complete automation. Semantic information depends on a "stance" (those humanities classes do pay off) since the same information can mean different things depending on context and perspective. In addition spam will continue to be a significant problem.
deleted 6y ago
[deleted]
mimixco 6y ago
We've been working on an alternative to SemWeb based on a new language for marking up facts inside of text. It doesn't rely on schema but allows the user to define fact relationships as they happen in writing. More info here: https://news.ycombinator.com/item?id=24662705 https://news.ycombinator.com/item?id=24662705
mitchtbaum 6y ago
I have it working, https://github.com/speakeasy-engine https://github.com/speakeasy-engine We can have semantic social networking now, https://github.com/patchworkuniversity/radiojade https://github.com/patchworkuniversity/radiojade
- mitchtbaum 6y ago
  This was our initial Alice and Bob: https://github.com/foundpatternscellar/ping-pong https://github.com/foundpatternscellar/ping-pong
Animats 6y ago
The trouble with the "semantic web" is that the cost is paid by those creating content, and the benefit is obtained by those running web scrapers.

86 points by headalgorithm 6y ago

The problem isn't the technology, it's the people. People don't want to share their data. You have to get people to actually accept that their competitors might be able to easily scrape data out of their website. They don't want that, and they'll pay money to keep it from happening.

egfx 6y ago
> People don't want to share their data Where did you get that notion? Just take a look at the internet this last decade. What people crave now is a semantic web that is connected to a community of computers able to very quickly verify and distinguish what’s real and what’s disconnected from the truth.
- Uehreka 6y ago
  I think they meant that companies don’t want to share their data. If you look at things like the Twitter API and how it’s been slowly locked down over the years, it’s clear that this is the case.
astrobe_ 6y ago
This, and the economic model of the web. You can't insert ads in APIs.
Vinnl 6y ago
That really depends on the type of data, I think. For example, I'm quite sure restaurant owners love it when their restaurant address is shared far and wide, so people can find the restaurant. When the data is your USP, sure. But there's a ton of data that's not.

bobosha 6y ago

Former semantic web researcher here. The problem with semantic web is that it's way too complicated for the average user (unlike html), with no clear payoff/ I see the need for Machine learning to automate several of these tasks. It is a laudable effort, but I'm not sure if it'd be anything more than some marginal improvements in data interop on a web-scale.

echelon 6y ago
You're right. It's hard to publish data with semantic markup and understand the schemas (or even the benefit of using them). Nobody is writing FOAF or any other RDF/OWL by hand. In order to bootstrap the process, authorship tools need to produce semantic markup automatically on the behalf of users. Unfortunately the content silos do not want to do that. They're happy to keep the data to themselves and run Snowflake / big data queries on them all day long. They just don't want you doing the same. It's no benefit to them if your friend or interest graph escapes their hold. This is a difficult chicken and egg problem. A good angle of attack might be for the distributed web folks to take an interest. ActivityPub, Mastodon, and Matrix/Riot/(whatever they rebranded to) would be beneficiaries of schema ingestion. The more they can mark up the data they produce, the more useful it becomes. A tooling ecosystem can grow around semantic data. Technically open standard APIs in a distributed web are already semantic, it may just be that their schemas don't map well to queryable triple stores. But that's just a transformation away. RDF is just a format for expressing your ontology in an open and compatible way others can reference and build upon.
- Karrot_Kream 6y ago
  The ActivityPub standard is built atop JSON-LD, which is a Semantic Web standard. I don't think anyone has really tried to parse the data exposed by the Fediverse for much though.
dumb1224 6y ago
I was working in RDF/ontology space. IMO the additional domain knowledge required to model data correctly and publish them in a way an average user can comprehend and navigate properly is disproportional to the benefit of such data model. By the time everything is properly modelled the knowledge itself might be a bit obsolete (speaking from bio-tech related fields).

rshnotsecure 6y ago

The semantic web was at one time "the next big thing". It was one of those technologies that topped the Gartner Hype Cycle sometime during the first Web 2.0 years. Interesting to see it back again. It had many problems but was always an interesting concept.

simongray 6y ago

I've had to dive into this recently as part of my work (I'm in academia). While the original vision seems to have failed from a lack of incentives, the idea behind RDF as way to model connected data and its many serialisation formats is actually quite helpful. It seems to have found a niche in academia when it comes to sharing and integrating large graph datasets. I just consider RDF and most of the other semantic web standards as a framework for modelling and serialising graph data. It's pretty good at that.

tasogare 6y ago
No, it’s not even good for that. The fact you need yo reify properties (relationships) to annotate them is a big failure, which is not present in more general graph models.
- Karrot_Kream 6y ago
  I think this is a matter of opinion. I'm personally a fan of reified properties, but this is the source of differences between property graph databases and RDF triple stores.
- simongray 6y ago
  Works fine for my use cases (e.g. representing multiple WordNets as labeled property graphs in Neo4j) and I quite like the simplicity of data modelling based on triplets of data literals. What do you suggest as an alternative to RDF?
- emmanueloga_ 6y ago
  Not true. See "reification is a red herring" [1]. Also, there's now a more natural way to add properties to relationships if really needed, it is call RDF* [2], and it is already implemented of a lot of triplestores, including RDF4J, Jena, StarDog and more. 1: http://www.snee.com/bobdc.blog/2018/04/reification-is-a-red-herring.html http://www.snee.com/bobdc.blog/2018/04/reification-is-a-red-... 2: https://blog.liu.se/olafhartig/2019/01/10/position-statement-rdf-star-and-sparql-star/ https://blog.liu.se/olafhartig/2019/01/10/position-statement...

hmottestad 6y ago

I'm a committer on Eclipse RDF4J and currently working on our implementation of SHACL. I've worked with semantic web now for 7-8 years I think. Seen quite a few projects struggle or fail mainly due to performance issues. That is why I'm doing my best with optimising RDF4J.

emmanueloga_ 6y ago
Hey there! I actually saw this on my own experiments! I was experimenting with validating some data through the SHACL version of schema.org [1]. To be fair my findings are not much more than "anecdata" for now, since a proper benchmark should use a bigger dataset, allow the JVM to warm up, etc... But from a quick comparison, the RDF4j implementation was on my tests from 2 to 3 times faster than the "reference" one [2] when validating some small in-memory data. 1: http://datashapes.org/schema http://datashapes.org/schema 2: https://github.com/topquadrant/shacl https://github.com/topquadrant/shacl
- hmottestad 6y ago
  You will notice a much bigger difference once you have a larger database and want to make changes to your data. RDF4J is able to analyse your changes and validate only the smallest amount of data to make sure that everything is still valid.
  - emmanueloga_ 6y ago
    Great to know!

enumjorge 6y ago

I appreciate the spirit of the semantic web, but I've never really been clear on how it would accomplish its stated goals. Let's imagine the web has lots of annotated, machine-readable data. You still need a client to explore that data, right? The browser is not going to know how to make sense of the relationships between your data points. How is the end user going to consume the information, a web app? As the dominance of companies like Facebook, Google and Twitter have shown, the average user doesn't care about de-centralization. They care about a good user experience. Let's take the recipe scenario presented in the article. What's to stop Google from ingesting all of this semantic data, and then using their army of software devs to integrate the results into Google's search results? Google has invested in natural language processing that can allow their web crawlers to make semantic sense of written text. I'm sure they would love it if a swath of volunteers did this for them instead. Sure, some small, indie shop could use the data too. But Google can integrate with their translation engine, their home speakers and Android. They'll make the experience nice enough that your aunt looking for a scone recipe won't want to leave their ecosystem, even if a more free, de-centralized alternative is available.

jcelerier 6y ago
> What's to stop Google from ingesting all of this semantic data, and then using their army of software devs to integrate the results into Google's search results? that's pretty much exactly what they were doing with freebase (https://developers.google.com/freebase/guide/basic_concepts https://developers.google.com/freebase/guide/basic_concepts)
elliottinvent 6y ago
Yes, I think the semantic web is suited to big companies with large teams and huge processing capabilities – they hoover the data up and add it to their own graph. As you've pointed out, the trend is for Google to pull it out of the website and display it on the results page, cutting the content creator out entirely. Building a tool or service on top of the semantic web is out of reach for most developers because of inconsistencies and complexities, unless they use the tools provided by the giants – and that goes directly against the principles of the semantic web. The exception here is niche use cases and academia working with huge datasets, where the semantic web seems to be incredibly useful.

emmanueloga_ 6y ago

The problem with the "semantic web" projects is that they are always proposing "pie in the sky" projects that require a big leap of faith to get invested on. I see solidproject.org as an example of this. I've been exploring semantic technologies from the cold and incredible practical angle of "my own needs" :-). The RDF model offers a lot even when keeping data for myself: I can use RDF to infer new things from existing data and visualize relationships. An RDF triplestore can be used to store pretty much anything, and works well for CRUD. RDF is about graphs, so a triple store can be used to model documents too, since trees are a subset of all graphs. RDF4J has really nice documentation and supports a lot of modern features. The getting started guide is great [1], and there's also a super friendly gitter room to ask questions [2]. 1: https://rdf4j.org/documentation/tutorials/getting-started/ https://rdf4j.org/documentation/tutorials/getting-started/ 2: https://gitter.im/eclipse/rdf4j https://gitter.im/eclipse/rdf4j

darkhorse13 6y ago
>The problem with the "semantic web" projects is that they are always proposing "pie in the sky" projects I have read this sentiment countless of times, but I don't quite understand it. Maybe because I am not familiar with the semantic web. Can you please suggest some reading list for a person who is unfamiliar with semantic web to better understand these issues?
- emmanueloga_ 6y ago
  The first thing you should read is the RDF primer [1]. It explains the basic data model that the semantic web uses as building block for anything else. The gist of it is to provide a way to model data as a graph, and make it almost trivial to exchange data between parties. A simple example is, if you and I want to share, say, user data, and you have a "people" table and I have "Users", and my fields are "first_name" "last_name" and yours "given" and "proper", and so on, how do we match the same user across tables? RDF provides a solution for that. The "pie in the sky" part is that, from this basic building block, a whole host of other technologies can be built, some of them dealing with artificial intelligence, machines being able to _infer_ more information from what you have already said [2]. Inferences can work best when there's more data, that's were the ability to mix your data with third party databases comes handy. Solid adds even more stuff to the table. In their words: derived from "social linked data", is a proposed set of conventions and tools for building decentralized social applications based on Linked Data principles. I think "decentralized" is the keyword here. I haven't even started looking at it :-) 1: https://www.w3.org/TR/rdf11-primer/ https://www.w3.org/TR/rdf11-primer/ 2: https://www.w3.org/standards/semanticweb/inference https://www.w3.org/standards/semanticweb/inference

elliottinvent 6y ago

The semantic web makes sense for large datasets provided by tech-enabled companies, but the overwhelming majority of entities represented on the web don't have large datasets or tech teams. They have a small amount of standard data they want to get out to customers – contact data, company data (tax registration or whatever), GPS locations for stores, etc. They don't need 99.9% of the complexities of Schema.org. They only consider a website from their viewpoint – a human viewing the website through browser. The HTML (including microdata, JSON-LD, etc) is terrifying for most of them. So at best, their web designer builds in some semantic web standards but the data can be inaccurate or become out of data quickly because the website owner doesn't even know it's there. Even big companies have adopted semantic web standards in such a confusing and inconsistent way that it's difficult to rely on the data being available in an expected format, so that a developer can build something on top of it. Unless semantic web standards can be adopted by small businesses in a simple way (point, click and type) I don't think they will ever be widely adopted. Finally, I don't think it's efficient to hide the 1kb of machine usable data inside 99kb of HTML designed for browser rendering. Shameless plug: A small team and I have built an alternative to the semantic web, to store machine readable data in DNS. It can be populated by simple user friendly online forms. Explainer and summary: https://www.num.uk https://www.num.uk Best way to play around with it: https://app.numserver.com/tools/editor/add https://app.numserver.com/tools/editor/add

Vinnl 6y ago

Interesting that the article doesn't touch on Tim Berners-Lee's current focus: https://solidproject.org https://solidproject.org (Disclosure: I work for his startup, Inrupt, which he founded to give that project a boost. Opinions here are my own though.) Although it's primarily known for allowing apps to store their users' data in a place under their users' control, for better or worse, it's built on many SemWeb technologies. And even though I'm still somewhat disillusioned about the Semantic Web (like Aaron Schwarz, I used to be enthusiastic about it way back when), but there certainly is a chance it will be interesting again when your personal (potentially private) data can be part of the Semantic Web as well. For example, if they'd have written their reviews in a Solid-based website, linking them to MusicBrainz would be a relatively small step.

laurencerowe 6y ago
The universal login aspect of Solid sounds rather like Mozilla’s Persona. What about Solid makes it more likely to succeed against the incumbents than Persona?
- Vinnl 6y ago
  There are several things that make it different, making the reason Persona failed not apply (though obviously it might still fail for different reasons): - There are several organisations behind Solid that whose raison d'etre is Solid. I'd argue that Mozilla possibly pulled the plug for Persona too early, specifically, without first adding support for it in Firefox, thereby creating massive incentive for website owners to support it. But then, it wasn't their core focus, and there was no return on investment for them, so I can see why they cancelled it. - Ironically, a wider focus. The incentive for website owners to support Persona would have been that it would be easier for potential users to join - although they already got most of that benefit by allowing sign-in with Facebook, and most people did not actually use Persona. Solid, on the other hand, has several incentives to support it. It can give organisations access to data that the user is actively curating, rather than having to provide it to different organisations individually, with it getting outdated the moment they shared it. Additionally, given how, for many organisations, data is increasingly (post-GDPR and all that) a liability and not their primary competency, outsourcing management of user data to the user themselves makes sense. And finally, there will hopefully be more people with a Solid Pod, especially in specific markets - for example, IIRC the government of Flanders was looking at giving every citizen a Pod. - It can lean on the marketing benefits of it being TimBL's main focus, which may or may not be stronger than being one of Mozilla's projects. - Depending on your definition of success, it can be as successful as Mastodon/the Fediverse is. As opposed to Persona, people can and do write apps specifically to support Solid, because it's more than just a login. Just off the top of my head, and again, personal views. But hopefully that gives some insight.

kangnkodos 6y ago

The problem is the economic incentives. The people who have to do extra work are the people providing the data. The people who get the benefit are the people consuming the data.

jandrewrogers 6y ago

The semantic web currently scales poorly in two different dimensions. As long as this remains the case, it will see little adoption. First, the software technology scales very poorly unless the size of your data model is trivial. The semantic web is a type of graph search problem with all of the hard algorithm and data structure challenges that entails. Most interesting semantic web applications require data models operating at scales where existing graph platforms have pathological performance characteristics. The gap between capability and requirements is several orders of magnitude. You can't throw hardware at it; when I was first hired to do CS research on scaling the semantic web, I was working on literal supercomputers. Second, the human element of the semantic web doesn't scale. Semantics are contextual and subjective. As the number of people contributing to the data model increases, the consistency and quality of the data model decreases. This becomes an extremely expensive global coordination problem at scale that may be unsolvable. It is broadly recognized that you need a single, global arbiter of semantics in any scalable system just to maintain semantic consistency, which basically means strong AI. If we have the technology to solve these two problems, I suspect the Semantic Web will no longer be interesting.

pfraze 6y ago
I think you said it pretty well, but I'll re-iterate some of your second point. The human element defies any solution other than social coordination such as a specs registry or org. At some point the software needs to understand each other and that requires people to work together. A technical solution around schemas can assist the coordination work, but it can't replace it. EDIT to add: Since we can't remove human coordination, I question the value of arbitrary schema composition when it incurs the huge DX cost of using a URL to represent every attribute.
- infinite8s 6y ago
  Alan Kay had a comment on here from a few years ago on how this could scale: https://news.ycombinator.com/item?id=15320789 https://news.ycombinator.com/item?id=15320789
- bigmanwalter 6y ago

ilaksh 6y ago

I think that AI is getting close to the point that it is going to be a significant factor in the story of the semantic web from here on. What comes to mind is people training AI to parse web pages and mark them up to indicate different types of data inside of them. Such as the recipe example. So someone who authors a WordPress plugin for recipes could incorporate it. Or, and this is more scalable but obviously very difficult, it may be possible to create plugins for web frameworks that automatically inject the semantic markup using AI. Maybe. Then there could be another AI program that converts natural language queries into semantic knowledge queries. But I think that AI could play a big part in rolling it out. I mean, if you had the AI systems without the semantic web, for open systems you would still want a way for them to share data unambiguously.

gumby 6y ago

The semantic web can only work via automation -- pretty much complete automation. Semantic information depends on a "stance" (those humanities classes do pay off) since the same information can mean different things depending on context and perspective. In addition spam will continue to be a significant problem.

deleted 6y ago

[deleted]

mimixco 6y ago

We've been working on an alternative to SemWeb based on a new language for marking up facts inside of text. It doesn't rely on schema but allows the user to define fact relationships as they happen in writing. More info here: https://news.ycombinator.com/item?id=24662705 https://news.ycombinator.com/item?id=24662705

mitchtbaum 6y ago

I have it working, https://github.com/speakeasy-engine https://github.com/speakeasy-engine We can have semantic social networking now, https://github.com/patchworkuniversity/radiojade https://github.com/patchworkuniversity/radiojade

mitchtbaum 6y ago
This was our initial Alice and Bob: https://github.com/foundpatternscellar/ping-pong https://github.com/foundpatternscellar/ping-pong

Animats 6y ago

The trouble with the "semantic web" is that the cost is paid by those creating content, and the benefit is obtained by those running web scrapers.