4 ms·
I worked on the Semantic Web, designing core data infrastructure, back when it was still hot. It disappeared because it had two fatal flaws: it was intrinsicall
by jandrewrogers 8y ago
I worked on the Semantic Web, designing core data infrastructure, back when it was still hot. It disappeared because it had two fatal flaws: it was intrinsically non-scalable, both conceptually and technically.
First, there is no universal "semantic". The meaning of things is ambiguous, and given a broad enough pool of people and cultural contexts, it becomes nigh impossible to converge on a single consistent model for individual terms and concepts in practice. A weak form of this is very evident in global data interchange systems and implementations, and the Semantic Web took this bug factory and dialed it up to eleven. In short, the idea of the Semantic Web requires semantics to be axiomatic and in the real world they are inductive and deeply contextual. (Hence the data model rule of "store the physics, not the interpretation of the physics.") That said, it is often possible to build an adequate semantic model in sufficiently narrow domains -- you just can't generalize it to everything and everybody.
Second, implementation at scale requires an extremely large graph database, and graph database architectures tend to be extremely slow and non-scalable. They were back then and still are today. This is actually what killed the Semantic Web companies -- their systems became unusable at 10-100B edges but it was clear that you needed to have semantic graphs in the many trillions of edges before the idea even started to become interesting. Without an appropriate data infrastructure technology, the Semantic Web was just a nice idea. Organizations using semantic models today carefully restrict the models to keep the number of edges small enough that performance will be reasonable on the platforms available.
The Semantic Web disappeared because it is an AI-Complete problem in the abstract. This was not well understood by its proponents and the systems they designed to implement it were very, very far from AI-Complete.
- openfuture 8y agoThis is why category theory is so important! It allows us to move between axiomatic systems by looking at the structure they have in common. I am convinced that the 'semantic web' will be accomplished via some easy to use version control meets functors gui program.
- jeremiem 8y agoYour first point is being worked on and there are a few upper ontologies being used. I find BFO [1] quite promising. I believe that we actually can generalize modelling. We have dictionaries filled with definitions, given enough time and discipline, I don't see why we couldn't make them formal. It's not an engineering problem though. [1] http://basic-formal-ontology.org/ http://basic-formal-ontology.org/
- subhobroto 8y agoYour experience shows! I made a few observations in my own comment. One being that there is no usable graph store you and I can use as of 2018. Another being about monetizing the Semantic Web when playing the role of the data/ontology provider. You provide all the data while the consumers (Siri, Alexa and Google Home) get the glory: https://news.ycombinator.com/item?id=18036041 https://news.ycombinator.com/item?id=18036041
- tokyodude 8y agoThird, you can't force people to use the correct semantics. They'll use them wrong on purpose for fun and profit. Mark some disturbing content as wholesome, mark it as whatever is popular at the moment to get it in front of more eyeballs, mark it as something only tangentially related in the hope there's a cross over of market, mark it wrong because they don't actually know better.
- smadge 8y agoI don’t really understand this argument, because there already are lies published on the internet. What difference does it make if those lies are published in a standardized machine readable format or not?
- moltensyntax 8y agoWe as humans have a lot of intuitive tools for knowing whether a source of data is trustworthy. AI could possibly approach this ability given enough training... we'd need to do something like add a "trust" score to every node in the graph.
- tokyodude 8y agoWhat's the point of having the marks at all if they are not actually correct?
- Volt 8y agoIf you're answering the question, I think it would be good to answer it directly.
- PeterisP 8y agoA human-targeted web structure that contains some lies is still useful for humans because humans can filter out those lies with somewhat satisfactory efficiency. A machine-targeted web structure that contains some lies is not useful for machines because they can't filter out those lies yet. It might become useful when they can (but that might be a hard-AI problem), but it's simply not usable until that point.
- jbreckmckye 8y ago> First, there is no universal "semantic". The meaning of things is ambiguous, and given a broad enough pool of people and cultural contexts, it becomes nigh impossible to converge on a single consistent model for individual terms and concepts in practice It sounds like the Semantic Web failed because we tried treating a longstanding (and possibly unresolvable) ontological problem as a straightforward and technical one. I don’t really follow academic philosophy, but is it known these days if such categorisation problems are even “solvable” in the general case?