4 ms·

Among the 30-odd technologies that make up the Semantic Web[1] (it never died, it's just a collection of tech, lots of organizations use it daily) are graph dat

Among the 30-odd technologies that make up the Semantic Web[1] (it never died, it's just a collection of tech, lots of organizations use it daily) are graph databases[2]. Graph databases are necessary to implement semantic web databases. SQLite is not a graph database. Even if you used SQLite to implement a graph database, it would not solve any significant problems of the semantic web, such as access to data, taxonomies, ontologies, lexicons, tagging, user interfaces to semantic data management, etc. It's a really odd suggestion that you would just copy around a database or leave it on the internet for people to copy from. For the BBS mentioned here, that might actually be illegal, as it might contain PII, and on other sites possibly PHI. Many countries now have laws that require user data to remain in-country. Besides the challenges of just organizing data semantically, there still needs to be work done on data security controls to prevent leaking sensitive information. The funny thing is, that isn't even hard to do with the semantic web. You classify the data that needs protecting and build functions and queries to match. You can tie that data to a unique ID so that people can "own" their data wherever it goes, and sign it with a user's digital certificate which can also expire. But all of that (afaik) doesn't exist yet. Everyone is more concerned with blockchains and SQL, either because the fancy new tech is sexier, or the old boring tech doesn't require any work to implement. The Semantic Web never caught on because it's really fucking hard to get right. No companies are investing in making it easier. Maybe in 20 years somebody will get bored enough over a holiday to make a simple website creation tool that implicitly creates semantic web sites that are easy to reason about. It'll probably be a WordPress plugin. [1] https://en.wikipedia.org/wiki/Semantic_Web https://en.wikipedia.org/wiki/Semantic_Web [2] https://graphdb.ontotext.com/documentation/enterprise/introduction-to-semantic-web.html https://graphdb.ontotext.com/documentation/enterprise/introd...

sekao 5y ago
> Graph databases are necessary to implement semantic web databases. The online docs (and TBL himself) rarely mention of graph databases, but obviously the idea is tied tightly to RDF. Separating it from that implementation detail is part of the point, though. Getting people to represent their data via an additional format was never going to work. > For the BBS mentioned here, that might actually be illegal, as it might contain PII Can't imagine the purpose you had in even making this point. In theory, any arbitrary database exposed publicly could be illegal to replicate due to copyright, PII laws, etc. But that has nothing at all to do with a technical discussion of a technique for exposing data. What a bizarre point to make. As an aside, I'm glad you removed the "Uh........." from the beginning of your post. We're all making an effort to reduce the typical HN snark in the comments, and there's always room for improvement :D
zozbot234 5y ago
> Graph databases are necessary to implement semantic web databases. This just isn't true, on multiple levels. RDF is an interoperability standard that does not per se depend on a 'graph-like' data model - you can very much expose plain old relational data via RDF, and this is quite intended. Additionally, modern general-purpose RDBMS's support graph-focused data models quite well, despite being built on 'relational' principles - there's no need for special tech when working with general-purpose graph models, unless you're doing some sort of heavy-duty network analytics.
- 0xbadcafebee 5y ago
  You're talking about extending a database design created 50 years ago to work with models and methods that involve significantly different operations and concepts. Let the RDBMS die so we can make something that is much more powerful and requires less fidgeting and squinting to work the way we want. RDBMS were a niche research project for a decade before they started to catch on in business apps. They've stayed around forever because they're just functional enough to be dangerous. But we've already hit the upper limits of both reliability and performance years ago (remember NoSQL?) and we just keep bolting on features because nobody wants to leave them. The old designs and implementations are holding us back.
  - mdavidn 5y ago
    You and I work on very different kinds of projects. I find myself fidgeting and squinting at the database far more when it fails to enforce a schema or provide ACID transactions.
  - gmfawcett 5y ago
    Was this supposed to be a rallying cry, or a grumpy rant? I'm getting a mixed message.
    - 0xbadcafebee 5y ago
      That's... a perfect summary of everything I think about technology. Very grumpy and very optimistic. :)
Karrot_Kream 5y ago
The semantic web failed to become widely popular because: 1. Graph databases on top of triple stores are a lot less scalable than relational databases or key-value stores, and this is how semantic data is meant to be stored/queried. 2. Data is valuable. Handing out data for free in a machine-consumable way is both expensive (machines can request data much more quickly than a human) and a recipe for copycats. The incentives just aren't there. TBL's Solid project is about trying to separate semantic data providers from the presentation layer and opening up the possibility of payment from these data providers to try to improve the incentives around semantic data sharing.
NetOpWibby 5y ago
Thanks for the links!
nescioquid 5y ago
> The Semantic Web never caught on because it's really fucking hard to get right. No companies are investing in making it easier. I really appreciate this point. I had the opportunity to work on an exploratory project with an experienced ontologist (yes, you really need one of those, I think). The tools were fascinating (reasoners quickly became necessary) but I had the feeling that many of these tools were at a comparatively early stage of maturity. Trying to explain to people how the system would work was a challenge as it required a primer on theory and application -- we glazed many eyes. The CTO wanted to know if we could use blockchain somehow. Another group addressed a slice of the problem with technologies already in use and that decided the matter.
- zozbot234 5y ago
  > reasoners quickly became necessary Ouch. Most uses of reasoners/inference are quite computationally-intensive, to the point of making "reasoning" quickly infeasible. But if you really want, you can do all this stuff in traditional databases by defining appropriate 'views' and having your application query them. You could even use custom database triggers to enable inserts/updates on views.
  - hobofan 5y ago
    Really depends on the reasoner in use. I'd really take most current public benchmarks on reasoner performance with a grain of salt, as most implementations out there are mostly academic-grade non-production systems. E.g. Stardog does most of their reasoning via query rewriting (and also lean on some restrictions). That way you can leverage DBs to do what they are good at. If you can then on top of that build some clever caching or incremental computation, you should be fine for even pretty huge dataset sizes.
smarx007 5y ago
There is a wiki extension to embed RDF data: https://www.mediawiki.org/wiki/Extension:LinkedWiki https://www.mediawiki.org/wiki/Extension:LinkedWiki There is a WP plugin to expose some basic data: https://wordpress.org/plugins/wp-linked-data/ https://wordpress.org/plugins/wp-linked-data/ and a Swedish startup Metasolutions has Wordpress plugins for embedding any kind of RDF information: https://docs.entryscape.com/en/blocks/ https://docs.entryscape.com/en/blocks/
simonw 5y ago
SQLite could actually make a really good basis for building a graph database, thanks to "Many Small Queries Are Efficient In SQLite": https://www.sqlite.org/np1queryprob.html https://www.sqlite.org/np1queryprob.html I took advantage of that for my datasette-graphql plugin - it's not a graph database, but it does allow deeply nested graph-like queries that take advantage of SQLite's fast small query performance: https://datasette.io/plugins/datasette-graphql https://datasette.io/plugins/datasette-graphql

sekao 5y ago

> Graph databases are necessary to implement semantic web databases. The online docs (and TBL himself) rarely mention of graph databases, but obviously the idea is tied tightly to RDF. Separating it from that implementation detail is part of the point, though. Getting people to represent their data via an additional format was never going to work. > For the BBS mentioned here, that might actually be illegal, as it might contain PII Can't imagine the purpose you had in even making this point. In theory, any arbitrary database exposed publicly could be illegal to replicate due to copyright, PII laws, etc. But that has nothing at all to do with a technical discussion of a technique for exposing data. What a bizarre point to make. As an aside, I'm glad you removed the "Uh........." from the beginning of your post. We're all making an effort to reduce the typical HN snark in the comments, and there's always room for improvement :D

zozbot234 5y ago

> Graph databases are necessary to implement semantic web databases. This just isn't true, on multiple levels. RDF is an interoperability standard that does not per se depend on a 'graph-like' data model - you can very much expose plain old relational data via RDF, and this is quite intended. Additionally, modern general-purpose RDBMS's support graph-focused data models quite well, despite being built on 'relational' principles - there's no need for special tech when working with general-purpose graph models, unless you're doing some sort of heavy-duty network analytics.

0xbadcafebee 5y ago
You're talking about extending a database design created 50 years ago to work with models and methods that involve significantly different operations and concepts. Let the RDBMS die so we can make something that is much more powerful and requires less fidgeting and squinting to work the way we want. RDBMS were a niche research project for a decade before they started to catch on in business apps. They've stayed around forever because they're just functional enough to be dangerous. But we've already hit the upper limits of both reliability and performance years ago (remember NoSQL?) and we just keep bolting on features because nobody wants to leave them. The old designs and implementations are holding us back.
- mdavidn 5y ago
  You and I work on very different kinds of projects. I find myself fidgeting and squinting at the database far more when it fails to enforce a schema or provide ACID transactions.
- gmfawcett 5y ago
  Was this supposed to be a rallying cry, or a grumpy rant? I'm getting a mixed message.
  - 0xbadcafebee 5y ago
    That's... a perfect summary of everything I think about technology. Very grumpy and very optimistic. :)

Karrot_Kream 5y ago

The semantic web failed to become widely popular because: 1. Graph databases on top of triple stores are a lot less scalable than relational databases or key-value stores, and this is how semantic data is meant to be stored/queried. 2. Data is valuable. Handing out data for free in a machine-consumable way is both expensive (machines can request data much more quickly than a human) and a recipe for copycats. The incentives just aren't there. TBL's Solid project is about trying to separate semantic data providers from the presentation layer and opening up the possibility of payment from these data providers to try to improve the incentives around semantic data sharing.

NetOpWibby 5y ago

Thanks for the links!

nescioquid 5y ago

> The Semantic Web never caught on because it's really fucking hard to get right. No companies are investing in making it easier. I really appreciate this point. I had the opportunity to work on an exploratory project with an experienced ontologist (yes, you really need one of those, I think). The tools were fascinating (reasoners quickly became necessary) but I had the feeling that many of these tools were at a comparatively early stage of maturity. Trying to explain to people how the system would work was a challenge as it required a primer on theory and application -- we glazed many eyes. The CTO wanted to know if we could use blockchain somehow. Another group addressed a slice of the problem with technologies already in use and that decided the matter.

zozbot234 5y ago
> reasoners quickly became necessary Ouch. Most uses of reasoners/inference are quite computationally-intensive, to the point of making "reasoning" quickly infeasible. But if you really want, you can do all this stuff in traditional databases by defining appropriate 'views' and having your application query them. You could even use custom database triggers to enable inserts/updates on views.
- hobofan 5y ago
  Really depends on the reasoner in use. I'd really take most current public benchmarks on reasoner performance with a grain of salt, as most implementations out there are mostly academic-grade non-production systems. E.g. Stardog does most of their reasoning via query rewriting (and also lean on some restrictions). That way you can leverage DBs to do what they are good at. If you can then on top of that build some clever caching or incremental computation, you should be fine for even pretty huge dataset sizes.

smarx007 5y ago

There is a wiki extension to embed RDF data: https://www.mediawiki.org/wiki/Extension:LinkedWiki https://www.mediawiki.org/wiki/Extension:LinkedWiki There is a WP plugin to expose some basic data: https://wordpress.org/plugins/wp-linked-data/ https://wordpress.org/plugins/wp-linked-data/ and a Swedish startup Metasolutions has Wordpress plugins for embedding any kind of RDF information: https://docs.entryscape.com/en/blocks/ https://docs.entryscape.com/en/blocks/

simonw 5y ago

SQLite could actually make a really good basis for building a graph database, thanks to "Many Small Queries Are Efficient In SQLite": https://www.sqlite.org/np1queryprob.html https://www.sqlite.org/np1queryprob.html I took advantage of that for my datasette-graphql plugin - it's not a graph database, but it does allow deeply nested graph-like queries that take advantage of SQLite's fast small query performance: https://datasette.io/plugins/datasette-graphql https://datasette.io/plugins/datasette-graphql