3 ms·
When it comes to these older sites, it's hard for an automated engine to discover what they're about because (1) they lack the kind of structured description th
by zozbot234 5y ago
When it comes to these older sites, it's hard for an automated engine to discover what they're about because (1) they lack the kind of structured description that's far more common on modern sites - including, to some limited extent, spammy ones (although extensive structured descriptions would nonetheless tend to favor legitimate content) - and that powers "smart" suggestions in search results. Also, (2) the web directories that would've provided an accurate description back when those sites were current are now dead. What we'd need to improve the Web search ecosystem is for "non-commercial", hobbyist sites to work on addressing both of these problems.
- marginalia_nu 5y agoHTML has had metadata tags for ever, it's just that they quickly stopped being used by search engines were so inaccurate and prone to abuse. Even now, the heavy presence of these types of tags is arguably a marker that a website is really interested in its google ranking, and probably fairly spammy. Any sort of description or tagging or keywords or genre description needs third party vetting to be of any use what so ever. It's simply too profitable to misrepresent your websites for it to be any other way.
- zozbot234 5y agoThose metadata tags were just a simple textual description and a bunch of keywords with no reference to any controlled vocabulary. This is what made them so easy to abuse. Modern schema-based structured data is vastly different, and with a bit of human supervision (that's the "third party vetting") it's feasible to tell when the site is lying. (Of course, low-quality bulk content can also be given an accurate description. But this is good for users, who can then more easily filter out that sort of content.) One could even let this vetting happen in decentralized fashion, by extending Web Annotation standards to allow for claims of the sort "this page/site includes accurate/inaccurate structured content."
- marginalia_nu 5y agoThe thing is "a bit of human supervision" is difficult on a scale of ten thousand Wikipedias. It pretty much needs to be done completely automatically.