4 ms·

That Yahoo took so long to give up the DMOZ directory-style mentality is just another example of technologists favoring a neatly ordered solution over an organi

by holdenc 10y ago

That Yahoo took so long to give up the DMOZ directory-style mentality is just another example of technologists favoring a neatly ordered solution over an organic one. In the crowd-sourced organic model, there is less curation, and more a time-based record of fleeting curiosity.

dredmorbius 10y ago
Curation worked, early, in that there was relatively little online content. As late as 1998, the quality content online was perhaps 10,000 books: Despite the hype, the Web just isn't all that impressive as an information resource. The static, publicly accessible HTML text on the Web is roughly equivalent in size to a million books. The UC Berkeley Library has 8 million volumes, and the average quality of the Berkeley library content is much, much higher! If 10% of the material on the Web is ``useful,'' then that means there are about 100,000 useful book-equivalents on the Web, which is the size of good public library. The actual figure for ``useful'' is probably more like 1%, which is 10,000 books, or half the size of an average mall bookstore. http://people.ischool.berkeley.edu/~hal/Papers/japan/index.html http://people.ischool.berkeley.edu/~hal/Papers/japan/index.h... I'm leaning strongly toward curation via tagging, using standard ontologies, and vetted through designated / qualified experts for top content. A mix of crowdsourcing and expertise seems most viable. Quasi-related -- a rant on Pocket suggesting improved "tags" functionality (I swear there's another, better, though I cannot find it in the usual places). https://ello.co/dredmorbius/post/3eucmp_s0tumjuoxmfclbw https://ello.co/dredmorbius/post/3eucmp_s0tumjuoxmfclbw
- naveen99 10y ago
  Maybe, but the web lets you ask questions for which there is no answer yet. But you might get one tomorrow once you ask. Also youtube is an amazing resource that books can't compete with. Tags and ontologies also can't compete with search, Bigquery and data lakes.
  - dredmorbius 10y ago
    Read Varian's paper (above), it anticipates a great deal, especially his commentary on Gresham's Law and content. There's also one from a couple of Stanford researchers a few years back which suggest a similar problem: Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is "The Effect of Cellular Phone Use Upon Driver Attention", a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving.... It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers. http://goo.gl/CTwEmw http://goo.gl/CTwEmw Including in my tagging schemas for Pocket are quality indicators. I've been using an arbitrarily selected ranking scale of "a-1, a-2, ... a-5" to indicate quality (higher is better), and also tagging items with keywords indicating negative attributes (usually "bullshit", though others apply). Being able to group classifications by reputation of author or publisher is also a key element. While there are hazards in argument from authority, there are frequently benefits both to recognising expertise and identifying non-credible or impeached sources. If a source routinely pursues some objective rather than the truth, you might tend to question its value. PageRank essentially relies on the implicit reputation vetting of links, using link sources as a basis for quality assessment. It's useful but also limited. And as noted, the bias toward advertising and specific gain (see also Celine's 2nd Law, from Robert Anton Wilson's "Illuminatus" trilogy) raises issues. I've been doing significant amounts of research, much of it online, but much also tracking down published book and article sources (most of those through illicit download sites -- really the only way the information can be obtained reasonably), and what I'm finding is that the noise associated with online data is just boggling. Books, it turns out, compete exceptionally well with YouTube. Ultimately they compliment one another. Sometimes I want to read, sometimes I want a lecture or demo. Both have their place. Excessive data ultimately isn't illuminating. Yes, online search can be useful at finding out, say, what's happening right here, now (transit applications are a huge one), or within a given social circle, but both of those are domains in which an exclusion scope is set which rules out virtually all available results -- you're looking for information about a specific location and time, or concerning a specific group of people. When it comes to general information, "reliable sources" dwindle. If you're looking for some established authority or text, you're left with the specific sources. If it's news, typically you're going to want some vetted news source -- and yes, that includes informally vetted sources on Twitter or YouTube. One of the most annoying things about both sources are the crap leaches who instantly show up to clog any trending keyword topic with crap content. That shit's got to get stomped hard. For new and developing information, ways to broadly pre-filter and bubble up quality content are an interesting problem and one I'm actively working on. Community rather than broadcast methods strike me as more viable.

by holdenc 10y ago

dredmorbius 10y ago

Curation worked, early, in that there was relatively little online content. As late as 1998, the quality content online was perhaps 10,000 books: Despite the hype, the Web just isn't all that impressive as an information resource. The static, publicly accessible HTML text on the Web is roughly equivalent in size to a million books. The UC Berkeley Library has 8 million volumes, and the average quality of the Berkeley library content is much, much higher! If 10% of the material on the Web is ``useful,'' then that means there are about 100,000 useful book-equivalents on the Web, which is the size of good public library. The actual figure for ``useful'' is probably more like 1%, which is 10,000 books, or half the size of an average mall bookstore. http://people.ischool.berkeley.edu/~hal/Papers/japan/index.html http://people.ischool.berkeley.edu/~hal/Papers/japan/index.h... I'm leaning strongly toward curation via tagging, using standard ontologies, and vetted through designated / qualified experts for top content. A mix of crowdsourcing and expertise seems most viable. Quasi-related -- a rant on Pocket suggesting improved "tags" functionality (I swear there's another, better, though I cannot find it in the usual places). https://ello.co/dredmorbius/post/3eucmp_s0tumjuoxmfclbw https://ello.co/dredmorbius/post/3eucmp_s0tumjuoxmfclbw

naveen99 10y ago
Maybe, but the web lets you ask questions for which there is no answer yet. But you might get one tomorrow once you ask. Also youtube is an amazing resource that books can't compete with. Tags and ontologies also can't compete with search, Bigquery and data lakes.
- dredmorbius 10y ago
  Read Varian's paper (above), it anticipates a great deal, especially his commentary on Gresham's Law and content. There's also one from a couple of Stanford researchers a few years back which suggest a similar problem: Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is "The Effect of Cellular Phone Use Upon Driver Attention", a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving.... It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers. http://goo.gl/CTwEmw http://goo.gl/CTwEmw Including in my tagging schemas for Pocket are quality indicators. I've been using an arbitrarily selected ranking scale of "a-1, a-2, ... a-5" to indicate quality (higher is better), and also tagging items with keywords indicating negative attributes (usually "bullshit", though others apply). Being able to group classifications by reputation of author or publisher is also a key element. While there are hazards in argument from authority, there are frequently benefits both to recognising expertise and identifying non-credible or impeached sources. If a source routinely pursues some objective rather than the truth, you might tend to question its value. PageRank essentially relies on the implicit reputation vetting of links, using link sources as a basis for quality assessment. It's useful but also limited. And as noted, the bias toward advertising and specific gain (see also Celine's 2nd Law, from Robert Anton Wilson's "Illuminatus" trilogy) raises issues. I've been doing significant amounts of research, much of it online, but much also tracking down published book and article sources (most of those through illicit download sites -- really the only way the information can be obtained reasonably), and what I'm finding is that the noise associated with online data is just boggling. Books, it turns out, compete exceptionally well with YouTube. Ultimately they compliment one another. Sometimes I want to read, sometimes I want a lecture or demo. Both have their place. Excessive data ultimately isn't illuminating. Yes, online search can be useful at finding out, say, what's happening right here, now (transit applications are a huge one), or within a given social circle, but both of those are domains in which an exclusion scope is set which rules out virtually all available results -- you're looking for information about a specific location and time, or concerning a specific group of people. When it comes to general information, "reliable sources" dwindle. If you're looking for some established authority or text, you're left with the specific sources. If it's news, typically you're going to want some vetted news source -- and yes, that includes informally vetted sources on Twitter or YouTube. One of the most annoying things about both sources are the crap leaches who instantly show up to clog any trending keyword topic with crap content. That shit's got to get stomped hard. For new and developing information, ways to broadly pre-filter and bubble up quality content are an interesting problem and one I'm actively working on. Community rather than broadcast methods strike me as more viable.