14 ms·

When Yahoo Ruled the Valley: Stories of the Original ‘Surfers’

103 points by miraj 10y ago

gkop 10y ago
Where does human curation of the web live now? Is it all temporal (Reddit, Facebook, Twitter, HN, etc), or is there somewhere central that humans organize it "for all time"? Would a modern Yahoo with the staff surfers replaced by the crowd provide value to the world? Could it possibly work without being horribly gamed? Are there good examples of having tried this idea?
- pavlov 10y ago
  Pinterest is more curation-oriented than the other social networks... And Wikipedia of course functions as a sort of directory for many topics.
- empath75 10y ago
  Dmoz still exists and is still run by AOL.
  - dredmorbius 10y ago
    Indeed it does. I'd forgotten it was AOL that ended up with it though, that's rather discordant.
- firasd 10y ago
  Interesting to ponder. I'm working on an app for podcast discovery (think Youtube for podcasts; I've put the twitter link in my HN profile page if anyone is interested in updates) and I've been thinking about this. I am generally anti-curation in my current approach. In this context I don't like the concept of people asking you to upvote something on Product Hunt, or leave a review on iTunes so it filters to the top of the list and gets some editor's attention, etc. I kinda prefer the Netflix or Youtube approach that tries to figure out what you'd like based on your viewing history, or items related to what you're viewing, or what other people who like what you're viewing have also liked... (Related, I also don't like the idea of 'categories' where you have to select whether an item is 'History' or 'Politics'. I'd like my software to be able to group items intelligently without manual intervention...) So in a way, with something like the way Netflix replaced explicit star-ratings based recommendations with viewing-history based recommendations, we are still looking for curation but in a subtler, more algorithmic way... same with Google PageRank and Facebook Newsfeed for that matter. Maybe algorithmic filtering is just curation at scale.
  - Broken_Hippo 10y ago
    I wish you luck. I prefer these sorts of curations much better than piecing together things based on reviews. I haven't had Netflix for years: Youtube can be fairly pleasant, until I click on something that "poisons" the feed. Surprisingly, Spotify does fairly well on recommendations for me, considering I have eclectic and sometimes unpopular listening habits. I'd consider the app you are working on :)
    - dredmorbius 10y ago
      On YouTube: 1. Deleting from history any "poison" content -- generally, stupid shit -- "that corner of YouTube" stuff, helps. 2. I've largely abandoned logged-in use of YouTube for numerous reasons. Its app is all but unusable due to pre-roll ads. (Tasteful banners I can stomach.). Routinely purging all history, and/or resetting device ID helps. 3. Purposeful and directed use of YouTube, or, frankly, any other service, is best. Search for what you're interested in and read/view that. Much of my use is autodidactic and informational. There's actually a wealth of information online now, if you can filter out the crud. 4. Being able to blacklist the stupid shit channels would help a hell of a lot. If I could personally banish to the depths of Hades any channel that posted stupid crap, things would be markedly improved.
      - Broken_Hippo 10y ago
        I do some of these from time to time, and weirdly, the 'back' button on the browser helps temporarily. #3 is my usual mode for initial watching, but I'm prone to go with suggestions from that when I'm watching. Sometimes, even with best intentions, it gets messy. Number 4, that would be most excellent. :D
        dredmorbius 10y ago
        If you're on desktop, installing Stylish and removing YouTube's comments and Suggestions panes is a huge win. You'll still see suggestions in-video, but those are slightly less annoying. (I've also noticed that, well fed and pruned, YouTube's suggestions actually tend toward useful these days, though that's with exceptionally aggressive pruning.) Yet another option is to use youtube-dl to download and play content locally. I do this pretty much exclusively on desktop, and find it quite helpful.
- personlurking 10y ago
  I wonder what value such curation has in today's terms, even if it were for a niche category. Plus, how does a modern 'surfer' know what they don't know (a site that should be included but they haven't come across it yet)? Also, I wouldn't imagine there to be any sort of IP to such an idea, as the sites listed belong to other people/companies.
  - wsterling 10y ago
    "Plus, how does a modern 'surfer' know what they don't know (a site that should be included but they haven't come across it yet)?" Yahoo had a form that allowed you to submit your site to be added to the directory. You included the URL and categories you would like to be listed in. It would then be evaluated and added by a Yahoo employee. It was one of the first things I would do after creating a new web site in the early 90s.
- holdenc 10y ago
  That Yahoo took so long to give up the DMOZ directory-style mentality is just another example of technologists favoring a neatly ordered solution over an organic one. In the crowd-sourced organic model, there is less curation, and more a time-based record of fleeting curiosity.
  - dredmorbius 10y ago
    Curation worked, early, in that there was relatively little online content. As late as 1998, the quality content online was perhaps 10,000 books: Despite the hype, the Web just isn't all that impressive as an information resource. The static, publicly accessible HTML text on the Web is roughly equivalent in size to a million books. The UC Berkeley Library has 8 million volumes, and the average quality of the Berkeley library content is much, much higher! If 10% of the material on the Web is ``useful,'' then that means there are about 100,000 useful book-equivalents on the Web, which is the size of good public library. The actual figure for ``useful'' is probably more like 1%, which is 10,000 books, or half the size of an average mall bookstore. http://people.ischool.berkeley.edu/~hal/Papers/japan/index.html http://people.ischool.berkeley.edu/~hal/Papers/japan/index.h... I'm leaning strongly toward curation via tagging, using standard ontologies, and vetted through designated / qualified experts for top content. A mix of crowdsourcing and expertise seems most viable. Quasi-related -- a rant on Pocket suggesting improved "tags" functionality (I swear there's another, better, though I cannot find it in the usual places). https://ello.co/dredmorbius/post/3eucmp_s0tumjuoxmfclbw https://ello.co/dredmorbius/post/3eucmp_s0tumjuoxmfclbw
    - naveen99 10y ago
      Maybe, but the web lets you ask questions for which there is no answer yet. But you might get one tomorrow once you ask. Also youtube is an amazing resource that books can't compete with. Tags and ontologies also can't compete with search, Bigquery and data lakes.
      - dredmorbius 10y ago
        Read Varian's paper (above), it anticipates a great deal, especially his commentary on Gresham's Law and content. There's also one from a couple of Stanford researchers a few years back which suggest a similar problem: Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is "The Effect of Cellular Phone Use Upon Driver Attention", a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving.... It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers. http://goo.gl/CTwEmw http://goo.gl/CTwEmw Including in my tagging schemas for Pocket are quality indicators. I've been using an arbitrarily selected ranking scale of "a-1, a-2, ... a-5" to indicate quality (higher is better), and also tagging items with keywords indicating negative attributes (usually "bullshit", though others apply). Being able to group classifications by reputation of author or publisher is also a key element. While there are hazards in argument from authority, there are frequently benefits both to recognising expertise and identifying non-credible or impeached sources. If a source routinely pursues some objective rather than the truth, you might tend to question its value. PageRank essentially relies on the implicit reputation vetting of links, using link sources as a basis for quality assessment. It's useful but also limited. And as noted, the bias toward advertising and specific gain (see also Celine's 2nd Law, from Robert Anton Wilson's "Illuminatus" trilogy) raises issues. I've been doing significant amounts of research, much of it online, but much also tracking down published book and article sources (most of those through illicit download sites -- really the only way the information can be obtained reasonably), and what I'm finding is that the noise associated with online data is just boggling. Books, it turns out, compete exceptionally well with YouTube. Ultimately they compliment one another. Sometimes I want to read, sometimes I want a lecture or demo. Both have their place. Excessive data ultimately isn't illuminating. Yes, online search can be useful at finding out, say, what's happening right here, now (transit applications are a huge one), or within a given social circle, but both of those are domains in which an exclusion scope is set which rules out virtually all available results -- you're looking for information about a specific location and time, or concerning a specific group of people. When it comes to general information, "reliable sources" dwindle. If you're looking for some established authority or text, you're left with the specific sources. If it's news, typically you're going to want some vetted news source -- and yes, that includes informally vetted sources on Twitter or YouTube. One of the most annoying things about both sources are the crap leaches who instantly show up to clog any trending keyword topic with crap content. That shit's got to get stomped hard. For new and developing information, ways to broadly pre-filter and bubble up quality content are an interesting problem and one I'm actively working on. Community rather than broadcast methods strike me as more viable.
- avar 10y ago
  The right answer to this is Google: You can search for any combination of things you want from any time. No need for a centrally maintained directory.
  - dkarapetyan 10y ago
    Google is not the right answer. Far from it. Lack of imagination is what makes it the de-facto standard. Why do I need a centralized body to do filtering and curation? Why can't it be a collaborative effort augmented with some machine learning? People should be in charge of their own information bubbles.
    - avar 10y ago
      I don't mean that Google literally 100% replaces that use-case, but that the answer is that search engines got good enough that directory curation was no longer marketable. So nowadays what you'd previously look up in something like dmoz you now just search for in Google.
  - DanBC 10y ago
    Google is great. It's also given us a billion shitty SEO optimized sites that hit keywords but provide little value.
  - Spooky23 10y ago
    Google was magic in 2001. Today, it's impossible to find many categories of information. That's why stack exchange exists. When I'm doing market research or looking for certain types of information, being aware of forums like HN is more productive. Long term the movement of everything behind walled gardens like Facebook will make it worse.
- city41 10y ago
  Delicious in its heyday would have qualified as a crowd sourced directory.
  - agumonkey 10y ago
    I kinda abuse reddit as a vetted public bookmark service.
- cvs268 10y ago
  Well, there"s Wikipedia...
more_corn 10y ago
"I don't know why the company exists."
nxzero 10y ago
Google still uses huge amounts of human curation, it's just harder to see; examples include: reCAPTCHA, Google Quality Raters, Google Maps Editors, etc.
- frik 10y ago
  Right. Google has employed a lot of human curators. Though it's little known. Some articles covered it over the years but with little details.
joezydeco 10y ago
No mention of Scott Yanoff?
apathy 10y ago
Xoom.com was once a site where, in exchange for free web space (kind of like Geocities), you gave the company permission to spam you. "We Know You'll Love This!" was pretty much our standard line, but we were profitable, in an age where such a thing was practically unheard of. So... We used to have "Porn Whackers"* at Xoom who would go through user sites that had been reported/complained about/whatever and decide whether the shady stuff that people put on their free pages was 1) kiddy porn (freeze the account, send directly to the FBI) 2) legal, but simply in bad taste (do nothing) 3) copyrighted software, PDFs, that sort of thing (delete) 4) copyrighted porn (initially delete, but see below) At some point it dawned on us that we were throwing away money by simply deleting the sites in category 4. I wrote a redirect and we cut a deal with Playboy to display ads for their website instead of a regular 404 page when a visitor landed on a nabbed-for-copyright-infringement freebie site. Needless to say, the pornwhackers were awesome to drink with. When you stare into the abyss for 8 hours a day, you develop some excellent coping mechanisms. Either that or you found something else to do. Plus there's that magical feeling of being one of the first humans ever to see a guy's tied-up nuts being stomped on by a crazy woman in high heels. We never did figure out whether that was a nasty domestic dispute or some sort of fetish thingy. Eventually Xoom got bought by GE, who had the bright idea (pause not) to make the spam emails "opt-in". Way to take a moneymaking idea and turn it into a turd, guys. Eventually they realized that they'd killed the goose that laid the sort-of-golden eggs and we all sought other employment. I miss SF from back then. My apartment in the Mission was $650/month and the 21 Club was still the best dive on Earth. The Excite guys were especially hysterical, they'd show up to the colocation facility so zonked on heroin that servers which were literally on fire gave them pause in terms of how to deal with the situation. Plus the amount of pretty girls around was obscene. Combined with all the free booze, it wasn't a half bad scene. Ah well, nothing good ever lasts. * Yes, the obvious thing occurred to us. Nobody cared. * * There may have been a 5th category, "legal and in good taste", but out of 17 million users and 20TB of sites, I never recalled seeing one that fit this description. There's a reason we talked about categorizing Porn & Warez.
- mistersquid 10y ago
  (Apologies, in advance, for so long a quotation.) Your comment that the "the pornwhackers were awesome to drink with. When you stare into the abyss for 8 hours a day, you develop some excellent coping mechanisms" reminded me of an amazing article published on The Verge and written by Catherine Buni and Soraya Chemaly. [0] (The article was featured here on HN 91 days ago.[1]) The article examines the psychological cost of content moderation and reading it helped me appreciate even more the difficult work content moderators perform to make the web a better place. (Hats off to dang, sctb and others whose user handles I don't know.) Anyway, here's a teaser from "The Secret Rules of the Internet" in case you missed it the first time it came around. > Mora-Blanco sat next to Misty Ewing-Davis, who, having been > on the job a few months, counted as an old hand. On the > table before them was a single piece of paper, folded in > half to show a bullet-point list of instructions: Remove > videos of animal abuse. Remove videos showing blood. Remove > visible nudity. Remove pornography. Mora-Blanco recalls her > teammates were a "mish-mash" of men and women; gay and > straight; slightly tipped toward white, but also Indian, > African-American, and Filipino. Most of them were friends, > friends of friends, or family. They talked and made jokes, > trying to make sense of the rules. "You have to find humor," > she remembers. "Otherwise it’s just painful." > > Videos arrived on their screens in a never-ending queue. > After watching a couple seconds apiece, SQUAD members > clicked one of four buttons that appeared in the upper right > hand corner of their screens: "Approve" — let the video > stand; "Racy" — mark video as 18-plus; "Reject" — remove > video without penalty; "Strike" — remove video with a > penalty to the account. Click, click, click. But that day > Mora-Blanco came across something that stopped her in her > tracks. > > "Oh, God," she said. > > Mora-Blanco won’t describe what she saw that morning. For > everyone’s sake, she says, she won’t conjure the > staggeringly violent images which, she recalls, involved a > toddler and a dimly lit hotel room. [0] http://www.theverge.com/2016/4/13/11387934/internet-moderator-history-youtube-facebook-reddit-censorship-free-speech http://www.theverge.com/2016/4/13/11387934/internet-moderato... [1] https://news.ycombinator.com/item?id=11500234 https://news.ycombinator.com/item?id=11500234 EDIT: formatting
greggman 10y ago
I remember the movie Frequency (2000) about a radio that transmits through time. Somewhere in the movie a character from the future tells a character in the past to "buy Yahoo". https://en.wikipedia.org/wiki/Frequency_(film) https://en.wikipedia.org/wiki/Frequency_(film)
- guessbest 10y ago
  That movie was made before the dot-com collapse. Also, it could have been product placement.
discardorama 10y ago
> Today, with more than one billion websites across the globe Huh, what? There are estimated to be more than a trillion sites out there, from what I've read.
- chubot 10y ago
  Nah, there are on the order of magnitude of a trillion web pages (though it's hard to count given dynamic pages and such), but a billion domains (which also has many dupes). The number of domains is definitely closer to a billion than a trillion. This number can be verified pretty easily (unlike the number of web pages).
frik 10y ago
"Surfing the web" used to be a common term one to two decades ago. https://www.washingtonpost.com/news/the-intersect/wp/2015/01/30/when-did-we-stop-surfing-the-web/ https://www.washingtonpost.com/news/the-intersect/wp/2015/01... http://www.dictionary.com/browse/surfing-the-web http://www.dictionary.com/browse/surfing-the-web
- drauh 10y ago
  It started with Gopher, IIRC: "Surf the Internet." I recall having a GIF of the gopher mascot with those words.[0] It was sometime between 1992 and 1995; I'm fairly sure it was before Netscape. The Jargon File[1] doesn't mention Gopher, specifically, but 1995 is just after Netscape was released. [0] This PDF of an article by P. Frana (p.31) is the only reference I can find to this image[2] I clearly remember. https://instructure-uploads.s3.amazonaws.com/account_17700000000000001/attachments/869722/Frana%202004%20Before%20the%20Web%20There%20Was%20Gopher.pdf?AWSAccessKeyId=AKIAJFNFXH2V2O7RPCAA&Expires=1468822467&Signature=velQlCiqpRa6IpXii5Z5sQbJON4%3D&response-content-disposition=attachment%3B%20filename%3D%22Frana%202004%20Before%20the%20Web%20There%20Was%20Gopher.pdf%22%3B%20filename%2A%3DUTF-8%27%27Frana%25202004%2520Before%2520the%2520Web%2520There%2520Was%2520Gopher.pdf https://instructure-uploads.s3.amazonaws.com/account_1770000... [1] http://catb.org/~esr/jargon/html/S/surf.html http://catb.org/~esr/jargon/html/S/surf.html [2] https://imgur.com/EHYdgMG https://imgur.com/EHYdgMG

103 points by miraj 10y ago

gkop 10y ago

Where does human curation of the web live now? Is it all temporal (Reddit, Facebook, Twitter, HN, etc), or is there somewhere central that humans organize it "for all time"? Would a modern Yahoo with the staff surfers replaced by the crowd provide value to the world? Could it possibly work without being horribly gamed? Are there good examples of having tried this idea?

pavlov 10y ago
Pinterest is more curation-oriented than the other social networks... And Wikipedia of course functions as a sort of directory for many topics.
empath75 10y ago
Dmoz still exists and is still run by AOL.
- dredmorbius 10y ago
  Indeed it does. I'd forgotten it was AOL that ended up with it though, that's rather discordant.
firasd 10y ago
Interesting to ponder. I'm working on an app for podcast discovery (think Youtube for podcasts; I've put the twitter link in my HN profile page if anyone is interested in updates) and I've been thinking about this. I am generally anti-curation in my current approach. In this context I don't like the concept of people asking you to upvote something on Product Hunt, or leave a review on iTunes so it filters to the top of the list and gets some editor's attention, etc. I kinda prefer the Netflix or Youtube approach that tries to figure out what you'd like based on your viewing history, or items related to what you're viewing, or what other people who like what you're viewing have also liked... (Related, I also don't like the idea of 'categories' where you have to select whether an item is 'History' or 'Politics'. I'd like my software to be able to group items intelligently without manual intervention...) So in a way, with something like the way Netflix replaced explicit star-ratings based recommendations with viewing-history based recommendations, we are still looking for curation but in a subtler, more algorithmic way... same with Google PageRank and Facebook Newsfeed for that matter. Maybe algorithmic filtering is just curation at scale.
- Broken_Hippo 10y ago
  I wish you luck. I prefer these sorts of curations much better than piecing together things based on reviews. I haven't had Netflix for years: Youtube can be fairly pleasant, until I click on something that "poisons" the feed. Surprisingly, Spotify does fairly well on recommendations for me, considering I have eclectic and sometimes unpopular listening habits. I'd consider the app you are working on :)
  - dredmorbius 10y ago
    On YouTube: 1. Deleting from history any "poison" content -- generally, stupid shit -- "that corner of YouTube" stuff, helps. 2. I've largely abandoned logged-in use of YouTube for numerous reasons. Its app is all but unusable due to pre-roll ads. (Tasteful banners I can stomach.). Routinely purging all history, and/or resetting device ID helps. 3. Purposeful and directed use of YouTube, or, frankly, any other service, is best. Search for what you're interested in and read/view that. Much of my use is autodidactic and informational. There's actually a wealth of information online now, if you can filter out the crud. 4. Being able to blacklist the stupid shit channels would help a hell of a lot. If I could personally banish to the depths of Hades any channel that posted stupid crap, things would be markedly improved.
    - Broken_Hippo 10y ago
      I do some of these from time to time, and weirdly, the 'back' button on the browser helps temporarily. #3 is my usual mode for initial watching, but I'm prone to go with suggestions from that when I'm watching. Sometimes, even with best intentions, it gets messy. Number 4, that would be most excellent. :D
      - dredmorbius 10y ago
        If you're on desktop, installing Stylish and removing YouTube's comments and Suggestions panes is a huge win. You'll still see suggestions in-video, but those are slightly less annoying. (I've also noticed that, well fed and pruned, YouTube's suggestions actually tend toward useful these days, though that's with exceptionally aggressive pruning.) Yet another option is to use youtube-dl to download and play content locally. I do this pretty much exclusively on desktop, and find it quite helpful.
personlurking 10y ago
I wonder what value such curation has in today's terms, even if it were for a niche category. Plus, how does a modern 'surfer' know what they don't know (a site that should be included but they haven't come across it yet)? Also, I wouldn't imagine there to be any sort of IP to such an idea, as the sites listed belong to other people/companies.
- wsterling 10y ago
  "Plus, how does a modern 'surfer' know what they don't know (a site that should be included but they haven't come across it yet)?" Yahoo had a form that allowed you to submit your site to be added to the directory. You included the URL and categories you would like to be listed in. It would then be evaluated and added by a Yahoo employee. It was one of the first things I would do after creating a new web site in the early 90s.
holdenc 10y ago
That Yahoo took so long to give up the DMOZ directory-style mentality is just another example of technologists favoring a neatly ordered solution over an organic one. In the crowd-sourced organic model, there is less curation, and more a time-based record of fleeting curiosity.
- dredmorbius 10y ago
  Curation worked, early, in that there was relatively little online content. As late as 1998, the quality content online was perhaps 10,000 books: Despite the hype, the Web just isn't all that impressive as an information resource. The static, publicly accessible HTML text on the Web is roughly equivalent in size to a million books. The UC Berkeley Library has 8 million volumes, and the average quality of the Berkeley library content is much, much higher! If 10% of the material on the Web is ``useful,'' then that means there are about 100,000 useful book-equivalents on the Web, which is the size of good public library. The actual figure for ``useful'' is probably more like 1%, which is 10,000 books, or half the size of an average mall bookstore. http://people.ischool.berkeley.edu/~hal/Papers/japan/index.html http://people.ischool.berkeley.edu/~hal/Papers/japan/index.h... I'm leaning strongly toward curation via tagging, using standard ontologies, and vetted through designated / qualified experts for top content. A mix of crowdsourcing and expertise seems most viable. Quasi-related -- a rant on Pocket suggesting improved "tags" functionality (I swear there's another, better, though I cannot find it in the usual places). https://ello.co/dredmorbius/post/3eucmp_s0tumjuoxmfclbw https://ello.co/dredmorbius/post/3eucmp_s0tumjuoxmfclbw
  - naveen99 10y ago
    Maybe, but the web lets you ask questions for which there is no answer yet. But you might get one tomorrow once you ask. Also youtube is an amazing resource that books can't compete with. Tags and ontologies also can't compete with search, Bigquery and data lakes.
    - dredmorbius 10y ago
      Read Varian's paper (above), it anticipates a great deal, especially his commentary on Gresham's Law and content. There's also one from a couple of Stanford researchers a few years back which suggest a similar problem: Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is "The Effect of Cellular Phone Use Upon Driver Attention", a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving.... It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers. http://goo.gl/CTwEmw http://goo.gl/CTwEmw Including in my tagging schemas for Pocket are quality indicators. I've been using an arbitrarily selected ranking scale of "a-1, a-2, ... a-5" to indicate quality (higher is better), and also tagging items with keywords indicating negative attributes (usually "bullshit", though others apply). Being able to group classifications by reputation of author or publisher is also a key element. While there are hazards in argument from authority, there are frequently benefits both to recognising expertise and identifying non-credible or impeached sources. If a source routinely pursues some objective rather than the truth, you might tend to question its value. PageRank essentially relies on the implicit reputation vetting of links, using link sources as a basis for quality assessment. It's useful but also limited. And as noted, the bias toward advertising and specific gain (see also Celine's 2nd Law, from Robert Anton Wilson's "Illuminatus" trilogy) raises issues. I've been doing significant amounts of research, much of it online, but much also tracking down published book and article sources (most of those through illicit download sites -- really the only way the information can be obtained reasonably), and what I'm finding is that the noise associated with online data is just boggling. Books, it turns out, compete exceptionally well with YouTube. Ultimately they compliment one another. Sometimes I want to read, sometimes I want a lecture or demo. Both have their place. Excessive data ultimately isn't illuminating. Yes, online search can be useful at finding out, say, what's happening right here, now (transit applications are a huge one), or within a given social circle, but both of those are domains in which an exclusion scope is set which rules out virtually all available results -- you're looking for information about a specific location and time, or concerning a specific group of people. When it comes to general information, "reliable sources" dwindle. If you're looking for some established authority or text, you're left with the specific sources. If it's news, typically you're going to want some vetted news source -- and yes, that includes informally vetted sources on Twitter or YouTube. One of the most annoying things about both sources are the crap leaches who instantly show up to clog any trending keyword topic with crap content. That shit's got to get stomped hard. For new and developing information, ways to broadly pre-filter and bubble up quality content are an interesting problem and one I'm actively working on. Community rather than broadcast methods strike me as more viable.
avar 10y ago
The right answer to this is Google: You can search for any combination of things you want from any time. No need for a centrally maintained directory.
- dkarapetyan 10y ago
  Google is not the right answer. Far from it. Lack of imagination is what makes it the de-facto standard. Why do I need a centralized body to do filtering and curation? Why can't it be a collaborative effort augmented with some machine learning? People should be in charge of their own information bubbles.
  - avar 10y ago
    I don't mean that Google literally 100% replaces that use-case, but that the answer is that search engines got good enough that directory curation was no longer marketable. So nowadays what you'd previously look up in something like dmoz you now just search for in Google.
- DanBC 10y ago
  Google is great. It's also given us a billion shitty SEO optimized sites that hit keywords but provide little value.
- Spooky23 10y ago
  Google was magic in 2001. Today, it's impossible to find many categories of information. That's why stack exchange exists. When I'm doing market research or looking for certain types of information, being aware of forums like HN is more productive. Long term the movement of everything behind walled gardens like Facebook will make it worse.
city41 10y ago
Delicious in its heyday would have qualified as a crowd sourced directory.
- agumonkey 10y ago
  I kinda abuse reddit as a vetted public bookmark service.
cvs268 10y ago
Well, there"s Wikipedia...

more_corn 10y ago

"I don't know why the company exists."

nxzero 10y ago

Google still uses huge amounts of human curation, it's just harder to see; examples include: reCAPTCHA, Google Quality Raters, Google Maps Editors, etc.

frik 10y ago
Right. Google has employed a lot of human curators. Though it's little known. Some articles covered it over the years but with little details.

joezydeco 10y ago

No mention of Scott Yanoff?

apathy 10y ago

Xoom.com was once a site where, in exchange for free web space (kind of like Geocities), you gave the company permission to spam you. "We Know You'll Love This!" was pretty much our standard line, but we were profitable, in an age where such a thing was practically unheard of. So... We used to have "Porn Whackers"* at Xoom who would go through user sites that had been reported/complained about/whatever and decide whether the shady stuff that people put on their free pages was 1) kiddy porn (freeze the account, send directly to the FBI) 2) legal, but simply in bad taste (do nothing) 3) copyrighted software, PDFs, that sort of thing (delete) 4) copyrighted porn (initially delete, but see below) At some point it dawned on us that we were throwing away money by simply deleting the sites in category 4. I wrote a redirect and we cut a deal with Playboy to display ads for their website instead of a regular 404 page when a visitor landed on a nabbed-for-copyright-infringement freebie site. Needless to say, the pornwhackers were awesome to drink with. When you stare into the abyss for 8 hours a day, you develop some excellent coping mechanisms. Either that or you found something else to do. Plus there's that magical feeling of being one of the first humans ever to see a guy's tied-up nuts being stomped on by a crazy woman in high heels. We never did figure out whether that was a nasty domestic dispute or some sort of fetish thingy. Eventually Xoom got bought by GE, who had the bright idea (pause not) to make the spam emails "opt-in". Way to take a moneymaking idea and turn it into a turd, guys. Eventually they realized that they'd killed the goose that laid the sort-of-golden eggs and we all sought other employment. I miss SF from back then. My apartment in the Mission was $650/month and the 21 Club was still the best dive on Earth. The Excite guys were especially hysterical, they'd show up to the colocation facility so zonked on heroin that servers which were literally on fire gave them pause in terms of how to deal with the situation. Plus the amount of pretty girls around was obscene. Combined with all the free booze, it wasn't a half bad scene. Ah well, nothing good ever lasts. * Yes, the obvious thing occurred to us. Nobody cared. * * There may have been a 5th category, "legal and in good taste", but out of 17 million users and 20TB of sites, I never recalled seeing one that fit this description. There's a reason we talked about categorizing Porn & Warez.

mistersquid 10y ago
(Apologies, in advance, for so long a quotation.) Your comment that the "the pornwhackers were awesome to drink with. When you stare into the abyss for 8 hours a day, you develop some excellent coping mechanisms" reminded me of an amazing article published on The Verge and written by Catherine Buni and Soraya Chemaly. [0] (The article was featured here on HN 91 days ago.[1]) The article examines the psychological cost of content moderation and reading it helped me appreciate even more the difficult work content moderators perform to make the web a better place. (Hats off to dang, sctb and others whose user handles I don't know.) Anyway, here's a teaser from "The Secret Rules of the Internet" in case you missed it the first time it came around. > Mora-Blanco sat next to Misty Ewing-Davis, who, having been > on the job a few months, counted as an old hand. On the > table before them was a single piece of paper, folded in > half to show a bullet-point list of instructions: Remove > videos of animal abuse. Remove videos showing blood. Remove > visible nudity. Remove pornography. Mora-Blanco recalls her > teammates were a "mish-mash" of men and women; gay and > straight; slightly tipped toward white, but also Indian, > African-American, and Filipino. Most of them were friends, > friends of friends, or family. They talked and made jokes, > trying to make sense of the rules. "You have to find humor," > she remembers. "Otherwise it’s just painful." > > Videos arrived on their screens in a never-ending queue. > After watching a couple seconds apiece, SQUAD members > clicked one of four buttons that appeared in the upper right > hand corner of their screens: "Approve" — let the video > stand; "Racy" — mark video as 18-plus; "Reject" — remove > video without penalty; "Strike" — remove video with a > penalty to the account. Click, click, click. But that day > Mora-Blanco came across something that stopped her in her > tracks. > > "Oh, God," she said. > > Mora-Blanco won’t describe what she saw that morning. For > everyone’s sake, she says, she won’t conjure the > staggeringly violent images which, she recalls, involved a > toddler and a dimly lit hotel room. [0] http://www.theverge.com/2016/4/13/11387934/internet-moderator-history-youtube-facebook-reddit-censorship-free-speech http://www.theverge.com/2016/4/13/11387934/internet-moderato... [1] https://news.ycombinator.com/item?id=11500234 https://news.ycombinator.com/item?id=11500234 EDIT: formatting

greggman 10y ago

I remember the movie Frequency (2000) about a radio that transmits through time. Somewhere in the movie a character from the future tells a character in the past to "buy Yahoo". https://en.wikipedia.org/wiki/Frequency_(film) https://en.wikipedia.org/wiki/Frequency_(film)

guessbest 10y ago
That movie was made before the dot-com collapse. Also, it could have been product placement.

discardorama 10y ago

> Today, with more than one billion websites across the globe Huh, what? There are estimated to be more than a trillion sites out there, from what I've read.

chubot 10y ago
Nah, there are on the order of magnitude of a trillion web pages (though it's hard to count given dynamic pages and such), but a billion domains (which also has many dupes). The number of domains is definitely closer to a billion than a trillion. This number can be verified pretty easily (unlike the number of web pages).

frik 10y ago

"Surfing the web" used to be a common term one to two decades ago. https://www.washingtonpost.com/news/the-intersect/wp/2015/01/30/when-did-we-stop-surfing-the-web/ https://www.washingtonpost.com/news/the-intersect/wp/2015/01... http://www.dictionary.com/browse/surfing-the-web http://www.dictionary.com/browse/surfing-the-web

drauh 10y ago
It started with Gopher, IIRC: "Surf the Internet." I recall having a GIF of the gopher mascot with those words.[0] It was sometime between 1992 and 1995; I'm fairly sure it was before Netscape. The Jargon File[1] doesn't mention Gopher, specifically, but 1995 is just after Netscape was released. [0] This PDF of an article by P. Frana (p.31) is the only reference I can find to this image[2] I clearly remember. https://instructure-uploads.s3.amazonaws.com/account_17700000000000001/attachments/869722/Frana%202004%20Before%20the%20Web%20There%20Was%20Gopher.pdf?AWSAccessKeyId=AKIAJFNFXH2V2O7RPCAA&Expires=1468822467&Signature=velQlCiqpRa6IpXii5Z5sQbJON4%3D&response-content-disposition=attachment%3B%20filename%3D%22Frana%202004%20Before%20the%20Web%20There%20Was%20Gopher.pdf%22%3B%20filename%2A%3DUTF-8%27%27Frana%25202004%2520Before%2520the%2520Web%2520There%2520Was%2520Gopher.pdf https://instructure-uploads.s3.amazonaws.com/account_1770000... [1] http://catb.org/~esr/jargon/html/S/surf.html http://catb.org/~esr/jargon/html/S/surf.html [2] https://imgur.com/EHYdgMG https://imgur.com/EHYdgMG