12 ms·

Why We Desperately Need a New (and Better) Google

222 points by vyrotek 16y ago

petervandijck 16y ago
The argument being that Google is loosing the war against spam. A new and better Google will likely be Google itself. What we really need is a way to discover content that's not search.
- meterplech 16y ago
  For many things there is utility to social content finding. Sites like HN work for news/discussion, and Twitter and Facebook work for a random amalgam of things. I'm interested in the future of social search startups that somehow curate content from friends. In the articles example- asking friends if they liked their Dishwasher, and if yes what brand it is. That's the most like how people IRL make these decisions. I know there are some startups in this space as well, hope they do well!
  - nhangen 16y ago
    Furthermore, I'd love to see a simple thumbs up or thumbs down system integrated into the browser or Google search. Did this page help you find what you were looking for? Was this page useful? etc
    - ehsanul 16y ago
      The problem with that, is again, spam. How do you differentiate fake votes from real ones? You can do per-IP limiting, and probably other things I'm not aware of (please comment about them), but it's still pretty open to abuse I'd say.
      - sukuriant 16y ago
        Open to exploitation theoretically, or open to exploitation practically? Can we really know this until we try? I think something like this would be a cool thing for Google to test for a year (a voting system might take a while to set up properly)
      - spidaman 16y ago
        Google has proven to be very effective data collectors (see "What they know" stuff from WSJ e.g. http://online.wsj.com/article/SB10001424052748703309704575413553851854026.html http://online.wsj.com/article/SB1000142405274870330970457541...) - it may on the one hand be surprising they don't use this data more effectively to identify and thwart spam. OTOH, they're making a lot of money on spammy SERPs.
      - nhangen 16y ago
        I suppose you do it similarly to the way Amazon or iTunes implements it for reviews...you make sure the user is logged in via a Google account. Sure, it could be exploited, and I'm guessing that's why they haven't implemented it, but there's got to be a solution that would make it work.
    - itg 16y ago
      That can be easily gamed. We already see this with online marketing/pr companies that try to game other sites such as digg/reddit/etc.
    - jbrennan 16y ago
      I'd even be OK with "Thumbs Up/Down this result just for your own personal book-keeping", so that you can promote/demote results in your own search history (try searching for "_____ tutorial photoshop" and you'll get tons of splog links before you eventually hit a golden result. It'd be nice to save time later). Google briefly had a feature like this in their results, but they've removed it. All that's left is the Star system, which isn't quite the same.
    - cmurphycode 16y ago
      Google did try this. I don't think it ever left the lab, but I used it for a while. I think they may have canned it; I just realized I haven't seen it in a while. It was called searchwiki. I think it was originally just supposed to apply to your own results, but I'm sure they analyzed the tendencies to see if it would've helful to use overall. Honestly, such a system would be ridiculously easy to game with a botnet, so there needs to be significant work in the area beforehand.
      - rhizome 16y ago
        I think the 'X" was live for maybe a month or two. I figured either or both that there was a resource/algorithm problem that prevented them from continuing in a fruitful way, or that it was a bit of user survey they could use in aggregate as preference weighting on the indexes/results. That is, not as a function of personal filtering, but to see if there were commonalities in what people considered to be bad sites. This may be further supported by the fact that the lifetime of the feature may have been too short for gamers to exploit it.
  - ktsmith 16y ago
    Having just bought a new washer and dryer I can honestly say that my friends opinions were not a factor in the decision at all even though we did speak about it before I made the purchase. The main reason being other than some superficial elements of the appliances my friends can't really speak intelligently about their own appliances. They tend to not remember beyond the initial purchase point what many of the features even are. For example I asked a friend if his washer contained a heater and while it does he didn't even have a clue that it did. I have a relative working at Home Depot and we just discussed this topic at length given my own recent purchase. Most of their appliance customers come in with some idea of how much money they want to spend and what basic feature set they would like. Then they are looking for a knowledgeable sales associate to explain to them the differences and benefits of the various models. They may have "heard from a friend" that a particular model was good or a particular feature was good but even that information is usually incomplete or incorrect. I would fully expect "social" results for these types of queries to result in even more misinformation.
- binomial 16y ago
  Working on it - http://jellly.com/ http://jellly.com/ Also, just got a Firefox addon working that provides automatic related links to most web pages. Unfortunately, it relies on search now. But I'm sure that any sufficiently popular content-discovery tool will have a lot of spammers trying to game it. It's not easy to fight that.
  - javadi82 16y ago
    I am sorry; I do not understand how this is different from the "Similar" link that is placed beside the "Cached" link on Google's SERPs. Could you please elaborate?
- dawgr 16y ago
  How would it be possible to discover content without searching? If I want to discover content about something I have to tell the computer what that something is about, that would be searching. If you are talking about directories, then I wouldn't use that.
replicatorblog 16y ago
It will be interesting to see how this impacts the Android/iOS battle. Search revenue funds almost all of Google's other activities so if people start using other search engines or find alternate ways to get their content it could impact the level they can spend on phones. With a push to a mobile first world the Android model is especially sensitive to spam. On a full size browser you have a lot more context and results for a given search. 5 Results may be spam, but you can work around them. If the average phone screen shows 3-5 results and all of them are spam you will quickly find alternate tools. Google ignoring spam is like Microsoft ignoring the cloud.
- petervandijck 16y ago
  "Google ignoring spam is like Microsoft ignoring the cloud." Great line!
  - zachrose 16y ago
    That's the kind of line that I wish I could email to people in 1960 as an important business insight from 2011.
    - petervandijck 16y ago
      To them, it would sound like "Bla ignoring bla is like bla ignoring bla."
      - AngryParsley 16y ago
        They would parse the words, but not the correct meanings. The impression they'd come away with would be something like: (Extremely large number) ignoring canned meat is like (random company name) ignoring the high-altitude water vapor.
    - VMG 16y ago
      I just got mail from someone in 2060 saying "Scroomz is ignoring zork like Yooka ignored mulk"
klbarry 16y ago
Isn't the issue, of course, that spammers have no incentive to game other search engines since they're not worth the time? Any search engine that gets big will have the problem.
- bambax 16y ago
  Came here to say this but you said it before! ;-) Also, "crowd-sourced curated lists of websites" sound like the old Yahoo directory of yore. They will either become obsolete very quickly or spammers will find a way to penetrate and dominate them.
  - jerf 16y ago
    I believe the Yahoo lists were actually centralized. Hypothetically a fully-distributed approach could work better, in the style of Wikipedia. Realistically, well, I have no idea but it's worth a try.
    - ehsanul 16y ago
      I agree. One particular try failed [1], but that doesn't mean it can't work. [1] http://en.wikipedia.org/wiki/Wikia_Search http://en.wikipedia.org/wiki/Wikia_Search
      - dasil003 16y ago
        Not only that, but Yahoo didn't think search was important, so it's not even a real try of enhancing search with curation. The truth is, if some company comes up with a better search engine, whatever ideas behind it are not going to sound like an obvious win up front—if they did then Google would already be doing that. Instead they'll have to create a search engine that is better, but somehow antithetical to Google's business model so that they can't just copy it, because there's no way for a startup to come up with enough resources to stay materially ahead of Google in pure search. And of course that's only half the battle; then you have to be better enough that users can be bothered to switch (or a browser deal coup). Personally I haven't found the spam problem to be nearly as bad as the echo chamber makes out. I think silicon valley types just have a good imagination about how good it could be.
Matt_Cutts 16y ago
"Google does provide an option to search within a date range, but these are the dates when website was indexed rather than created; which means the results are practically useless." I believe the author is mistaken on this point. Quick proof is to do a search for [matt cutts] and you'll see the root page of my blog. Click "More search tools" on the left and click the "Past week" link. Now you'll only see pages created the last week, even though lots of pages on my site were indexed in the last week.
- amichail 16y ago
  BTW, why is it that Google has not opened up its core search engine to third party developers so that their code can be used to bring up some of the search results by default (without requiring the user to subscribe to third party features)? Most people have never used Google's Subscribed Links feature because it is not enabled by default. User feedback can be used to determine which third party code to use in which contexts. Spam/unhelpful features would be detected quickly. There would be intense competition among third party developers for highly desired features. Something like this could give you Wolfram Alpha like features among other things such as custom UIs for various searches (e.g., travel). The Google App Engine could be used for computation. You could pay third party developers by how often their code is used in search results.
  - Matt_Cutts 16y ago
    Just speaking for me personally, I'd be a fan of trying ideas like that. The potential danger would be that some code could introduce latency or other things that would make the search experience worse instead of better.
    - amichail 16y ago
      There are huge risks with this approach, but there's also a lot of potential. Search could look quite different. It would be an ideal place to experiment with and profit from novel search ideas. For example, third party developers may experiment with query-induced flash mobs where people who just performed a similar query could collaborate in real-time to find the information they need. Finding ways to prevent such an ecosystem from descending into absolute chaos would be a fascinating challenge.
DanielBMarkham 16y ago
This is exactly what blogger Paul Kedrosky found when trying to buy a dishwasher. He wrote about how he began Googleing for information…and Googleing…and Googleing. He couldn’t make head or tail of the results. Paul concluded that the “the entire web is spam when it comes to major appliance reviews”. So I happen to know somebody who is taking a small section of the home appliance market and creating content around it -- reviews, news, advice, a place for other consumers to talk to each other. Of course to do this you need to have income, so they are going to use some sort of ad-supported model. My question is very simple: is their project a spam site or not? To some, I guess it would qualify. To others, not. You see, there are two questions when it comes to search results: 1) Am I being presented results that match the query I entered? and 2) Am I being presented results that match what I want to know? These are two entirely different things. A third-grader looking for information on a movie star might find a games page with all sorts of information on that star -- all sponsored by some kind of adsensey stuff. And he's very happy. A researcher typing in the same question gets the same page? He's pissed. There is no universal answer for any one question. It's all dependent on the culture, education, and intent of the user -- all of which are not easily communicated to a search engine. Look -- this is a real problem. I hate it. Sucks to go to pages you don't like. All I'm saying is that it's more complicated than "we need a new Google" Finding what you want exactly when you want it is a difficult and non-trivial problem. We just got lucky in that Google found a simple algorithm that can be helpful in some situations. It may be that we're seeing the natural end of the usefulness of that algorithm. </soapbox>
- ktsmith 16y ago
  > is their project a spam site or not? To me that depends a LOT on how they present the advertisements and what they do on pages where they don't have information for a product. My biggest complaint with so called review sites is that they are presented more advertisements than content. They also tend to have automatically generated pages for every model number you can imagine, including incorrect ones that they receive searches for. On those pages there tends to be links to shopping sites and prices for unrelated appliances and products. I absolutely consider that to be spam because they are content free. The problem with google searches lately, and especially for things like appliances, is that the spammers and content mills are clearly winning. In my current search for a washer and dryer the manufacturers page was on page four or five or the results. The first few pages were flooded with bogus content pages, sale pages and unrelated pages. Trying to filter out shopping sites and explicitly target specific keywords and filter others doesn't help. There are a growing number of search topics for which google is simply broken.
mark_l_watson 16y ago
I just tried two test queries on blekko and google. Small sample, but there did seem to be less link-bate results on blekko. The issue is whether their results are close to being as up to date as google's results. I was interested that blekko seems to have done a lot with a modest amount of funding. Also, I wonder if they are getting some monetization with the association with Facebook.
meadhikari 16y ago
Professor, you could've proved your point by linking to at least one example of how Blekko found a founders work and listed it by date (as the task required), instead you have hashtags on health, finance, etc. The truth is that nobody has arranged that information in the way you want, if it existed at all, that venture database where you found the 500 companies would've been the natural place to look.. CrunchBase maybe?! Thought Worth mentioning
apollo 16y ago
This may be a bit of a tangent, but I want to see the results of the VC system survey.
oliverdamian 16y ago
How about a P2P search/bookmarking platform where peers could publish search/bookmarking histories ranked by like/dislike/spam votes which other peers can subscribe to. Publishing peers can also be ranked according to number of subscribers. Actually P2P curation could be the next level up from raw centralised search. Is there anything like this out there already?
buro9 16y ago
I love Google products, but I can't help but agree. I'm currently trying to find a colour laser printer that has good performance (quality vs speed) with a reasonable running cost over the life of the printer (at least a few years). All I'm getting is either the manufacturers slant (PR) or spam sites all harvesting the same reviews. To solve this I now look for vertical based search sites. In this case http://www.printershowcase.com/small-officecolorlaser.aspx http://www.printershowcase.com/small-officecolorlaser.aspx is the best I've found... but it's hardly to printers what dpreview is to cameras. I stick with Google because it largely works well, but when I know what I want to see and that it must exist but cannot find it... then I find myself looking elsewhere all the time. DDG and Blekko I use in these cases, but even they're not solving these kinds of needs.
- stephenbez 16y ago
  I trust Amazon for my product reviews.
  - buro9 16y ago
    Beyond books and music... I don't. Too many products without reviews. No control for the reviews. Example... the HP CP4025 appears to be good (I have a HP Z800 workstation so thought it was worth checking HP for the hell of it)... yet I haven't found a balanced and comparable review for it. I'd like to see it set against other small office printers and to see the average cost per print. It's on Amazon twice: http://www.amazon.co.uk/Color-LaserJet-Enterprise-CP4025n-1000Base-T/dp/B0036LAFMS/ref=sr_1_1?s=officeproduct&ie=UTF8&qid=1293915915&sr=1-1 http://www.amazon.co.uk/Color-LaserJet-Enterprise-CP4025n-10... http://www.amazon.co.uk/Color-LaserJet-Enterprise-CP4025dn-1000Base-T/dp/B0036L37U0/ref=sr_1_2?s=officeproduct&ie=UTF8&qid=1293915915&sr=1-2 http://www.amazon.co.uk/Color-LaserJet-Enterprise-CP4025dn-1... But no reviews. I'd like Google to take my search terms and help me find stuff... you know, that "sort the world's data" thing, to make sense of my terms and show me the results I'm looking for. Google isn't working for me. Amazon isn't working for me. It takes hours and hours of searching in circles to research even the most basic purchasing decision.
  - tedunangst 16y ago
cletus 16y ago
This issue in a roundabout kind of way touches on Facebook. The issue of social search has a lot of mindshare. Some think it is the future of search. I disagree. One of the things that made search successful anduseful early on was scale. Instead of having to go to the librar or ask your friends you can effectively canvas the connected world. I find the notion that friends' recommendations will replace that as nothing short of bizarre. It's like a huge step backwards. The argument is that you can filter out the garbage as your social graph will provide a level of curation. Let me give you a concrete example. If I wanted t buy a camera I'd stil need t go to dpreview and other sites. It's highly likely that my friends don't really know a lot about this (but some will have an opinion anyway). This same idea of human curation is behind such sites ad Mahalo and the garbage sites themselves to a degree. Of course at some point computers will be powerful enough to generate this garbage content. Blekko's idea of slash tags s interesting (to a degree) but if it's successful its easily reproducible. Google is still in the box seat here but of course that's no barrier to a link-baiting TC title. Personally I'm an optimist. I believe that, much like email spam, the garbage from AC, DM and others I'd a transitional problem (email spam is basically a solved problem now if you use a half-decent email provider). If they succeed we won't be able to find anything. I don't believe that'll happen so these services are therefore doomed. So betting on Demand Media is (to quote Tyler) like betting on the Mayans (meaning betting they're right about the world ending in 2012: it doesnt really matter if you're right). So my money is on Google being the better Google.
- richcollins 16y ago
  How does filtering based on a trust network imply that the network will only include your friends?
  - stcredzero 16y ago
    Indeed. For such filtering to work, a significant portion of your friends should be searching the entire network and not just filtering through friends. But this might just work out to an early-adopter/follower divide. Friend network filtering will work better for the followers. The early-adopters/leaders will continue to use other means of search and will feed the friend-network.
d4nt 16y ago
It's interesting that the way of "gaming" Google appears to be in having thousands of people generating SEO friendly content. I think Google's problem is that it's pushed SEO to the point where the definition of Spam depends either on a subjective view of what kind of site the user is looking for, or it's just mildly worse than something else that's out there (e.g. When I search for something coding related and get one of the stackoverflow scrapers). Where do we go from here? Well, I don't think the answer is just a radically new way of indexing/ranking websites. That might work in the short term but the spammers will soon catch up. The answer probably lies in a combination of better language interpretation, context sensitivity using browsing history and location, and user profiling based on the social graph and search history. All of which google seems to be working on.
JusticeJones 16y ago
Tell me, how exactly is writing a sensationalized article that targets one of the Internet's oldest and largest communities to get fed by CPM advertising any different than what they decry? People have said this time and time again, but they never seem to debut let alone promise any sort of technology to address the issue. They just leave that end of the deal up in the air. As if to say that it's o.k. to spin topics as long as they strike a social nerve, but those who're less graceful at the craft are undeserving of the benefits which they themselves reap. If the search giants had any balls they'd cut the "Internet Marketing" community off at the knees. Because the money making methods pushed by that community either don't work or are unsustainable, so they're entirely reliant on a steady stream of new recruits. If they want to promote gaming your system don't let them reap any benefits from it.
- eitland 16y ago
  What about a three strike approach like the one that was suggested a few days ago with AdSense? Domains frequently being excluded by power searchers could be good signal. (googling to find on that used to pester my search results, kods.net, it seems it has finally got banned. Hooray!)
kokon 16y ago
CMIIW, but is that the reason why Google acquired MetaWeb a few months ago? I'm expecting to see some improvement on that front.
tokenadult 16y ago
Let me see if I correctly understand the learned professor's article. In his view, the problem is that a user using a free search engine to find information will find a lot of information about people who want to sell products and services, gaining money by exerting their time and effort. What he hopes to obtain for free is email addresses of persons to whom he wants to send his survey, so that he can use their time and effort without compensating them to produce something of value to him. Exactly how is this a problem? People who actively like to be contacted by random persons surfing the Internet make their contact information readily available (and answer questions sent through those publicly visible contact channels). But to many other persons, not being readily visible on the Internet is a feature rather than a bug. (Disclaimer: my contact information is readily visible on the Internet, so readily visible that it has been used by point-of-view pushers on Wikipedia to give me harassing telephone calls.)
- tedunangst 16y ago
  I think he was actually trying to piece together the background stories for the people who didn't respond to his emails, not find their emails. I don't think the info he was searching for was particularly private either. For instance, trying to find out the company a CEO worked at before their current one. The problem is that content copiers will produce so many copies of the PR announcement for their current job, it's impossible to find the announcement for their previous job. I've tried doing this exact search and it's very frustrating.
  - narrator 16y ago
    I often find that things that really don't exist wind up with spam results. For instance, if you type in "free ipad" into google you will likely get thousands of search results, all spam because they don't actually exist. Similarly, contact and personal information for CEOs of major corporations does not exist online either and any search will turn up spam. One more example, to add to the many. If you get a genuine wrong number call from somebody who made a simple mistake and type their caller id into the internet, you'll just get a bunch of reverse phone lookup spam while if you search for a phone number of a know telemarketer or bill collector, you'll likely get a full dossier on that company.
jrussbowman 16y ago
One of the new things I am working on with unscatter.com is getting quicker access to reviews and blog posts using the blekko api. The next release will be a major change as I've dumped most of the current search providers in favor of blekko and have moved realtime search to it's own page with analysis by providing lists of links in the realtime feed. Nothing is released yet unfortunately. The site is officially a hobby for me write more but I hope to have the new stuff up in the next week or two. I may just hide the realtime stuff and get the blekko feeds up sooner rather than later. Now that I am focusing building the site to fit my needs getting up to date info about products and technology, the bulk of my personal searches, is the top priority. Have to admit the blekko api has helped. In the mean time I would suggest the slash tags /reviews and /blogs with /date on blekko would be very helpful if you are doing product searches. With unscatter I am really only providing shortcuts for the with additional ui tweaks. Disclaimer: I am in no way associated with blekko other than having been given permission to use their api for a personal project.
- jrussbowman 16y ago
  Sorry meant to say the site is a hobby for me right more. Wrote these comment with the swype keyboard on my phone so I apologize for any problems reading them
EGreg 16y ago
Hey, so what you are basically saying is, "the best computer algorithms in the world" (you know, Google has like > 578690 Ph. D's) are not good enough to have effective search, so we should introduce the human element. Fair enough. There is the Open Directory Project (which is pretty old) and of course there is Facebook, Twitter, and other, human-curated services. Starting a whole new company to do search and compete with Google (and Bing)? Seems like a waste of time as Google can just copy what you are doing and incorporate it into its already massive site (complete with traffic, audience, and lots of other goodies). Instead, why not get Google to add more social recommendation and feedback features?
ams6110 16y ago
Why would it be so difficult for Google to filter out spam sites? E.g. DuckDuckGo filters out eHow.com results, because they are low quality and tend to be spammy. Oh of course, it's not in Google's interest to do this, because they make money from the spam sites. So I don't expect Google to really "solve" this problem.... their trick is to stay useful enough that users don't abandon them, but allow enough spam into the search results to provide revenue. A tricky balance...
- gsmaverick 16y ago
  If Google were to attempt to filter out sites, they would likely be met with lawsuits and anti-trust actions.
  - ams6110 16y ago
    Well they already rank results, so filtering is not a big extra step. If you like, just push spammy pages down rather than eliminate them entirely. Then it's just enhanced ranking, not filtering.
- moultano 16y ago
  None of Google's ranking decisions are concerned with revenue from AdSense. Period.
  - CamperBob 16y ago
    And I should believe this because....?
    - moultano 16y ago
      I work in Search Quality at Google, and this is stated policy. Revenue from any source (the ads on the results page included) is not a metric used to make ranking launch decisions. If me saying so isn't enough evidence for you, consider that it makes sense. Google knows that losing the lead in search would be much more damaging then shutting down all of AdSense.
danilocampos 16y ago
An obvious improvement to Google whose absence shocks the hell out of me would be this: Personal domain blacklist. There's a lot of spammy bullshit on the web and Google seems to have given up on keeping this away from me. Fine. But for my specific searches, there's usually a handful of offenders who, if I never, ever saw them again, it would improve my search experience by an order of magnitude. So let me personalize search by blacklisting these clowns. Why can't I filter my search results so that when I search for a programming issue, I never see these assholes from "Efreedom" who scrape and republish Stack Overflow? I don't, personally, need an algorithmic solution to spam. Just let me define spam for my personal searches and, for me, the problem is mostly solved. (Also blacklisted: Yahoo Answers, Experts Exchange.)
- mmb 16y ago
  I use the OptimizeGoogle (and previously CustomizeGoogle) Firefox plugin for this: http://www.optimizegoogle.com/ http://www.optimizegoogle.com/
- steveklabnik 16y ago
  Just a small note, SO puts all of its content under a Creative Commons license[1][2], which they are (in my non-lawyerly opinion) following. Now, this doesn't mean that filtering them wouldn't be useful to you, since at first glance it appears they're solely a duplicate. Just pointing out that they're not actually doing anything wrong, and they're (probably) not scraping.[3] SO has specifically said that this is okay.[4] 1: http://wiki.creativecommons.org/Case_Studies/StackOverflow.com http://wiki.creativecommons.org/Case_Studies/StackOverflow.c... 2: http://creativecommons.org/licenses/by-sa/2.5/ http://creativecommons.org/licenses/by-sa/2.5/ 3: http://blog.stackoverflow.com/2009/06/stack-overflow-creative-commons-data-dump/ http://blog.stackoverflow.com/2009/06/stack-overflow-creativ... 4: http://blog.stackoverflow.com/2009/06/attribution-required/ http://blog.stackoverflow.com/2009/06/attribution-required/
  - danilocampos 16y ago
    > SO has specifically said that this is okay. It doesn't look like Jeff is that okay with it, especially when it comes at the cost of Stack Overflow's own ranking: Sorry, this is absolutely necessary, otherwise we get demolished by scrapers using our own content in Google ranking. – This is from a question about Stack Overflow's SEO strategy: http://meta.stackoverflow.com/questions/71906/first-tag-in-the-title-of-the-page-is-not-that-convenient http://meta.stackoverflow.com/questions/71906/first-tag-in-t...
stcredzero 16y ago
He couldn’t make head or tail of the results. Paul concluded that the “the entire web is spam when it comes to major appliance reviews”. A simple solution to this: Consumer Reports. A subscription is well worth it! The likelihood that it will pay for itself in the next year is very high.
- ams6110 16y ago
  Consumer reports are probably about as objective a source as you can find, but I don't believe they are without their biases. They tend to give a lot of weight to things like value and reliability, and less on aesthetics, though the latter may be an important factor for some consumers. For tech products, the review them from the standpoint of an "average consumer" and probably won't evaluate factors that matter a lot to many readers here. They are also politically left-leaning, if that matters to you.
  - stcredzero 16y ago
    For tech products, the review them from the standpoint of an "average consumer" Yeah, don't use them to rate tech products. But as far as coventional appliances go, they are a very good resource.
didip 16y ago
I just created a blekko account after reading this article (good job TC! It works this time.) They seriously need to hire a capable UX person. The logged-in interface is full of problems: * Twitter-like status update. I believe this has nothing to do with search. * Form with 10+ fields on creating a slashtag. You cannot possibly expect me to enter all domain names I could think of into that tiny <textarea>? * I finally created /python but I have no idea how to improve or update the slashtag. I cannot update that slashtag from search results page. Overall, very frustrating experience.
Dramatize 16y ago
I'd like to have the option (like facebook has when you mouse over a post in your feed) to hide all results from X website. If you tied that with the ability to follow other people and their search edits, the number of spammy results could be reduced.
sdghjtykty 16y ago
------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ --------
sdghjtykty 16y ago
http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec
kmfrk 16y ago
How does yegg deal with this on DuckDuckGo? A lot of us use his search engine, and it's a great one at that, which is not worth forgetting.
- epi0Bauqu 16y ago
  By way more aggressively removing useless sites.
DTrejo 16y ago
http://duckduckgo.com/ http://duckduckgo.com/ works very well for me. - less spam - programmer oriented results, when relevant - more legible search results

222 points by vyrotek 16y ago

The argument being that Google is loosing the war against spam. A new and better Google will likely be Google itself. What we really need is a way to discover content that's not search.

meterplech 16y ago
For many things there is utility to social content finding. Sites like HN work for news/discussion, and Twitter and Facebook work for a random amalgam of things. I'm interested in the future of social search startups that somehow curate content from friends. In the articles example- asking friends if they liked their Dishwasher, and if yes what brand it is. That's the most like how people IRL make these decisions. I know there are some startups in this space as well, hope they do well!
- nhangen 16y ago
  Furthermore, I'd love to see a simple thumbs up or thumbs down system integrated into the browser or Google search. Did this page help you find what you were looking for? Was this page useful? etc
  - ehsanul 16y ago
    The problem with that, is again, spam. How do you differentiate fake votes from real ones? You can do per-IP limiting, and probably other things I'm not aware of (please comment about them), but it's still pretty open to abuse I'd say.
    - sukuriant 16y ago
      Open to exploitation theoretically, or open to exploitation practically? Can we really know this until we try? I think something like this would be a cool thing for Google to test for a year (a voting system might take a while to set up properly)
    - spidaman 16y ago
      Google has proven to be very effective data collectors (see "What they know" stuff from WSJ e.g. http://online.wsj.com/article/SB10001424052748703309704575413553851854026.html http://online.wsj.com/article/SB1000142405274870330970457541...) - it may on the one hand be surprising they don't use this data more effectively to identify and thwart spam. OTOH, they're making a lot of money on spammy SERPs.
    - nhangen 16y ago
      I suppose you do it similarly to the way Amazon or iTunes implements it for reviews...you make sure the user is logged in via a Google account. Sure, it could be exploited, and I'm guessing that's why they haven't implemented it, but there's got to be a solution that would make it work.
  - itg 16y ago
    That can be easily gamed. We already see this with online marketing/pr companies that try to game other sites such as digg/reddit/etc.
  - jbrennan 16y ago
    I'd even be OK with "Thumbs Up/Down this result just for your own personal book-keeping", so that you can promote/demote results in your own search history (try searching for "_____ tutorial photoshop" and you'll get tons of splog links before you eventually hit a golden result. It'd be nice to save time later). Google briefly had a feature like this in their results, but they've removed it. All that's left is the Star system, which isn't quite the same.
  - cmurphycode 16y ago
    Google did try this. I don't think it ever left the lab, but I used it for a while. I think they may have canned it; I just realized I haven't seen it in a while. It was called searchwiki. I think it was originally just supposed to apply to your own results, but I'm sure they analyzed the tendencies to see if it would've helful to use overall. Honestly, such a system would be ridiculously easy to game with a botnet, so there needs to be significant work in the area beforehand.
    - rhizome 16y ago
      I think the 'X" was live for maybe a month or two. I figured either or both that there was a resource/algorithm problem that prevented them from continuing in a fruitful way, or that it was a bit of user survey they could use in aggregate as preference weighting on the indexes/results. That is, not as a function of personal filtering, but to see if there were commonalities in what people considered to be bad sites. This may be further supported by the fact that the lifetime of the feature may have been too short for gamers to exploit it.
- ktsmith 16y ago
  Having just bought a new washer and dryer I can honestly say that my friends opinions were not a factor in the decision at all even though we did speak about it before I made the purchase. The main reason being other than some superficial elements of the appliances my friends can't really speak intelligently about their own appliances. They tend to not remember beyond the initial purchase point what many of the features even are. For example I asked a friend if his washer contained a heater and while it does he didn't even have a clue that it did. I have a relative working at Home Depot and we just discussed this topic at length given my own recent purchase. Most of their appliance customers come in with some idea of how much money they want to spend and what basic feature set they would like. Then they are looking for a knowledgeable sales associate to explain to them the differences and benefits of the various models. They may have "heard from a friend" that a particular model was good or a particular feature was good but even that information is usually incomplete or incorrect. I would fully expect "social" results for these types of queries to result in even more misinformation.
binomial 16y ago
Working on it - http://jellly.com/ http://jellly.com/ Also, just got a Firefox addon working that provides automatic related links to most web pages. Unfortunately, it relies on search now. But I'm sure that any sufficiently popular content-discovery tool will have a lot of spammers trying to game it. It's not easy to fight that.
- javadi82 16y ago
  I am sorry; I do not understand how this is different from the "Similar" link that is placed beside the "Cached" link on Google's SERPs. Could you please elaborate?
dawgr 16y ago
How would it be possible to discover content without searching? If I want to discover content about something I have to tell the computer what that something is about, that would be searching. If you are talking about directories, then I wouldn't use that.

replicatorblog 16y ago

It will be interesting to see how this impacts the Android/iOS battle. Search revenue funds almost all of Google's other activities so if people start using other search engines or find alternate ways to get their content it could impact the level they can spend on phones. With a push to a mobile first world the Android model is especially sensitive to spam. On a full size browser you have a lot more context and results for a given search. 5 Results may be spam, but you can work around them. If the average phone screen shows 3-5 results and all of them are spam you will quickly find alternate tools. Google ignoring spam is like Microsoft ignoring the cloud.

petervandijck 16y ago
"Google ignoring spam is like Microsoft ignoring the cloud." Great line!
- zachrose 16y ago
  That's the kind of line that I wish I could email to people in 1960 as an important business insight from 2011.
  - petervandijck 16y ago
    To them, it would sound like "Bla ignoring bla is like bla ignoring bla."
    - AngryParsley 16y ago
      They would parse the words, but not the correct meanings. The impression they'd come away with would be something like: (Extremely large number) ignoring canned meat is like (random company name) ignoring the high-altitude water vapor.
  - VMG 16y ago
    I just got mail from someone in 2060 saying "Scroomz is ignoring zork like Yooka ignored mulk"

klbarry 16y ago

Isn't the issue, of course, that spammers have no incentive to game other search engines since they're not worth the time? Any search engine that gets big will have the problem.

bambax 16y ago
Came here to say this but you said it before! ;-) Also, "crowd-sourced curated lists of websites" sound like the old Yahoo directory of yore. They will either become obsolete very quickly or spammers will find a way to penetrate and dominate them.
- jerf 16y ago
  I believe the Yahoo lists were actually centralized. Hypothetically a fully-distributed approach could work better, in the style of Wikipedia. Realistically, well, I have no idea but it's worth a try.
  - ehsanul 16y ago
    I agree. One particular try failed [1], but that doesn't mean it can't work. [1] http://en.wikipedia.org/wiki/Wikia_Search http://en.wikipedia.org/wiki/Wikia_Search
    - dasil003 16y ago
      Not only that, but Yahoo didn't think search was important, so it's not even a real try of enhancing search with curation. The truth is, if some company comes up with a better search engine, whatever ideas behind it are not going to sound like an obvious win up front—if they did then Google would already be doing that. Instead they'll have to create a search engine that is better, but somehow antithetical to Google's business model so that they can't just copy it, because there's no way for a startup to come up with enough resources to stay materially ahead of Google in pure search. And of course that's only half the battle; then you have to be better enough that users can be bothered to switch (or a browser deal coup). Personally I haven't found the spam problem to be nearly as bad as the echo chamber makes out. I think silicon valley types just have a good imagination about how good it could be.

Matt_Cutts 16y ago

"Google does provide an option to search within a date range, but these are the dates when website was indexed rather than created; which means the results are practically useless." I believe the author is mistaken on this point. Quick proof is to do a search for [matt cutts] and you'll see the root page of my blog. Click "More search tools" on the left and click the "Past week" link. Now you'll only see pages created the last week, even though lots of pages on my site were indexed in the last week.

amichail 16y ago
BTW, why is it that Google has not opened up its core search engine to third party developers so that their code can be used to bring up some of the search results by default (without requiring the user to subscribe to third party features)? Most people have never used Google's Subscribed Links feature because it is not enabled by default. User feedback can be used to determine which third party code to use in which contexts. Spam/unhelpful features would be detected quickly. There would be intense competition among third party developers for highly desired features. Something like this could give you Wolfram Alpha like features among other things such as custom UIs for various searches (e.g., travel). The Google App Engine could be used for computation. You could pay third party developers by how often their code is used in search results.
- Matt_Cutts 16y ago
  Just speaking for me personally, I'd be a fan of trying ideas like that. The potential danger would be that some code could introduce latency or other things that would make the search experience worse instead of better.
  - amichail 16y ago
    There are huge risks with this approach, but there's also a lot of potential. Search could look quite different. It would be an ideal place to experiment with and profit from novel search ideas. For example, third party developers may experiment with query-induced flash mobs where people who just performed a similar query could collaborate in real-time to find the information they need. Finding ways to prevent such an ecosystem from descending into absolute chaos would be a fascinating challenge.

DanielBMarkham 16y ago

This is exactly what blogger Paul Kedrosky found when trying to buy a dishwasher. He wrote about how he began Googleing for information…and Googleing…and Googleing. He couldn’t make head or tail of the results. Paul concluded that the “the entire web is spam when it comes to major appliance reviews”. So I happen to know somebody who is taking a small section of the home appliance market and creating content around it -- reviews, news, advice, a place for other consumers to talk to each other. Of course to do this you need to have income, so they are going to use some sort of ad-supported model. My question is very simple: is their project a spam site or not? To some, I guess it would qualify. To others, not. You see, there are two questions when it comes to search results: 1) Am I being presented results that match the query I entered? and 2) Am I being presented results that match what I want to know? These are two entirely different things. A third-grader looking for information on a movie star might find a games page with all sorts of information on that star -- all sponsored by some kind of adsensey stuff. And he's very happy. A researcher typing in the same question gets the same page? He's pissed. There is no universal answer for any one question. It's all dependent on the culture, education, and intent of the user -- all of which are not easily communicated to a search engine. Look -- this is a real problem. I hate it. Sucks to go to pages you don't like. All I'm saying is that it's more complicated than "we need a new Google" Finding what you want exactly when you want it is a difficult and non-trivial problem. We just got lucky in that Google found a simple algorithm that can be helpful in some situations. It may be that we're seeing the natural end of the usefulness of that algorithm. </soapbox>

ktsmith 16y ago
> is their project a spam site or not? To me that depends a LOT on how they present the advertisements and what they do on pages where they don't have information for a product. My biggest complaint with so called review sites is that they are presented more advertisements than content. They also tend to have automatically generated pages for every model number you can imagine, including incorrect ones that they receive searches for. On those pages there tends to be links to shopping sites and prices for unrelated appliances and products. I absolutely consider that to be spam because they are content free. The problem with google searches lately, and especially for things like appliances, is that the spammers and content mills are clearly winning. In my current search for a washer and dryer the manufacturers page was on page four or five or the results. The first few pages were flooded with bogus content pages, sale pages and unrelated pages. Trying to filter out shopping sites and explicitly target specific keywords and filter others doesn't help. There are a growing number of search topics for which google is simply broken.

mark_l_watson 16y ago

I just tried two test queries on blekko and google. Small sample, but there did seem to be less link-bate results on blekko. The issue is whether their results are close to being as up to date as google's results. I was interested that blekko seems to have done a lot with a modest amount of funding. Also, I wonder if they are getting some monetization with the association with Facebook.

meadhikari 16y ago

Professor, you could've proved your point by linking to at least one example of how Blekko found a founders work and listed it by date (as the task required), instead you have hashtags on health, finance, etc. The truth is that nobody has arranged that information in the way you want, if it existed at all, that venture database where you found the 500 companies would've been the natural place to look.. CrunchBase maybe?! Thought Worth mentioning

apollo 16y ago

This may be a bit of a tangent, but I want to see the results of the VC system survey.

oliverdamian 16y ago

How about a P2P search/bookmarking platform where peers could publish search/bookmarking histories ranked by like/dislike/spam votes which other peers can subscribe to. Publishing peers can also be ranked according to number of subscribers. Actually P2P curation could be the next level up from raw centralised search. Is there anything like this out there already?

buro9 16y ago

I love Google products, but I can't help but agree. I'm currently trying to find a colour laser printer that has good performance (quality vs speed) with a reasonable running cost over the life of the printer (at least a few years). All I'm getting is either the manufacturers slant (PR) or spam sites all harvesting the same reviews. To solve this I now look for vertical based search sites. In this case http://www.printershowcase.com/small-officecolorlaser.aspx http://www.printershowcase.com/small-officecolorlaser.aspx is the best I've found... but it's hardly to printers what dpreview is to cameras. I stick with Google because it largely works well, but when I know what I want to see and that it must exist but cannot find it... then I find myself looking elsewhere all the time. DDG and Blekko I use in these cases, but even they're not solving these kinds of needs.

stephenbez 16y ago
I trust Amazon for my product reviews.
- buro9 16y ago
  Beyond books and music... I don't. Too many products without reviews. No control for the reviews. Example... the HP CP4025 appears to be good (I have a HP Z800 workstation so thought it was worth checking HP for the hell of it)... yet I haven't found a balanced and comparable review for it. I'd like to see it set against other small office printers and to see the average cost per print. It's on Amazon twice: http://www.amazon.co.uk/Color-LaserJet-Enterprise-CP4025n-1000Base-T/dp/B0036LAFMS/ref=sr_1_1?s=officeproduct&ie=UTF8&qid=1293915915&sr=1-1 http://www.amazon.co.uk/Color-LaserJet-Enterprise-CP4025n-10... http://www.amazon.co.uk/Color-LaserJet-Enterprise-CP4025dn-1000Base-T/dp/B0036L37U0/ref=sr_1_2?s=officeproduct&ie=UTF8&qid=1293915915&sr=1-2 http://www.amazon.co.uk/Color-LaserJet-Enterprise-CP4025dn-1... But no reviews. I'd like Google to take my search terms and help me find stuff... you know, that "sort the world's data" thing, to make sense of my terms and show me the results I'm looking for. Google isn't working for me. Amazon isn't working for me. It takes hours and hours of searching in circles to research even the most basic purchasing decision.
- tedunangst 16y ago

cletus 16y ago

This issue in a roundabout kind of way touches on Facebook. The issue of social search has a lot of mindshare. Some think it is the future of search. I disagree. One of the things that made search successful anduseful early on was scale. Instead of having to go to the librar or ask your friends you can effectively canvas the connected world. I find the notion that friends' recommendations will replace that as nothing short of bizarre. It's like a huge step backwards. The argument is that you can filter out the garbage as your social graph will provide a level of curation. Let me give you a concrete example. If I wanted t buy a camera I'd stil need t go to dpreview and other sites. It's highly likely that my friends don't really know a lot about this (but some will have an opinion anyway). This same idea of human curation is behind such sites ad Mahalo and the garbage sites themselves to a degree. Of course at some point computers will be powerful enough to generate this garbage content. Blekko's idea of slash tags s interesting (to a degree) but if it's successful its easily reproducible. Google is still in the box seat here but of course that's no barrier to a link-baiting TC title. Personally I'm an optimist. I believe that, much like email spam, the garbage from AC, DM and others I'd a transitional problem (email spam is basically a solved problem now if you use a half-decent email provider). If they succeed we won't be able to find anything. I don't believe that'll happen so these services are therefore doomed. So betting on Demand Media is (to quote Tyler) like betting on the Mayans (meaning betting they're right about the world ending in 2012: it doesnt really matter if you're right). So my money is on Google being the better Google.

richcollins 16y ago
How does filtering based on a trust network imply that the network will only include your friends?
- stcredzero 16y ago
  Indeed. For such filtering to work, a significant portion of your friends should be searching the entire network and not just filtering through friends. But this might just work out to an early-adopter/follower divide. Friend network filtering will work better for the followers. The early-adopters/leaders will continue to use other means of search and will feed the friend-network.

d4nt 16y ago

It's interesting that the way of "gaming" Google appears to be in having thousands of people generating SEO friendly content. I think Google's problem is that it's pushed SEO to the point where the definition of Spam depends either on a subjective view of what kind of site the user is looking for, or it's just mildly worse than something else that's out there (e.g. When I search for something coding related and get one of the stackoverflow scrapers). Where do we go from here? Well, I don't think the answer is just a radically new way of indexing/ranking websites. That might work in the short term but the spammers will soon catch up. The answer probably lies in a combination of better language interpretation, context sensitivity using browsing history and location, and user profiling based on the social graph and search history. All of which google seems to be working on.

JusticeJones 16y ago

Tell me, how exactly is writing a sensationalized article that targets one of the Internet's oldest and largest communities to get fed by CPM advertising any different than what they decry? People have said this time and time again, but they never seem to debut let alone promise any sort of technology to address the issue. They just leave that end of the deal up in the air. As if to say that it's o.k. to spin topics as long as they strike a social nerve, but those who're less graceful at the craft are undeserving of the benefits which they themselves reap. If the search giants had any balls they'd cut the "Internet Marketing" community off at the knees. Because the money making methods pushed by that community either don't work or are unsustainable, so they're entirely reliant on a steady stream of new recruits. If they want to promote gaming your system don't let them reap any benefits from it.

eitland 16y ago
What about a three strike approach like the one that was suggested a few days ago with AdSense? Domains frequently being excluded by power searchers could be good signal. (googling to find on that used to pester my search results, kods.net, it seems it has finally got banned. Hooray!)

kokon 16y ago

CMIIW, but is that the reason why Google acquired MetaWeb a few months ago? I'm expecting to see some improvement on that front.

tokenadult 16y ago

Let me see if I correctly understand the learned professor's article. In his view, the problem is that a user using a free search engine to find information will find a lot of information about people who want to sell products and services, gaining money by exerting their time and effort. What he hopes to obtain for free is email addresses of persons to whom he wants to send his survey, so that he can use their time and effort without compensating them to produce something of value to him. Exactly how is this a problem? People who actively like to be contacted by random persons surfing the Internet make their contact information readily available (and answer questions sent through those publicly visible contact channels). But to many other persons, not being readily visible on the Internet is a feature rather than a bug. (Disclaimer: my contact information is readily visible on the Internet, so readily visible that it has been used by point-of-view pushers on Wikipedia to give me harassing telephone calls.)

tedunangst 16y ago
I think he was actually trying to piece together the background stories for the people who didn't respond to his emails, not find their emails. I don't think the info he was searching for was particularly private either. For instance, trying to find out the company a CEO worked at before their current one. The problem is that content copiers will produce so many copies of the PR announcement for their current job, it's impossible to find the announcement for their previous job. I've tried doing this exact search and it's very frustrating.
- narrator 16y ago
  I often find that things that really don't exist wind up with spam results. For instance, if you type in "free ipad" into google you will likely get thousands of search results, all spam because they don't actually exist. Similarly, contact and personal information for CEOs of major corporations does not exist online either and any search will turn up spam. One more example, to add to the many. If you get a genuine wrong number call from somebody who made a simple mistake and type their caller id into the internet, you'll just get a bunch of reverse phone lookup spam while if you search for a phone number of a know telemarketer or bill collector, you'll likely get a full dossier on that company.

jrussbowman 16y ago

One of the new things I am working on with unscatter.com is getting quicker access to reviews and blog posts using the blekko api. The next release will be a major change as I've dumped most of the current search providers in favor of blekko and have moved realtime search to it's own page with analysis by providing lists of links in the realtime feed. Nothing is released yet unfortunately. The site is officially a hobby for me write more but I hope to have the new stuff up in the next week or two. I may just hide the realtime stuff and get the blekko feeds up sooner rather than later. Now that I am focusing building the site to fit my needs getting up to date info about products and technology, the bulk of my personal searches, is the top priority. Have to admit the blekko api has helped. In the mean time I would suggest the slash tags /reviews and /blogs with /date on blekko would be very helpful if you are doing product searches. With unscatter I am really only providing shortcuts for the with additional ui tweaks. Disclaimer: I am in no way associated with blekko other than having been given permission to use their api for a personal project.

jrussbowman 16y ago
Sorry meant to say the site is a hobby for me right more. Wrote these comment with the swype keyboard on my phone so I apologize for any problems reading them

EGreg 16y ago

Hey, so what you are basically saying is, "the best computer algorithms in the world" (you know, Google has like > 578690 Ph. D's) are not good enough to have effective search, so we should introduce the human element. Fair enough. There is the Open Directory Project (which is pretty old) and of course there is Facebook, Twitter, and other, human-curated services. Starting a whole new company to do search and compete with Google (and Bing)? Seems like a waste of time as Google can just copy what you are doing and incorporate it into its already massive site (complete with traffic, audience, and lots of other goodies). Instead, why not get Google to add more social recommendation and feedback features?

ams6110 16y ago

Why would it be so difficult for Google to filter out spam sites? E.g. DuckDuckGo filters out eHow.com results, because they are low quality and tend to be spammy. Oh of course, it's not in Google's interest to do this, because they make money from the spam sites. So I don't expect Google to really "solve" this problem.... their trick is to stay useful enough that users don't abandon them, but allow enough spam into the search results to provide revenue. A tricky balance...

gsmaverick 16y ago
If Google were to attempt to filter out sites, they would likely be met with lawsuits and anti-trust actions.
- ams6110 16y ago
  Well they already rank results, so filtering is not a big extra step. If you like, just push spammy pages down rather than eliminate them entirely. Then it's just enhanced ranking, not filtering.
moultano 16y ago
None of Google's ranking decisions are concerned with revenue from AdSense. Period.
- CamperBob 16y ago
  And I should believe this because....?
  - moultano 16y ago
    I work in Search Quality at Google, and this is stated policy. Revenue from any source (the ads on the results page included) is not a metric used to make ranking launch decisions. If me saying so isn't enough evidence for you, consider that it makes sense. Google knows that losing the lead in search would be much more damaging then shutting down all of AdSense.

danilocampos 16y ago

An obvious improvement to Google whose absence shocks the hell out of me would be this: Personal domain blacklist. There's a lot of spammy bullshit on the web and Google seems to have given up on keeping this away from me. Fine. But for my specific searches, there's usually a handful of offenders who, if I never, ever saw them again, it would improve my search experience by an order of magnitude. So let me personalize search by blacklisting these clowns. Why can't I filter my search results so that when I search for a programming issue, I never see these assholes from "Efreedom" who scrape and republish Stack Overflow? I don't, personally, need an algorithmic solution to spam. Just let me define spam for my personal searches and, for me, the problem is mostly solved. (Also blacklisted: Yahoo Answers, Experts Exchange.)

mmb 16y ago
I use the OptimizeGoogle (and previously CustomizeGoogle) Firefox plugin for this: http://www.optimizegoogle.com/ http://www.optimizegoogle.com/
steveklabnik 16y ago
Just a small note, SO puts all of its content under a Creative Commons license[1][2], which they are (in my non-lawyerly opinion) following. Now, this doesn't mean that filtering them wouldn't be useful to you, since at first glance it appears they're solely a duplicate. Just pointing out that they're not actually doing anything wrong, and they're (probably) not scraping.[3] SO has specifically said that this is okay.[4] 1: http://wiki.creativecommons.org/Case_Studies/StackOverflow.com http://wiki.creativecommons.org/Case_Studies/StackOverflow.c... 2: http://creativecommons.org/licenses/by-sa/2.5/ http://creativecommons.org/licenses/by-sa/2.5/ 3: http://blog.stackoverflow.com/2009/06/stack-overflow-creative-commons-data-dump/ http://blog.stackoverflow.com/2009/06/stack-overflow-creativ... 4: http://blog.stackoverflow.com/2009/06/attribution-required/ http://blog.stackoverflow.com/2009/06/attribution-required/
- danilocampos 16y ago
  > SO has specifically said that this is okay. It doesn't look like Jeff is that okay with it, especially when it comes at the cost of Stack Overflow's own ranking: Sorry, this is absolutely necessary, otherwise we get demolished by scrapers using our own content in Google ranking. – This is from a question about Stack Overflow's SEO strategy: http://meta.stackoverflow.com/questions/71906/first-tag-in-the-title-of-the-page-is-not-that-convenient http://meta.stackoverflow.com/questions/71906/first-tag-in-t...

stcredzero 16y ago

He couldn’t make head or tail of the results. Paul concluded that the “the entire web is spam when it comes to major appliance reviews”. A simple solution to this: Consumer Reports. A subscription is well worth it! The likelihood that it will pay for itself in the next year is very high.

ams6110 16y ago
Consumer reports are probably about as objective a source as you can find, but I don't believe they are without their biases. They tend to give a lot of weight to things like value and reliability, and less on aesthetics, though the latter may be an important factor for some consumers. For tech products, the review them from the standpoint of an "average consumer" and probably won't evaluate factors that matter a lot to many readers here. They are also politically left-leaning, if that matters to you.
- stcredzero 16y ago
  For tech products, the review them from the standpoint of an "average consumer" Yeah, don't use them to rate tech products. But as far as coventional appliances go, they are a very good resource.

didip 16y ago

I just created a blekko account after reading this article (good job TC! It works this time.) They seriously need to hire a capable UX person. The logged-in interface is full of problems: * Twitter-like status update. I believe this has nothing to do with search. * Form with 10+ fields on creating a slashtag. You cannot possibly expect me to enter all domain names I could think of into that tiny <textarea>? * I finally created /python but I have no idea how to improve or update the slashtag. I cannot update that slashtag from search results page. Overall, very frustrating experience.

Dramatize 16y ago

I'd like to have the option (like facebook has when you mouse over a post in your feed) to hide all results from X website. If you tied that with the ability to follow other people and their search edits, the number of spammy results could be reduced.

sdghjtykty 16y ago

------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ -------- ------- http://path.to/e3067/ http://path.to/e3067/ --------

sdghjtykty 16y ago

http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec http://simurl.com/voblec

kmfrk 16y ago

How does yegg deal with this on DuckDuckGo? A lot of us use his search engine, and it's a great one at that, which is not worth forgetting.

epi0Bauqu 16y ago
By way more aggressively removing useless sites.

DTrejo 16y ago

http://duckduckgo.com/ http://duckduckgo.com/ works very well for me. - less spam - programmer oriented results, when relevant - more legible search results