6 ms·

Instead, the top 20+ results are all news articles repeating the same thing. I encounter this several times a week it seems (google, ddg, doesn't seem to matte

by stinos 10y ago

Instead, the top 20+ results are all news articles repeating the same thing. I encounter this several times a week it seems (google, ddg, doesn't seem to matter). Usually when trying to find the actual source of something, like a scientific study, the top hits are all news sites, often whith articles which look like copies of each other. I'm assuming they're ranked by number of visits and popularity in general of the site, but it makes it really hard to drill down to the source of something. To the point I usually give up. I am going to try the '-news' thing mentioned by another commenter though, hope that helps. Or are there any other tips?

nommm-nommm 10y ago
For scientific studies try Google Scholar scholar.google.com
throwaway2048 10y ago
Perfect example of this is trying to google for mailing list posts. All sorts of maximum ad-cram crappy sites (including ones that try to pretend to be a forum, with mailing list posts as "users") that maximize SEO gaming techniques dominate the results for 20+ pages. http://marc.info http://marc.info, which is the best archive avalible atm, plain, simple no bullshit layout and is linked everywhere anytime somebody wants to reference a mailing list archive, nowhere to be found.
- RJIb8RBYxzAMX9u 10y ago
  IMO it's because Gmane used to be highly ranked, but then it was shut down, so all those links are now gone, and the "void" has been filled with "crappy sites" like you said. Speaking of which, does anyone know what's going on with Gmane? There hasn't been any updates since the ownership change.
- mej10 10y ago
  I am sure they all have Google's ads on them, too. Only slightly suspicious.
  - robk 10y ago
    I can assure you adsense has no connection to crawl or rankings.
    - RyanZAG 10y ago
      How much connection does it have to making the removal of adsense spam pages from Google's index a priority? I have a feeling it lowers that priority significantly, and if those pages were hurting Google's bottom line, it would be priority #1 for the company to remove them.
    - emmelaich 10y ago
      Making money is certainly a motivation for doing it. A non-existant website won't get a search placing.
- rz2k 10y ago
  How does that compare to http://markmail.org http://markmail.org?
Declanomous 10y ago
I really hate how google will show every possible copy of a newswire article before it shows local sources, etc. I've had slightly better luck with DuckDuckGo on that front, but I always feel like there is a happy medium between Google's ridiculously "context-aware"* algorithm, and DuckDuckGo's "context-agnostic"* algorithm. What's really annoying is that if you use Google trends, it will let you know the context of what you are looking for, and you can clarify. I really hate it when I'm trying to look for something on Google, and I know it's just using the wrong interpretation of a word. Easy example: I do a fair amount of scripting in VBA. I often want to look up documentation for things in VBA. If I use VBA in my search term, I will get a bunch of 15 year old pages on Excel tips and tricks. If I search VBA MSDN, I will get a bunch of pages on VB.net or VB6. I specifically have to use VBA MSDN OFFICE ACCESS in order to get the correct results. DuckDuckGo does not have this problem, but I use date filtering a lot, where DDG falls apart. Search engines aside, it's ridiculous that Microsoft allowed this situation to happen in the first place. *I'm not sure if either of those terms is remotely accurate, but they seem to articulate what I'm trying to say.
- keltor 10y ago
  Every couple of years they seem to wipe out all these kinds of things, so you don't get them, then companies come up with new ways of writing their pages so that they all show up again. Remember when we had all those spammy answer sites that dominated Google searches for almost any technical question?
  - wmil 10y ago
    I'll never forget www.expertsexchange.com. Because that isn't a job for an amature.
    - paulddraper 10y ago
      Um, NSFW?
      - wmil 10y ago
wutbrodo 10y ago
Google search doesn't have much of an opinion about which sources to prefer other than "what users indicate answers their question". The vast majority of people _are_ looking for news coverage of scientific studies instead of the paper itself (people are generally unaware of how abysmally, unforgivably sloppy and dishonest much of science journalism is). If you're looking for an actual paper, why not just use Google Scholar?
- crdoconnor 10y ago
  They do. That's actually what panda was all about: "We actually came up with a classifier to say, okay, IRS or Wikipedia or New York Times over this side, and the low quality sites over this side." -- Matt Cutts
  - wutbrodo 10y ago
    How does that contradict what I said in the slightest? Here's the quote when it's not pulled out of context to distort its meaning[1]: ---- > Singhal: And based on that, we basically formed some definition of what could be considered low quality. In addition, we launched the Chrome Site Blocker [allowing users to specify sites they wanted blocked from their search results] earlier , and we didn’t use that data in this change. However, we compared and it was 84 percent overlap [between sites downloaded by the Chrome blocker and downgraded by the update]. So that said that we were in the right direction. > Wired.com: But how do you implement that algorithmically? > Cutts: I think you look for signals that recreate that same intuition, that same experience that you have as an engineer and that users have. Whenever we look at the most blocked sites, it did match our intuition and experience, but the key is, you also have your experience of the sorts of sites that are going to be adding value for users versus not adding value for users. And we actually came up with a classifier to say, okay, IRS or Wikipedia or New York Times is over on this side, and the low-quality sites are over on this side. And you can really see mathematical reasons … ---- Cutts was using IRS/Wikipedia/NYT as stand-ins for "sources people trust", not "sources of high quality by some magical objective standard that Google decides contra their users". Ultimately, their classifier is validated against what users blocked as low quality sites, since there's no grounding for quality that doesn't come from users. Hell, the quote you're pulling doesn't even make sense in the context of the exact example we're talking about! A lot of people would find a big newspaper like the NYT at the pinnacle of trustworthiness for factual content, more so than something like NBER or PLOS (where you can find the actual papers that NYT occasionally reports on, often not very well). [1] https://www.wired.com/2011/03/the-panda-that-hates-farms/ https://www.wired.com/2011/03/the-panda-that-hates-farms/
yoodenvranx 10y ago
We need some sort of "Adblock for Google search results".
- jumpCastle 10y ago
  Or a paid search engine incentivized to provide quality results rather than maximize ad network profits
  - greglindahl 10y ago
    Nice idea. But there's no sign that such a market exists. To put it into perspective, Google makes a nickle of ad revenue for every search. Would you pay that much?
    - jumpCastle 10y ago
      I might pay subscription. I agree that the market is not likely to be big. Also data network effects and difficulty with price discrimination can be big obstacles. EDIT: I think I misunderstood your question. I will probably pay that much for quality ad-free service. But of course it's a niche product.
sedachv 10y ago
Google provides a Chrome extension to block sites from Google Search results: https://chrome.google.com/webstore/detail/personal-blocklist-by-goo/nolijncfnkgaikbjbdaogikpmpbdcdef?hl=en https://chrome.google.com/webstore/detail/personal-blocklist... 3rd party addon for Firefox: https://addons.mozilla.org/En-us/firefox/addon/hide-unwanted-results-of-go/ https://addons.mozilla.org/En-us/firefox/addon/hide-unwanted... Another one for DDG in Firefox: https://addons.mozilla.org/en-US/firefox/addon/ddg-hide-unwanted-results/?src=cb-dl-created https://addons.mozilla.org/en-US/firefox/addon/ddg-hide-unwa...
- marak830 10y ago
  I never knew about this (and always wanted it). Thank you! I wish this was a feature for Google when logged in (although this is a better way I guess).
  - sedachv 10y ago
    It was a feature introduced in 2011 and discontinued in 2013: https://googlesystem.blogspot.com/2013/03/google-discontinues-blocked-sites.html https://googlesystem.blogspot.com/2013/03/google-discontinue...
    - marak830 10y ago
      Hmm of course I missed it when it was around sigh
      - JohnJamesRambo 10y ago
        "The Google Way"™
Karlozkiller 10y ago
If you're using Google to find scientific sources I would recommend trying Google Scholar. Not a perfect solution by any means, but it should get rid of news articles.
animal531 10y ago
I also have the problem when looking for new information. For the given topic you'll keep finding articles from 2014, and not of e.g. a new update that's been released recently.
- dredmorbius 10y ago
  Or the obverse: I'm interested in searching for an item not including current results. In some search engines (DDG, Google Mobile) you can only specify "last day / week / month / year", not excluding within recent intervals. Google Web, Books, and Scholar, desktop versions, are better in this regard, as specific intervals can be specified. The problem here is that for the Web, there's a great deal of newer content masquerading as older. The problem appears in print as well, but not quite to the same extent.
- ChoGGi 10y ago
  IF you add this to the search results, you can choose some preset times to search (week/month/year). ?tbs=li:1 http://jwebnet.net/advancedgooglesearch.html http://jwebnet.net/advancedgooglesearch.html https://stenevang.wordpress.com/2013/02/22/google-advanced-power-search-url-request-parameters/ https://stenevang.wordpress.com/2013/02/22/google-advanced-p...
  - animal531 10y ago
    Thanks for the info, I didn't know about some of those options.
dredmorbius 10y ago
If you have access to an academic library (community college will often suffice), you can directly access scholarly and periodical indices. These will save hours as compared to a general Web search, even on Google Scholar. Sci-Hub will turn up most of the results, LibGen and BookZZ many of the print sources.
xelxebar 10y ago
I sympathize with this a lot. At least for scientific articles, it usually helps if you're willing to spend enough time to familiarize yourself with the technical jargon. As a somewhat dated example, "Higgs particle evidence" will turn up lots of crappy new articles, but "CMS Higgs abundances" will immediately return high quality relevant results. Of course, you have to figure out the right lingo int the first place, but usually you can mine arXiv/PubMed/etc for this kind of thing. Wikipædia usually has links to some kind of primary source.

by stinos 10y ago

nommm-nommm 10y ago

For scientific studies try Google Scholar scholar.google.com

throwaway2048 10y ago

Perfect example of this is trying to google for mailing list posts. All sorts of maximum ad-cram crappy sites (including ones that try to pretend to be a forum, with mailing list posts as "users") that maximize SEO gaming techniques dominate the results for 20+ pages. http://marc.info http://marc.info, which is the best archive avalible atm, plain, simple no bullshit layout and is linked everywhere anytime somebody wants to reference a mailing list archive, nowhere to be found.

RJIb8RBYxzAMX9u 10y ago
IMO it's because Gmane used to be highly ranked, but then it was shut down, so all those links are now gone, and the "void" has been filled with "crappy sites" like you said. Speaking of which, does anyone know what's going on with Gmane? There hasn't been any updates since the ownership change.
mej10 10y ago
I am sure they all have Google's ads on them, too. Only slightly suspicious.
- robk 10y ago
  I can assure you adsense has no connection to crawl or rankings.
  - RyanZAG 10y ago
    How much connection does it have to making the removal of adsense spam pages from Google's index a priority? I have a feeling it lowers that priority significantly, and if those pages were hurting Google's bottom line, it would be priority #1 for the company to remove them.
  - emmelaich 10y ago
    Making money is certainly a motivation for doing it. A non-existant website won't get a search placing.
rz2k 10y ago
How does that compare to http://markmail.org http://markmail.org?

Declanomous 10y ago

I really hate how google will show every possible copy of a newswire article before it shows local sources, etc. I've had slightly better luck with DuckDuckGo on that front, but I always feel like there is a happy medium between Google's ridiculously "context-aware"* algorithm, and DuckDuckGo's "context-agnostic"* algorithm. What's really annoying is that if you use Google trends, it will let you know the context of what you are looking for, and you can clarify. I really hate it when I'm trying to look for something on Google, and I know it's just using the wrong interpretation of a word. Easy example: I do a fair amount of scripting in VBA. I often want to look up documentation for things in VBA. If I use VBA in my search term, I will get a bunch of 15 year old pages on Excel tips and tricks. If I search VBA MSDN, I will get a bunch of pages on VB.net or VB6. I specifically have to use VBA MSDN OFFICE ACCESS in order to get the correct results. DuckDuckGo does not have this problem, but I use date filtering a lot, where DDG falls apart. Search engines aside, it's ridiculous that Microsoft allowed this situation to happen in the first place. *I'm not sure if either of those terms is remotely accurate, but they seem to articulate what I'm trying to say.

keltor 10y ago
Every couple of years they seem to wipe out all these kinds of things, so you don't get them, then companies come up with new ways of writing their pages so that they all show up again. Remember when we had all those spammy answer sites that dominated Google searches for almost any technical question?
- wmil 10y ago
  I'll never forget www.expertsexchange.com. Because that isn't a job for an amature.
  - paulddraper 10y ago
    Um, NSFW?
    - wmil 10y ago

wutbrodo 10y ago

Google search doesn't have much of an opinion about which sources to prefer other than "what users indicate answers their question". The vast majority of people _are_ looking for news coverage of scientific studies instead of the paper itself (people are generally unaware of how abysmally, unforgivably sloppy and dishonest much of science journalism is). If you're looking for an actual paper, why not just use Google Scholar?

crdoconnor 10y ago
They do. That's actually what panda was all about: "We actually came up with a classifier to say, okay, IRS or Wikipedia or New York Times over this side, and the low quality sites over this side." -- Matt Cutts
- wutbrodo 10y ago
  How does that contradict what I said in the slightest? Here's the quote when it's not pulled out of context to distort its meaning[1]: ---- > Singhal: And based on that, we basically formed some definition of what could be considered low quality. In addition, we launched the Chrome Site Blocker [allowing users to specify sites they wanted blocked from their search results] earlier , and we didn’t use that data in this change. However, we compared and it was 84 percent overlap [between sites downloaded by the Chrome blocker and downgraded by the update]. So that said that we were in the right direction. > Wired.com: But how do you implement that algorithmically? > Cutts: I think you look for signals that recreate that same intuition, that same experience that you have as an engineer and that users have. Whenever we look at the most blocked sites, it did match our intuition and experience, but the key is, you also have your experience of the sorts of sites that are going to be adding value for users versus not adding value for users. And we actually came up with a classifier to say, okay, IRS or Wikipedia or New York Times is over on this side, and the low-quality sites are over on this side. And you can really see mathematical reasons … ---- Cutts was using IRS/Wikipedia/NYT as stand-ins for "sources people trust", not "sources of high quality by some magical objective standard that Google decides contra their users". Ultimately, their classifier is validated against what users blocked as low quality sites, since there's no grounding for quality that doesn't come from users. Hell, the quote you're pulling doesn't even make sense in the context of the exact example we're talking about! A lot of people would find a big newspaper like the NYT at the pinnacle of trustworthiness for factual content, more so than something like NBER or PLOS (where you can find the actual papers that NYT occasionally reports on, often not very well). [1] https://www.wired.com/2011/03/the-panda-that-hates-farms/ https://www.wired.com/2011/03/the-panda-that-hates-farms/

yoodenvranx 10y ago

We need some sort of "Adblock for Google search results".

jumpCastle 10y ago
Or a paid search engine incentivized to provide quality results rather than maximize ad network profits
- greglindahl 10y ago
  Nice idea. But there's no sign that such a market exists. To put it into perspective, Google makes a nickle of ad revenue for every search. Would you pay that much?
  - jumpCastle 10y ago
    I might pay subscription. I agree that the market is not likely to be big. Also data network effects and difficulty with price discrimination can be big obstacles. EDIT: I think I misunderstood your question. I will probably pay that much for quality ad-free service. But of course it's a niche product.

sedachv 10y ago

Google provides a Chrome extension to block sites from Google Search results: https://chrome.google.com/webstore/detail/personal-blocklist-by-goo/nolijncfnkgaikbjbdaogikpmpbdcdef?hl=en https://chrome.google.com/webstore/detail/personal-blocklist... 3rd party addon for Firefox: https://addons.mozilla.org/En-us/firefox/addon/hide-unwanted-results-of-go/ https://addons.mozilla.org/En-us/firefox/addon/hide-unwanted... Another one for DDG in Firefox: https://addons.mozilla.org/en-US/firefox/addon/ddg-hide-unwanted-results/?src=cb-dl-created https://addons.mozilla.org/en-US/firefox/addon/ddg-hide-unwa...

marak830 10y ago
I never knew about this (and always wanted it). Thank you! I wish this was a feature for Google when logged in (although this is a better way I guess).
- sedachv 10y ago
  It was a feature introduced in 2011 and discontinued in 2013: https://googlesystem.blogspot.com/2013/03/google-discontinues-blocked-sites.html https://googlesystem.blogspot.com/2013/03/google-discontinue...
  - marak830 10y ago
    Hmm of course I missed it when it was around sigh
    - JohnJamesRambo 10y ago
      "The Google Way"™

Karlozkiller 10y ago

If you're using Google to find scientific sources I would recommend trying Google Scholar. Not a perfect solution by any means, but it should get rid of news articles.

animal531 10y ago

I also have the problem when looking for new information. For the given topic you'll keep finding articles from 2014, and not of e.g. a new update that's been released recently.

dredmorbius 10y ago
Or the obverse: I'm interested in searching for an item not including current results. In some search engines (DDG, Google Mobile) you can only specify "last day / week / month / year", not excluding within recent intervals. Google Web, Books, and Scholar, desktop versions, are better in this regard, as specific intervals can be specified. The problem here is that for the Web, there's a great deal of newer content masquerading as older. The problem appears in print as well, but not quite to the same extent.
ChoGGi 10y ago
IF you add this to the search results, you can choose some preset times to search (week/month/year). ?tbs=li:1 http://jwebnet.net/advancedgooglesearch.html http://jwebnet.net/advancedgooglesearch.html https://stenevang.wordpress.com/2013/02/22/google-advanced-power-search-url-request-parameters/ https://stenevang.wordpress.com/2013/02/22/google-advanced-p...
- animal531 10y ago
  Thanks for the info, I didn't know about some of those options.

dredmorbius 10y ago

If you have access to an academic library (community college will often suffice), you can directly access scholarly and periodical indices. These will save hours as compared to a general Web search, even on Google Scholar. Sci-Hub will turn up most of the results, LibGen and BookZZ many of the print sources.

xelxebar 10y ago

I sympathize with this a lot. At least for scientific articles, it usually helps if you're willing to spend enough time to familiarize yourself with the technical jargon. As a somewhat dated example, "Higgs particle evidence" will turn up lots of crappy new articles, but "CMS Higgs abundances" will immediately return high quality relevant results. Of course, you have to figure out the right lingo int the first place, but usually you can mine arXiv/PubMed/etc for this kind of thing. Wikipædia usually has links to some kind of primary source.