16 ms·
Google may rank sites for queries that don't appear on the page at all
- eli 8y agoHasn't google has ranked pages for text that appears in links to them but not on them for a long time?
- jakubp 8y agoI don't know about links, but the phenomenon of getting results for a query like: A "B" with no B on the page has been particularly disturbing for some time. At least months, maybe longer, I can't say. I have no idea why they made a change this drastic (it is drastic when someone makes a deliberate effort to say "I need "B" in the page, it's not a whim or a random query" and the search engine shows results which violate the query _syntax_).
- goodcharles 8y agoYes this is true. Google will rank web pages, based on the inbound links pointing to the page, even if Google hasn't crawled the page itself. This happens regularly when a web page is blocked from crawling via Robots.txt file. Google still indexes and ranks the page, but Google has no idea what is on the page. Want to keep a web page out of rankings? Allow Google to crawl it (by not restricting via robots.txt), but use a NoIndex meta tag or X-Robots-Tag HTTP header to indicate that the page should not appear in search results.
- dazc 8y agoNot only will google index all those pages you are blocking them from but will also penalise you when they have indexed x number of them. Fix the problem with X-robots-tag, request removal from index (*while such option is available) and then wait for months for the penalty to be lifted.
- snthd 8y agoYes. https://en.wikipedia.org/wiki/Google_bomb https://en.wikipedia.org/wiki/Google_bomb > Google bombs date back as far as 1999, when a search for "more evil than Satan himself" resulted in the Microsoft homepage as the top result.
- larrik 8y agoYes, this was the whole point of Google, and how they beat Yahoo, etc.
- rtkwe 8y agoI think so. It definitely seems like a nice shortcut to link together adjacent concepts and queries. If a page talking about a particular topic (but in a different way or with different phrasing) than the original page it could help build out a kind of list of related topics.
- softwaredoug 8y agoI work on (non Google) search engines. This kind of thing has been happening for decades. Whether it’s synonyms, entity extraction, or embeddings to capture conceptual relationships, I’m surprised this is surprising... but then again I’m a search nerd :)
- jakubp 8y agoOn Google, at least a few years ago and back, when you put a query "X" you were guaranteed to see X in the HTML contents of the page. Or at least I can't remember getting a result that didn't satisfy that condition. Now it's common to not find X there.
- softwaredoug 8y agoYes you’re right on exact matches. For basic keyword searches. It’s not uncommon for search users to want inexact equivalencies, synonyms or conceptually close search results. (Search for cold, get back rhinovirus). Every user and use case has different definitions/tolerances of “this term appeared on the page” It’s a complicated topic, without simple solutions. I wrote about it here. https://opensourceconnections.com/blog/2018/12/07/synonyms-by-any-other-name-part-1/ https://opensourceconnections.com/blog/2018/12/07/synonyms-b...
- jakubp 8y agoIf users want equivalencies then they shouldn't use quotes, which are for exact match. Let's not play semantics here. Search syntax is clear - put things in quotes, get 'exact match' in return. Equivalent words are not exact match. Get smart with me, decrease usability for me. This is what is happening. We can argue whether there is place for more operators, or whether it's better on average for an average user to still be "outsmarted" by the engine, but it's clear what's going on - these aren't exact match results and the results often have zero relevance with what user is looking for precisely because the user knows the specific word WILL appear on the page they need and a page without that word WON'T be what they need. I don't know how much clearer we can be :) It used to work (or seemed to). Now it doesn't. It frustrates some people. That's it.
- taneq 8y agoThe annoying thing is when you get pages of results for (popular keyword) which is maybe tangentially related to (alternate interpretation of keyword you used).
- stordoff 8y agoIt's frustrating because it just translates into making Google harder to use, and multiple searches for the same thing are now common. I often see, on a four word search, the top five or so results all having "Missing: word3 word4". I often search for "<thing> <thing I would like to known about it>", and Google goes "Here's some general information about <thing> instead". It's not _useful_.
- XCabbage 8y agoBut does this happen because Google is ranking those results above ones that matched both words, or because nothing on the web matches both words? I'm often unsure what's going on when I experience this.
- r3bl 8y agoI think there's historical reference as well. If a site X publishes your exact terms but hasn't posted in a while, site Y that posts regularly appears to me like it'll be above it, even if it contains only a vague interpretation of your search term.
- stordoff 8y agoGenerally speaking, I can find the information I am looking for, usually by forcing Google to include those words.
- taneq 8y agoThere used to be a time where if you typed three words and got zero results, it was "no page on the ENTIRE INTERNET contains those three words." Now it's "these are what the algorithm thinks you want".
- vesinisa 8y agoGoogling 101: surround the search phrase with literal double quotes ("") to request an exact match. Includes the unwanted page: https://www.google.com/search?q=2109085405 https://www.google.com/search?q=2109085405 Does not include the unwanted page: https://www.google.com/search?q="2109085405" https://www.google.com/search?q="2109085405"
- paco3346 8y agoExcept this doesn't exact match anymore. It's just a strongly suggested phrase.
- Ivoirians 8y agoDo you have an example of this not working?
- mcbits 8y ago"thraed ripper" Maybe not the most compelling example, but I think it's usually with really blatant misspellings that Google will second guess my quoted terms, which can be annoying if I'm actually looking for the misspelled term.
- pedrosorio 8y agoNot very compelling because: - The exact search for that string returns nothing - You can still force it to do that quite easily by clicking "Search instead" right at the top If you search for a term with no occurrences which is very likely to be a misspelling of an existing popular result, what maximizes total usefulness for users? - No results (10% of the cases you are actually searching for something that does not exist) - Here are the results for the term you're likely searching for (90% of the time) + btw click this link right here to get (no) results for the nonsensical search term
- mcbits 8y ago
- ComputerGuru 8y agoRecently discussed here: https://news.ycombinator.com/item?id=19380212 https://news.ycombinator.com/item?id=19380212 Posted about it at length here: https://neosmart.net/blog/2016/on-the-growing-intentional-uselessness-of-google-search-results/ https://neosmart.net/blog/2016/on-the-growing-intentional-us...
- aboutruby 8y agoThis is a link to a thread (created by the parent commenter) with 15 comments on a post about DuckDuckGo. Link is from 2016 obviously.
- Neil44 8y agoI’ve seen this when googling the numberplate of an old Ferrari. Google knew what model it was and returned results as if I’d searched Ferrari 512 TR or whatever. I was pretty impressed / freaked out at the time.
- kjhughes 8y agoThis has been happening for many years now. In fact, Google introduced "verbatim" search back in 2011 for those who wished for search results that more strictly contained the exact search terms: https://searchenginewatch.com/sew/news/2126346/google-introduces-verbatim-searching https://searchenginewatch.com/sew/news/2126346/google-introd... Verbatim search is still available today. Searches from the URL bar can be made to use verbatim search by default via this search string: https://www.google.com/search?tbs=li:1&q=%s https://www.google.com/search?tbs=li:1&q=%s
- JohnFen 8y agoVerbatim mode is very insufficient. We need actual logical operators to go with it.
- MaulingMonkey 8y agoI just tried this on one of my old favorite games, "Stars!". I used to be able to search for that. But google ditches the !. Even quoted. Even verbatim. "Stars! game" (with or without quotes) helps a little. The right wikipedia page is linked. Mostly it's drowned out in a sea of stuff about the Dallas Stars, NBA all stars, etc. "Stars! game windows" is getting closer and has more results, but it's still cluttered up with results for various windows games, companies (StarDock), etc. that have nothing to do with what I want. The real trick here is to search for Stars AutoHost, since that was one of the hub websites of the game. Doesn't need to be verbatim, quoted, anything. Suddenly, the actual page titles actually have "Stars!" - complete with exclamation mark - in them. This isn't an edge case for me. C++, C#, ".NET", tons of tech thinks punctuation is cool and (ab)uses it. To say nothing of all the various operators these languages have. Bleh.
- scoutt 8y agoGoogle has become more and more useless lately, at least if you are searching for specific tech info and not "10 weird hacks to". For example, I've been doing lately some Android AOSP and BSP customization for a custom board. To get useful information and avoid "top 10 Android tricks" and Android hacking/cracking forums I often end up with a query composed of 10% search terms and 90% filters. Or if you do a search for name surname it may return results for "name" with "missing: surname". That is, how on earth if I search about (for example) Peter Petrelli a result about Peter Thiel would be the same thing? That's a full 50% of my search query that Google is discarding...
- IpV8 8y agoI installed a browser extension that adds a blocklist to google's search results. So every time I visit a '10 wierd hacks to...' type of page I block the entire domain from ever showing up in my results again. It take a while to filter them all out, but it drastically increased my search experience. Anyways I use duck duck go nowadays so now my plugin is irrelevant...
- fps_doug 8y agoSame here. There's still times where a !g actually improves the results but usually duckduckgo's results are fine.
- snaky 8y agoI wish there were a way to exchange that block lists between the users of this extension.
- stefanfisk 8y agoWhich extension are you using? Personal Blocklist broke for me a while back, and now my days are filled with scrolling past w3schools.com to get to developer.mozilla.org
- dhimes 8y agoA duck duck go search of something like <what I'm looking up> javascript mdn give me developer.mozilla.org at the top every time. It's somehow better than site:developer.mozilla.org for some weird reason.
- mqus 8y agomaybe I'm just weird but I always look at the green text below the blue links and expect to see my search terms (or their derived equivalents) with a bit of context. I can't understand why google would even list links where it shows me some context but without _any_ of my search terms in it. I probably wont click on them anyway(unless the title or url are containing search terms)
- deleted 8y ago[deleted]
- babyslothzoo 8y agoThe more Google uses this type of presumptive search behavior rather than what I'm actually searching for, the more I find myself using Duckduckgo and Bing for finding specific things with specific search queries. If I am searching for "xyz abc" I am searching for those specific terms for a reason, so please don't present me with results for the entirely unrelated "123", and no I am not talking about showing synonyms.
- ucaetano 8y agoIsn't this the core concept of PageRank? If there are 10 million websites out there linking to a single article when referencing X, even if such article doesn't include X verbatim, that article will rank high in queries for X.
- dumbfounder 8y agoIt has always been the case that the search terms do not necessarily appear on the resultant pages, this is actually the cornerstone of their algorithm. They use the text in the links and nearby text to apply to the site it is linking to. This was from day 1. It was why you would get good results for categorical searches like "search engine", "shopping site", "best auction site", etc. It didn't say those terms on the sites that showed up high in the list, but that's how everyone described them.
- kaikai 8y agoI always wrote elaborate search queries with most important terms first, no filler words, etc. Lately I'm using more natural language searches, because they return better results. Probably they're optimizing the engine to return the best results for the most people, and most people want to know "how do I turn on the furnace" not "[model number] furnace manual"
- lemoncucumber 8y agoThis headline is silly. Google has ranked sites for text that doesn't appear on the page basically for as long as Google has existed: https://en.wikipedia.org/wiki/Google_bomb https://en.wikipedia.org/wiki/Google_bomb
- utopcell 8y agoMore often than not, if a term is missing from a page, chances are it is on the anchor text of a link that points to the page.
- deleted 8y ago[deleted]
- tomerbd 8y agoI like google
- MockObject 8y agoTerm frequency–inverse document frequency is a technique that can retrieve documents related to a term, but which literally lack it. https://en.wikipedia.org/wiki/Tf%E2%80%93idf https://en.wikipedia.org/wiki/Tf%E2%80%93idf
- JohnFen 8y agoI wonder if this is related to the fact the quality of Google's search results has seriously declined for me over the years. I keep getting the feeling that Google is trying to second-guess what I'm looking for, and not only getting it wrong, but getting it more and more wrong as time goes by.
- hartator 8y ago> On perhaps Google is linking pages based on how searchers move around the web. For example, a searcher might first search for the phone numbers and not find what they’re looking for, so they search for something like the article title, and end up on the article. Google might see a high enough proportion of users behaving this way, and decide to save them the trouble of performing the second search. Or - way simpler - maybe a link to the page at the text "4843218317" on it.
- groestl 8y agoHere is how this works: https://www.link-assistant.com/news/keyword-refinements.html https://www.link-assistant.com/news/keyword-refinements.html Basically, Google tries to anticipate query refinements, improving the search as a skilled user would do. Most users are not skilled though, so this leads to an overall better search experience for them, but makes the experience for skilled users worse, due to false positives.
- mtsx 8y agoSir, 4843218317 appeared on your article so many time.. 48.. 2 times..32 3 times..18 once ..83 once...31 once...17 4 times...(maybe,its only for number search :) )...i'm a hacker from Bangladesh.{ha ha :) }
- deleted 8y ago[deleted]
- opportune 8y agoI don't mind google replacing keywords with similar word2vec or doc2vec embeddings (or some other statistical fuzzing) that they found typically increased relevancy/click-through on users. Not sure why anybody really does other than the fact it makes the reasoning behind why certain results show up more opaque.
- deleted 8y ago[deleted]
- crazygringo 8y agoIt doesn't seem completely random... the first two results which include the number are about Uber SMS messages, and the blog post is about Uber SMS messages. So Google has clearly determined that this number is somehow associated with sketchy Uber SMS messages, out of a very sparse signal... That seems pretty cool actually. Like, really cool.
- OnlyRepliesToBS 8y agofull of deceit