8 ms·

Google to divide its index, giving mobile users better and fresher content

76 points by kjhughes 10y ago

ocdtrekkie 10y ago
As someone who only uses my phone to keep up on things between desktops, it's pretty awful for me to hear that Google is going to update the desktop index slower than the mobile one. If I'm searching for something, I'm on a desktop. I don't care about which websites are best optimized for mobile or which websites use Google's latest attempt to move the entire web to their infrastructure. More and more, Google is demonstrating they're no longer willing or capable of meeting my needs as a search provider or a service provider, and I'm glad I don't use it anymore.
- userbinator 10y ago
  I still remember when Google was pretty much a "grep for the Web", and wished it would stay that way. But now it seems to be a pseudo-search that second-guesses your queries, deindexes or only partially indexes old sites (which are still very much alive), censors, and does all sorts of other things which I find unpleasant and frustrating. Finding popular and new content is the easy part; unfortunately Google has only focused on that and severely neglected what I think is the main use of a search engine: finding the obscure and rare content that exists somewhere, but is hard to get at. Unfortunately, DDG, Bing, and the others are still far behind the last time I used them. But maybe not for long...
  - sliverstorm 10y ago
    While they can sometimes frustrate me when I'm running a very specific search, there are some features that I often like, such as the one that includes close synonyms of search terms, and includes both plural and singular forms. Makes it more of a "perl for the Web", which has its place. https://xkcd.com/208/ https://xkcd.com/208/
  - ocdtrekkie 10y ago
    I've never found a drastic difference between Google, Bing, and DDG except in one category: Searching Google websites. Bing's YouTube results are frequently terrible, for example, where highly popular trending videos are not in the top results when searched for on Bing's video search. Google indexes Google+ posts well, nobody else does. I'm not sure if that's a fault of the other search providers, or Google unsurprisingly making it difficult for competitors to index websites that they obviously can index quite easily using internal APIs.
- arrty88 10y ago
  what is it you use now for search?
  - proaralyst 10y ago
    I'm not the parent, but I've been using DuckDuckGo heavily, primarily because of its bangs[0]. [0] https://duckduckgo.com/bang https://duckduckgo.com/bang
  - ocdtrekkie 10y ago
    I use Bing and DuckDuckGo somewhat interchangeably. Bing is particularly useful at work since I'm usually looking up Microsoft support issues (and they pay you to use their search in Amazon gift cards, which is nice), and DDG is obviously much better for privacy.
    - existencebox 10y ago
      Not sure why this is being downvoted. "Microsoft Rewards" is apparently a real program (discussed Here https://www.reddit.com/r/MicrosoftRewards https://www.reddit.com/r/MicrosoftRewards) which even I as a MSFTie knew almost nothing about. (Not speaking as advocation or any official/unofficial capacity yadda yadda) I also find myself "OK with" bing for when I'm doing work searches; but it does seem to be more miss than hit for my more general tech questions. Like with browsers, multiple search engines seems like the unfortunate current state of the world as each has its own pathology.
- carapace 10y ago
  I felt really weird the day I realized I don't use Google to "google" anymore. I use DuckDuckGo, when it fails I'll fall back to Google, but even then it's getting to be pretty useless compared to how it was. For example, I recently wanted to find a reference for an obscure quote that included the word "squaring" but all the search results from Google were for "square". So frustrating.
  - Achshar 10y ago
    You can put "squaring" in quotations and then google will search exactly for "squaring".
    - dawnerd 10y ago
      I've had mixed results using their modifiers. Remember when you used to be able to do something like -something and it would filter those out? Hasn't worked for me in months.
  - corpus 10y ago
    can you remember the query you tried?
    - carapace 10y ago
      No, but "squaring bucky fuller quote" just now showed the problem.
      - corpus 10y ago
        and what is the correct answer?
        carapace 10y ago
        Never did find it. In one of his books he points out that associating the second power with the area of a square (as opposed to some other base shape, like a triangle or circle) is a choice we make. Consider triangles (hallucinate the bases): Side Area 1 /\ 1 /\ 2 /\/\ 4 /\ 3 /\/\ 9 /\/\/\
rm999 10y ago
This was publicly announced more than a year ago, but still isn't live: http://www.thesempost.com/google-working-completely-separate-mobile-index-desktop-index/ http://www.thesempost.com/google-working-completely-separate... This title seems misleading, especially the "better" part. I think what's going on is a lot of the web has bifurcated into mobile versions and desktop versions of the same site. Mobile versions are often stripped down for speed reasons, but mobile searchers can still land on a page based on a search query that only exists on the desktop site. Google is fixing this in a pretty sensible way (although I don't get why they are explicitly making the mobile crawler more active or "fresher"). This won't affect sites that serve the same content on mobile and desktop. I suppose now the concern may swap over to sites that offer more content on mobile, but I don't know why anyone would do that.
- puzzle 10y ago
  Reality is much more complex than that. Google's index has been tiered for many years, see e.g. https://search.googleblog.com/2012/05/search-quality-highlights-53-changes.html https://search.googleblog.com/2012/05/search-quality-highlig... • New index tier. [launch codename "cantina", project codename "Indexing"] We keep our index in “tiers” where different documents are indexed at different rates depending on how relevant they are likely to be to users. This month we introduced an additional indexing tier to support continued comprehensiveness in search results.
  - ceph_ 10y ago
    This just means how often they scrape a site. Reddit gets scanned more often for changes than someone's personal website that only gets a few hits. You can test this. Posts on Reddit show up in search in minutes.
    - puzzle 10y ago
      It's not just a matter of freshness. They are also served by different backends with different trade-offs.
romanovcode 10y ago
Very simple decision for them. They care about android far more than desktop stuff.
- deelowe 10y ago
  The mobile market is bigger and still growing. Desktop is shrinking. It's a simple financial decision.
  - msandford 10y ago
    I don't get it. It's essentially free to give desktop users just as good of results as mobile users, isn't it? How does intentionally crippling desktop help mobile?
    - user5994461 10y ago
      Simple. Google makes money from search ad. Desktop users block ads. Mobile users don't.
      - ars 10y ago
        > Desktop users block ads. Mobile users don't. I do. Anyone who root's their phone can easily block ads.
        user5994461 10y ago
        You're confirming that >90% of mobile users cannot block ads at all.
      - tomascot 10y ago
        Firefox + ublock. I feel blocking ads on mobile is crucial considering how small data caps are.
        oopsies49 10y ago
al_biglan 10y ago
I may not understand the impact of this, but this sounds like "there is a limit to Google-scale solutions and we have hit it"? I know this is a gross over-simplification, but I'd appreciate if someone might offer more depth to this? FWIW: I have liked the duckduckgo approach of separating the "space" of the search, but I always figured this was a stop-gap until Google could grok the context from your situation (e.g. 4 searches in a row related to motorcycles and the 5th for AMA shouldn't give me anything on the American Medical Society, but instead prefer American Motorcycle Association) Now I'm wondering if there may be some limit to Google's ability to hit this in a timely manner with the amount of content available. Insights very much appreciated.
seonumber1 10y ago
How Providing New & Fresher Content To Mobile & Small Device Users Will Play A Beneficial Role?

76 points by kjhughes 10y ago

As someone who only uses my phone to keep up on things between desktops, it's pretty awful for me to hear that Google is going to update the desktop index slower than the mobile one. If I'm searching for something, I'm on a desktop. I don't care about which websites are best optimized for mobile or which websites use Google's latest attempt to move the entire web to their infrastructure. More and more, Google is demonstrating they're no longer willing or capable of meeting my needs as a search provider or a service provider, and I'm glad I don't use it anymore.

userbinator 10y ago
I still remember when Google was pretty much a "grep for the Web", and wished it would stay that way. But now it seems to be a pseudo-search that second-guesses your queries, deindexes or only partially indexes old sites (which are still very much alive), censors, and does all sorts of other things which I find unpleasant and frustrating. Finding popular and new content is the easy part; unfortunately Google has only focused on that and severely neglected what I think is the main use of a search engine: finding the obscure and rare content that exists somewhere, but is hard to get at. Unfortunately, DDG, Bing, and the others are still far behind the last time I used them. But maybe not for long...
- sliverstorm 10y ago
  While they can sometimes frustrate me when I'm running a very specific search, there are some features that I often like, such as the one that includes close synonyms of search terms, and includes both plural and singular forms. Makes it more of a "perl for the Web", which has its place. https://xkcd.com/208/ https://xkcd.com/208/
- ocdtrekkie 10y ago
  I've never found a drastic difference between Google, Bing, and DDG except in one category: Searching Google websites. Bing's YouTube results are frequently terrible, for example, where highly popular trending videos are not in the top results when searched for on Bing's video search. Google indexes Google+ posts well, nobody else does. I'm not sure if that's a fault of the other search providers, or Google unsurprisingly making it difficult for competitors to index websites that they obviously can index quite easily using internal APIs.
arrty88 10y ago
what is it you use now for search?
- proaralyst 10y ago
  I'm not the parent, but I've been using DuckDuckGo heavily, primarily because of its bangs[0]. [0] https://duckduckgo.com/bang https://duckduckgo.com/bang
- ocdtrekkie 10y ago
  I use Bing and DuckDuckGo somewhat interchangeably. Bing is particularly useful at work since I'm usually looking up Microsoft support issues (and they pay you to use their search in Amazon gift cards, which is nice), and DDG is obviously much better for privacy.
  - existencebox 10y ago
    Not sure why this is being downvoted. "Microsoft Rewards" is apparently a real program (discussed Here https://www.reddit.com/r/MicrosoftRewards https://www.reddit.com/r/MicrosoftRewards) which even I as a MSFTie knew almost nothing about. (Not speaking as advocation or any official/unofficial capacity yadda yadda) I also find myself "OK with" bing for when I'm doing work searches; but it does seem to be more miss than hit for my more general tech questions. Like with browsers, multiple search engines seems like the unfortunate current state of the world as each has its own pathology.
carapace 10y ago
I felt really weird the day I realized I don't use Google to "google" anymore. I use DuckDuckGo, when it fails I'll fall back to Google, but even then it's getting to be pretty useless compared to how it was. For example, I recently wanted to find a reference for an obscure quote that included the word "squaring" but all the search results from Google were for "square". So frustrating.
- Achshar 10y ago
  You can put "squaring" in quotations and then google will search exactly for "squaring".
  - dawnerd 10y ago
    I've had mixed results using their modifiers. Remember when you used to be able to do something like -something and it would filter those out? Hasn't worked for me in months.
- corpus 10y ago
  can you remember the query you tried?
  - carapace 10y ago
    No, but "squaring bucky fuller quote" just now showed the problem.
    - corpus 10y ago
      and what is the correct answer?
      - carapace 10y ago
        Never did find it. In one of his books he points out that associating the second power with the area of a square (as opposed to some other base shape, like a triangle or circle) is a choice we make. Consider triangles (hallucinate the bases): Side Area 1 /\ 1 /\ 2 /\/\ 4 /\ 3 /\/\ 9 /\/\/\

rm999 10y ago

This was publicly announced more than a year ago, but still isn't live: http://www.thesempost.com/google-working-completely-separate-mobile-index-desktop-index/ http://www.thesempost.com/google-working-completely-separate... This title seems misleading, especially the "better" part. I think what's going on is a lot of the web has bifurcated into mobile versions and desktop versions of the same site. Mobile versions are often stripped down for speed reasons, but mobile searchers can still land on a page based on a search query that only exists on the desktop site. Google is fixing this in a pretty sensible way (although I don't get why they are explicitly making the mobile crawler more active or "fresher"). This won't affect sites that serve the same content on mobile and desktop. I suppose now the concern may swap over to sites that offer more content on mobile, but I don't know why anyone would do that.

puzzle 10y ago
Reality is much more complex than that. Google's index has been tiered for many years, see e.g. https://search.googleblog.com/2012/05/search-quality-highlights-53-changes.html https://search.googleblog.com/2012/05/search-quality-highlig... • New index tier. [launch codename "cantina", project codename "Indexing"] We keep our index in “tiers” where different documents are indexed at different rates depending on how relevant they are likely to be to users. This month we introduced an additional indexing tier to support continued comprehensiveness in search results.
- ceph_ 10y ago
  This just means how often they scrape a site. Reddit gets scanned more often for changes than someone's personal website that only gets a few hits. You can test this. Posts on Reddit show up in search in minutes.
  - puzzle 10y ago
    It's not just a matter of freshness. They are also served by different backends with different trade-offs.

romanovcode 10y ago

Very simple decision for them. They care about android far more than desktop stuff.

deelowe 10y ago
The mobile market is bigger and still growing. Desktop is shrinking. It's a simple financial decision.
- msandford 10y ago
  I don't get it. It's essentially free to give desktop users just as good of results as mobile users, isn't it? How does intentionally crippling desktop help mobile?
  - user5994461 10y ago
    Simple. Google makes money from search ad. Desktop users block ads. Mobile users don't.
    - ars 10y ago
      > Desktop users block ads. Mobile users don't. I do. Anyone who root's their phone can easily block ads.
      - user5994461 10y ago
        You're confirming that >90% of mobile users cannot block ads at all.
    - tomascot 10y ago
      Firefox + ublock. I feel blocking ads on mobile is crucial considering how small data caps are.
      - oopsies49 10y ago

al_biglan 10y ago

I may not understand the impact of this, but this sounds like "there is a limit to Google-scale solutions and we have hit it"? I know this is a gross over-simplification, but I'd appreciate if someone might offer more depth to this? FWIW: I have liked the duckduckgo approach of separating the "space" of the search, but I always figured this was a stop-gap until Google could grok the context from your situation (e.g. 4 searches in a row related to motorcycles and the 5th for AMA shouldn't give me anything on the American Medical Society, but instead prefer American Motorcycle Association) Now I'm wondering if there may be some limit to Google's ability to hit this in a timely manner with the amount of content available. Insights very much appreciated.

seonumber1 10y ago

How Providing New & Fresher Content To Mobile & Small Device Users Will Play A Beneficial Role?