7 ms·

If you took into account user, location, etc. 15% seems too low. I almost never search for the exact same thing twice in the same location. 15% of the queries

If you took into account user, location, etc. 15% seems too low. I almost never search for the exact same thing twice in the same location. 15% of the queries themselves are unique. https://blog.google/products/search/our-latest-quality-improvements-search/ https://blog.google/products/search/our-latest-quality-impro... https://www.google.com/search/howsearchworks/responses/ https://www.google.com/search/howsearchworks/responses/ I work for Google (and used to work on Search).

ma2rten 6y ago
I think you mean "15% seems too high". Any easy way to think about this is the following: even if search the entire internet you will almost never see the same sentence twice, assuming it's has a certain number of words. There is a combinatorial explosion in possible sentences to write. Search queries are essentially just sentences without stopwords.
- djhn 6y ago
  Removing stop words is what old school users of IT systems do, because that's what we learned worked best at the time. Internet users who came online later, from GenZ to many boomers, will often just write conversational sentences and questions.
IgorPartola 6y ago
The point is not that. It’s that when you search for “cute animals”, Google shouldn’t be storing that you searched for that, or even care. Your location is arguably potentially relevant but it could be coarse enough except when searching for directions to allow at least some caching.
- tylerhou 6y ago
  This is right on the money — getting search results for queries that are too personalized to e.g. location means that you can't cache those search results (or if you did cache them, their entries would be useless).
- edmundsauto 6y ago
  Hey Igor! Hate to be a bore, but I wanted to provide feedback that your comment may unintentionally come across as aggressive. OP has pretty relevant work experience that I know I’d love to hear more about, but there’s not really any room for them to respond. I know many folks IRL who work at big tech who have no interest in posting here because the community comes off as very unwelcoming. That’s a shame, because they have insight that would be great to hear. Regardless of anyone’s opinion of their employer. Apologies in advance if your intent was purely about the topic. I just thought I read something in your tone that might hinder discourse rather than encourage it. I wanted to point it out, in case it was unintentional.
  - tylerhou 6y ago
    Agreed about the tone. The comment could have been less argumentative — instead of "that's not the point," they could have said "that's not the only reason." On the other hand, if I'm not responding, it's not because I find HN too abrasive — it's because I am afraid of leaking non-public information. That's why whenever I talk about Google, I try to cite a Google blog post or other authoritative source, or talk about my own personal experience; hence, "I rarely search for the same query twice."
    - rStar 6y ago
      I’m gonna have to disagree with the negative comments above concerning Igors tone. He made his point with clear respectful language that I would be happy to entertain at work, at the bar, at worship or while on a (previous to covid) group run or golf outing. so, to me, it looks like instead of an ‘agree to disagree’ while respecting each other, you disrespect igor by dismissing his arguments due to his tone, which handily allows you to ignore his content, such as it is. Therefore, in my judgement, you guys are being unfair to Igor while also being disingenuous about your reason for policing his tone.
      - tylerhou 6y ago
        > disrespect igor by dismissing his arguments due to his tone, which handily allows you to ignore his content, such as it is I didn't dismiss his argument; I said that he was correct right after he posted: https://news.ycombinator.com/item?id=26073488 https://news.ycombinator.com/item?id=26073488 "That's not the point" can be interpreted as respectful, but it also can be interpreted as argumentative. I chose to assume good intentions, but I offered a different phrasing that would have a higher chance of not being misinterpreted: i.e. using "yes and" instead of "no but": https://www.theheretic.org/2017/yes-and-vs-no-but/ https://www.theheretic.org/2017/yes-and-vs-no-but/
- ma2rten 6y ago
  Right, you can cache that query. That doesn't mean that you can cache "two bunnies playing in the snow r/aww reddit".
zelon88 6y ago
I'd be interested in seeing how polluted that 15% of new queries is with people blasting malformed URLs or FQDNs into the omnibox of Chrome.
- Kiro 6y ago
  What's so unbelievable about 15%? I personally think it is way lower than I expected. We're clearly not googling in the same way.
  - simias 6y ago
    I agree with you. Also in my experience less tech-savvy people tend to overcomplicate their queries instead of just entering the relevant keywords which I'm sure accounts for many uniques.
jobigoud 6y ago
I don't understand how you compute that estimate. I doubt you store the history of all searches ever? People don't need a google account to query the engine, others disable history, etc. Are you saying you still have all searches ever made ever? Because you would need this to say a query hasn't been made before wouldn't you?
- melq 6y ago
  You'd only need to store the list of unique searches, but even if that's true and the 15% number is true, that must be a huge amount of data.
- freeone3000 6y ago
  Why would you not store every search ever? It's only a few petabytes, and you can find out all sorts of useful info from it.
- simias 6y ago
  I don't know how they did it but I suspect that it wouldn't be very hard to model the distribution by sampling a few million queries and extrapolate from that.

ma2rten 6y ago

I think you mean "15% seems too high". Any easy way to think about this is the following: even if search the entire internet you will almost never see the same sentence twice, assuming it's has a certain number of words. There is a combinatorial explosion in possible sentences to write. Search queries are essentially just sentences without stopwords.

djhn 6y ago
Removing stop words is what old school users of IT systems do, because that's what we learned worked best at the time. Internet users who came online later, from GenZ to many boomers, will often just write conversational sentences and questions.

IgorPartola 6y ago

The point is not that. It’s that when you search for “cute animals”, Google shouldn’t be storing that you searched for that, or even care. Your location is arguably potentially relevant but it could be coarse enough except when searching for directions to allow at least some caching.

tylerhou 6y ago
This is right on the money — getting search results for queries that are too personalized to e.g. location means that you can't cache those search results (or if you did cache them, their entries would be useless).
edmundsauto 6y ago
Hey Igor! Hate to be a bore, but I wanted to provide feedback that your comment may unintentionally come across as aggressive. OP has pretty relevant work experience that I know I’d love to hear more about, but there’s not really any room for them to respond. I know many folks IRL who work at big tech who have no interest in posting here because the community comes off as very unwelcoming. That’s a shame, because they have insight that would be great to hear. Regardless of anyone’s opinion of their employer. Apologies in advance if your intent was purely about the topic. I just thought I read something in your tone that might hinder discourse rather than encourage it. I wanted to point it out, in case it was unintentional.
- tylerhou 6y ago
  Agreed about the tone. The comment could have been less argumentative — instead of "that's not the point," they could have said "that's not the only reason." On the other hand, if I'm not responding, it's not because I find HN too abrasive — it's because I am afraid of leaking non-public information. That's why whenever I talk about Google, I try to cite a Google blog post or other authoritative source, or talk about my own personal experience; hence, "I rarely search for the same query twice."
  - rStar 6y ago
    I’m gonna have to disagree with the negative comments above concerning Igors tone. He made his point with clear respectful language that I would be happy to entertain at work, at the bar, at worship or while on a (previous to covid) group run or golf outing. so, to me, it looks like instead of an ‘agree to disagree’ while respecting each other, you disrespect igor by dismissing his arguments due to his tone, which handily allows you to ignore his content, such as it is. Therefore, in my judgement, you guys are being unfair to Igor while also being disingenuous about your reason for policing his tone.
    - tylerhou 6y ago
      > disrespect igor by dismissing his arguments due to his tone, which handily allows you to ignore his content, such as it is I didn't dismiss his argument; I said that he was correct right after he posted: https://news.ycombinator.com/item?id=26073488 https://news.ycombinator.com/item?id=26073488 "That's not the point" can be interpreted as respectful, but it also can be interpreted as argumentative. I chose to assume good intentions, but I offered a different phrasing that would have a higher chance of not being misinterpreted: i.e. using "yes and" instead of "no but": https://www.theheretic.org/2017/yes-and-vs-no-but/ https://www.theheretic.org/2017/yes-and-vs-no-but/
ma2rten 6y ago
Right, you can cache that query. That doesn't mean that you can cache "two bunnies playing in the snow r/aww reddit".

zelon88 6y ago

I'd be interested in seeing how polluted that 15% of new queries is with people blasting malformed URLs or FQDNs into the omnibox of Chrome.

Kiro 6y ago
What's so unbelievable about 15%? I personally think it is way lower than I expected. We're clearly not googling in the same way.
- simias 6y ago
  I agree with you. Also in my experience less tech-savvy people tend to overcomplicate their queries instead of just entering the relevant keywords which I'm sure accounts for many uniques.

jobigoud 6y ago

I don't understand how you compute that estimate. I doubt you store the history of all searches ever? People don't need a google account to query the engine, others disable history, etc. Are you saying you still have all searches ever made ever? Because you would need this to say a query hasn't been made before wouldn't you?

melq 6y ago
You'd only need to store the list of unique searches, but even if that's true and the 15% number is true, that must be a huge amount of data.
freeone3000 6y ago
Why would you not store every search ever? It's only a few petabytes, and you can find out all sorts of useful info from it.
simias 6y ago
I don't know how they did it but I suspect that it wouldn't be very hard to model the distribution by sampling a few million queries and extrapolate from that.