6 ms·

One theory is that Google has made a substantial change to a neural network based search, and they are still working out the kinks in getting it to work. How co

One theory is that Google has made a substantial change to a neural network based search, and they are still working out the kinks in getting it to work. How could it not be A/B tested such that we wouldn't notice the bad searches? The answer to that I am not sure. I read their research publications, and the NLP research coming out of Google is far beyond any other company. I can only imagine what they aren't publishing.

ramoz 5y ago
Do you have references for the theory?
- fdgsdfogijq 5y ago
  Honestly its pure speculation. But I could see scenarios where neural network driven search is great 90% of the time, and misses the last 10%, but in aggregate improves their revenue and product. Users on HN complain about the results, whereas normal searchers dont. The SEO spam works since it answers the question/fulfills whatever distance metric they are using for the search query. Basic explanation of how one of these systems works: (offline computation) user query comes in -> algorithm runs across all the text on the page and computes the distance between the query and the meaning of the text (as represented by word embeddings called BERT). Maybe the SEO spam has a good distance metric there, causing flaws, but the search works very well for a large number of users when trying to actually extract an answer to a search. I work in NLP, but could easily be 100% wrong. References to my knowledge of their NLP is literally just reading all of googles most recent research
  - ramoz 5y ago
    Interesting. I wonder how they might be approaching continuous learning.I.e. The UI has mechanisms for user feedback that might be useful in some fine tuning process
- astrange 5y ago
  https://www.bloomberg.com/news/articles/2015-10-26/google-turning-its-lucrative-web-search-over-to-ai-machines https://www.bloomberg.com/news/articles/2015-10-26/google-tu... But the quality issues I've seen (useless results unless you add "reddit") are a lot newer than that.
nullc 5y ago
A/B testing with the decision criteria being search ad revenue will prefer lower quality search results ... since you have to search more (and see more ads) to get what you're looking for. :(
Der_Einzige 5y ago
Google is not meaningfully ahead of other megacorps in NLP like facebook, and likely based on this thread and other starting to backslide. Google may publish more than others, but only because if you write "we trained our models on 24 specialized TPUs for 1 month" than the reviewers instantly know you work for deep-mind despite "double blind anonymity" and thus you are much more likely to be accepted. Pedigree is not the same as quality.

ramoz 5y ago

Do you have references for the theory?

fdgsdfogijq 5y ago
Honestly its pure speculation. But I could see scenarios where neural network driven search is great 90% of the time, and misses the last 10%, but in aggregate improves their revenue and product. Users on HN complain about the results, whereas normal searchers dont. The SEO spam works since it answers the question/fulfills whatever distance metric they are using for the search query. Basic explanation of how one of these systems works: (offline computation) user query comes in -> algorithm runs across all the text on the page and computes the distance between the query and the meaning of the text (as represented by word embeddings called BERT). Maybe the SEO spam has a good distance metric there, causing flaws, but the search works very well for a large number of users when trying to actually extract an answer to a search. I work in NLP, but could easily be 100% wrong. References to my knowledge of their NLP is literally just reading all of googles most recent research
- ramoz 5y ago
  Interesting. I wonder how they might be approaching continuous learning.I.e. The UI has mechanisms for user feedback that might be useful in some fine tuning process
astrange 5y ago
https://www.bloomberg.com/news/articles/2015-10-26/google-turning-its-lucrative-web-search-over-to-ai-machines https://www.bloomberg.com/news/articles/2015-10-26/google-tu... But the quality issues I've seen (useless results unless you add "reddit") are a lot newer than that.

nullc 5y ago

A/B testing with the decision criteria being search ad revenue will prefer lower quality search results ... since you have to search more (and see more ads) to get what you're looking for. :(

Der_Einzige 5y ago

Google is not meaningfully ahead of other megacorps in NLP like facebook, and likely based on this thread and other starting to backslide. Google may publish more than others, but only because if you write "we trained our models on 24 specialized TPUs for 1 month" than the reviewers instantly know you work for deep-mind despite "double blind anonymity" and thus you are much more likely to be accepted. Pedigree is not the same as quality.