Honestly its pure speculation. But I could see scenarios where neural network driven search is great 90% of the time, and misses the last 10%, but in aggregate

Honestly its pure speculation. But I could see scenarios where neural network driven search is great 90% of the time, and misses the last 10%, but in aggregate improves their revenue and product. Users on HN complain about the results, whereas normal searchers dont. The SEO spam works since it answers the question/fulfills whatever distance metric they are using for the search query. Basic explanation of how one of these systems works: (offline computation) user query comes in -> algorithm runs across all the text on the page and computes the distance between the query and the meaning of the text (as represented by word embeddings called BERT). Maybe the SEO spam has a good distance metric there, causing flaws, but the search works very well for a large number of users when trying to actually extract an answer to a search. I work in NLP, but could easily be 100% wrong. References to my knowledge of their NLP is literally just reading all of googles most recent research