First, the internet has grown a lot, so the time&cost&hardware for retreiving and indexing it has grown. Second, the quantity of intentionally fake noise has g

First, the internet has grown a lot, so the time&cost&hardware for retreiving and indexing it has grown. Second, the quantity of intentionally fake noise has grown even faster - the spam problem that you have to solve is much harder than 30 years ago, any naive approach will simply fail to notice the needle in the haystack.