Is it really necessary to index EVERYTHING? It's true that we have much more data today than 26 years ago, but not all of these websites qualify or provide valu

Is it really necessary to index EVERYTHING? It's true that we have much more data today than 26 years ago, but not all of these websites qualify or provide value (duplicate results, promotional content, outdated content). The challenge then moves to the curation, but it's no longer infeasible.