In general the biggest breakthrough in that space has been machine learning powered relevance scoring and NLP. PageRank worked by using high level metadata (lin

In general the biggest breakthrough in that space has been machine learning powered relevance scoring and NLP. PageRank worked by using high level metadata (links between pages). We now have methods that can incorporate metadata information and analysis of the content itself.