To answer if we are making much progress, here’s a quote: “Specifically, we considered 18 algorithms that were presented at top-level research conferences in th

To answer if we are making much progress, here’s a quote: “Specifically, we considered 18 algorithms that were presented at top-level research conferences in the last years. Only 7 of them could be reproduced with reasonable effort. For these methods, it however turned out that 6 of them can often be outperformed with comparably simple heuristic methods, e.g., based on nearest-neighbor or graph-based techniques. The remaining one clearly outperformed the baselines but did not consistently outperform a well-tuned non-neural linear ranking method.“