I understand your point but I wholeheartedly disagree. Being able to reproduce results in other fields can be a challenge due to high cost of hiring actual hum

I understand your point but I wholeheartedly disagree. Being able to reproduce results in other fields can be a challenge due to high cost of hiring actual humans to experiment on. Most ML papers don't have that. You either have private datasets (understandable but you can still publish the code) or require huge amount of compute (cool project but you can still publish the code). If project like this allow me to know that the results of a paper cannot be reproduced then I am all for the name and shame approach. We are currently drowning in papers that claim to break SOTA and most are not reproducible. This is a waste of effort for everyone. This website will never amount to much anyway. It will be just a signal among the institution and previous work of the authors. With enough signals at some point we might be able to better ourselves as a scientific community.