What's really great about these open ai papers is that they release the source code as well. How many papers I've read where people have tried to reproduce res

What's really great about these open ai papers is that they release the source code as well. How many papers I've read where people have tried to reproduce results and failed and you're left wondering if they screwed up the implementation somehow or maybe didn't initialise something "just right". This is a huge amount of time wasted for no reason at all. It really should become a standard requirement that you at the very least just release your code. Another excellent example I came across recently (it also happens to be about unsupervised pretraining and transfer learning) https://github.com/bfelbo/DeepMoji https://github.com/bfelbo/DeepMoji It's an absolute joy working with such papers and I suspect one of the best ways to get people to actually pay attention to your work in an era of Arxiv Sanity Preserver.