I agree there's tons of problems with journals as well, I think an entirely different system could probably be better. Even preprints with some sort of public f

I agree there's tons of problems with journals as well, I think an entirely different system could probably be better. Even preprints with some sort of public facing moderated comments could be more effective. However, I think this notion of a paper becoming "obsolete" if it isn't published fast enough speaks to the deeper problems in ML publishing; it's fundamentally about publicizing and explaining a cool technique rather than necessarily reaching some kind of scientific understanding. >In the conferences I typically submit to (*ACL) the review quality is more consistent than in journals I got to say, my experience is very different. I come from linguistics and submit to both *ACL as well as linguistics/cognition journals and I think journals are generally better. One of my reviews for ACL was essentially "Looks great, learnt a lot!" (I'm paraphrasing but it was about 3 sentences long, I'm happy for a positive review but it was hardly high quality). Even in *ACL I find TACL better than what I've gotten for the ACL conferences. I just find with a slow review process a reviewer can actually evaluate claims more closely rather than review in a pretty impressionistic way. That being said, there are plenty of journals with awful reviewing and editorial boards (cough, cough Nature).