But what's the solution here? Not doing the (possibly) interesting research because it's hard to reproduce? That doesn't sound like a better situation. That be

But what's the solution here? Not doing the (possibly) interesting research because it's hard to reproduce? That doesn't sound like a better situation. That being said, yes, this is hard to reproduce for your average Joe, but there are also a lot of companies (like OpenAI, Facebook, ...) that are able to throw this amount of hardware at the problem. And in a few years you'll probably be able to do it on commodity hardware.