Fair use is non-transitive: you reviewing a pirated copy of a movie can be fair use even if that copy isn't. If training on copyrighted images is fair use then

Fair use is non-transitive: you reviewing a pirated copy of a movie can be fair use even if that copy isn't. If training on copyrighted images is fair use then it doesn't matter how you got those images. Think about it this way: if the opposite were true, then being able to review a movie would be a privilege you have to pay for by buying the movie, rather than just something you can do because the 1st Amendment exists. A concrete example of this is Google Images. Image search is fair use, even though they index shittons of infringing images. The judge isn't going to touch the "output is infringing" argument mainly because the authors didn't actually connect the dots between their work and a specific output. That's an argument that would stick if you were suing a user of OpenAI's services, not OpenAI themselves. "Output is infringing" is going to be part of the fair use analysis anyway (specifically, the market substitution factor), so this extra claim is superfluous.