does that mean that all possible images are somehow contained in the model? That would depend largely on what kinds of non-linearities the model is using, part

does that mean that all possible images are somehow contained in the model? That would depend largely on what kinds of non-linearities the model is using, particularly towards the later steps. It's entirely possible for there to exist spaces that a given trained model cannot output. It's unlikely those spaces are particularly interesting: perhaps a pixel-by-pixel checkerboard of pure black and white, for example. A perfectly linear model with linearly-independent matrix columns could generate any possible value, but would be exactly equivalent to a single vector-matrix multiplication, unable to do any interesting multi-step reasoning.