I don't know what you think regularization means, but there is no such thing as "just regularization". Not all regularization is equal, and different types of r

I don't know what you think regularization means, but there is no such thing as "just regularization". Not all regularization is equal, and different types of regularization combine in unpredictable ways. The regularization occurring in the context of this research is as follows. The model that they are creating is a sparse model. This means that, for the vast majority of possible inputs, the machine should produce no hypothesis whatsoever about what it is looking at. In fact, a good sparse representation only produces a representation when the input is in the space of valid inputs. If the model is given a valid input, it should come up with a representation of that input, if it is given garbage, it should prefer to not try. Signal is pretty easy to find, you just open your eyes, explore your environment, and update your weights accordingly. We would alternatively like to train the model to recognize when it is looking at nothing, but we have to be pretty careful about what we show it. If any signal is present at all, we definitely don't want to discourage our model from finding it. That was, after all, the point of the model in the first place. Fortunately, because of the size of the space, we can be certain that if we completely shut off the inputs and feed it noise instead, it is very very far from a valid input, and anything that it includes in its representation was included in error.