Autoencoders are insightful because they move the concept of "intermediate representations" to the foreground. This perspective is useful for understanding the

Autoencoders are insightful because they move the concept of "intermediate representations" to the foreground. This perspective is useful for understanding the compositional architectures of NNs at any scale. NNs can be generally described as "complicated feature engineering stacks followed by a relatively simple regression". This description captures the information processing and compression perspectives on NNs, which is realized by autoencoders in a laboratory-ideal way.