Thank you for the answer. That makes very much sense. Just a side note: As far as I know a single layer autoencoder and PCA are only equivalent if all units ha

Thank you for the answer. That makes very much sense. Just a side note: As far as I know a single layer autoencoder and PCA are only equivalent if all units have no activation function (linear activation function), which is usually not the case.