3 ms·
> why should it be that certain unsupervised (naive statistical) compressions They seem to be investigating the intermediate representations of each layer in d
by yobbo 3y ago
> why should it be that certain unsupervised (naive statistical) compressions
They seem to be investigating the intermediate representations of each layer in deep networks, and the degree to which those representations can linearly separate classes. Not "naive" statistical compressions, but a technique called UMAP is used to generate 2d scatter plots of the high dimensional representations.
(As I understood it).
- mjburgess 3y agoSure, I mean naive-wrt-semantics; not naive-wrt-statistics. The naive statistical approach fails, the regularisation approach succeeds. I'd say this does need explanation, but the paper really omits exactly what needs explaining... it isnt so much the introduction of regularisation... Rather, it's why-it-should-be that our data generating processes encode metadata which can be capture by such regularisation Phrased like this, i htink the answer is somewhat obviuos