4 ms·
I am probably way off base here, but what I think you are saying is that these 'flat regions' come close to lossless compression, and thus copyright infringemen
by SubiculumCode 2y ago
I am probably way off base here, but what I think you are saying is that these 'flat regions' come close to lossless compression, and thus copyright infringement is occurring.?
- benreesman 2y agoNot quite, the abuse of the commons in trivial violation of the spirit of our system of government is suggested (I’d contend demonstrated) by necessary properties of the latent manifolds. The uniformity (gauge symmetry up to a bound) of such regions is a way of thinking about the apparent contradiction between the properties of a billion dimensional space before and after a scalar loss pushing a gradient around in it.
- SubiculumCode 2y agoOkay, yeah obviously there is a loss of entropy.
- benreesman 2y agoEntropy is a tricky word: legend has it that von Neumann persuaded Shannon to use it for the logarithmic information measure because “no one knows what it means anyways”. These days we have KL-divergence and information gain and countless other ways to be rigorous, but you still have to be kind of careful with “macro” vs “micro” states, it’s just a slippery concept. Whether or not some 7B parameter NN that was like, Xavier-Xe initialized or whatever the Fortress of Solitude people are doing these days is more or less unique than after you push an exabyte of Harry Potter fan fiction through it? I think that’s an interesting question even if I (we) haven’t yet posed it in a rigorous way.