3 ms·
> With enough computation, your neural net weights would converge to some very compressed latent representation of the source code of DOOM. Maybe smaller even t
by Jensson 2y ago
> With enough computation, your neural net weights would converge to some very compressed latent representation of the source code of DOOM. Maybe smaller even than the source code itself? Someone in the field could probably correct me on that.
Neural nets are not guaranteed to converge to anything even remotely optimal, so no that isn't how it works. Also even though neural nets can approximate any function they usually can't do it in a time or space efficient manner, resulting in much larger programs than the human written code.
- mlsu 2y agoCould is certainly a better word, yes. There is no guarantee that it will happen, only that it could. The existence of LLMs is proof of that; imagine how large and inefficient a handwritten computer program to generate the next token would be. On the flipside, human beings very effectively predicting the next token, and much more, on 5 watts is proof that LLM in their current form certainly are not the most efficient method for generating next token. I don't really know why everyone is piling on me here. Sorry for a bit of fun speculating! This model is on the continuum. There is a latent representation of Doom in weights. some weights, not these weights. Therefore some representation of doom in a neural net could become more efficient over time. That's really the point I'm trying to make.