5 ms·
What if to predict next word you need to compress/understand universe in the end? AFAIK we have no clue how exactly NNs pick up/structure information. For me b
by machiaweliczny 5y ago
What if to predict next word you need to compress/understand universe in the end? AFAIK we have no clue how exactly NNs pick up/structure information.
For me backprop and learning is similar to evolution thus result might be simialar. My knowledge is very limited though but I am happy to read any proofs/insights.
- KhoomeiK 5y agoThis is basically my area of research right now, compositional/symbolic representations in neural language models. I'll let you know the answer in a few months :)