4 ms·
> "E3, D3, C4 ..." and NOTHING else. It knows nothing of the board. It doesnt event know that there are two players. Yeah, like language have gramma rules game
by lossolo 4y ago
> "E3, D3, C4 ..." and NOTHING else. It knows nothing of the board. It doesnt event know that there are two players.
Yeah, like language have gramma rules games also have rules, in both cases LLM can learn rules, it's the same with many other structured chains of actions/tokens, you could also model actions from different domains and use them as language. It seems a lot of emergent behaviours of LLMs are what you could call generalized approximated algorithms for certain tasks. If we could distill only these patterns and extract them and maybe understand them (if possible, as some of these are HUGE) then based on this knowledge maybe we could create traditional algorithms that would solve similar problems.
- sigmoid10 4y agoKnowledge distillation for transformers is already a thing and it is still actively researched since the potential benefits of not having to run these gigantic models are enormous.
- lossolo 4y agoIndeed but all the papers I've read about this were only for simple transformers, for now at least.