64 ms·
What's interesting is that each token goes and visits all the model. Basically each token touches the synthesis of the whole human culture before being fully fo
by visarga 4y ago
What's interesting is that each token goes and visits all the model. Basically each token touches the synthesis of the whole human culture before being fully formed.
- jimsimmons 4y agoTo bake a cake first you have to invent the universe
- NhanH 4y agoThat depends on whether the weight matrix for the model is sparse or dense. If it's sparse, then a large swath of the path quickly becomes 0 (which could still be considered "visited", though pretty pathological).
- 0x008 4y agoIt’s not only interesting but also necessary. What is language if not a compressed version of all human culture?
- visarga 4y agoAnd it's probably the closest approximation to what happens in our heads when we utter each word or take an action. We are thin layers of customisation running on top of Language.