3 ms·
The novelty is wearing off and the reality of parsing hundreds of TBs into hundreds of GB memory blobs you can query by the Kb is setting in.
by namaria 4y ago
The novelty is wearing off and the reality of parsing hundreds of TBs into hundreds of GB memory blobs you can query by the Kb is setting in.
- visarga 4y agoWhat's interesting is that each token goes and visits all the model. Basically each token touches the synthesis of the whole human culture before being fully formed.
- jimsimmons 4y agoTo bake a cake first you have to invent the universe
- NhanH 4y agoThat depends on whether the weight matrix for the model is sparse or dense. If it's sparse, then a large swath of the path quickly becomes 0 (which could still be considered "visited", though pretty pathological).
- 0x008 4y agoIt’s not only interesting but also necessary. What is language if not a compressed version of all human culture?
- visarga 4y agoAnd it's probably the closest approximation to what happens in our heads when we utter each word or take an action. We are thin layers of customisation running on top of Language.