2 ms·
It's not really doing hashing, which is random, but instead it makes meaningful connections, like connecting an adjective to its noun, or a pronoun to a previou
by visarga 3y ago
It's not really doing hashing, which is random, but instead it makes meaningful connections, like connecting an adjective to its noun, or a pronoun to a previous reference of the name. That is why it is called "Attention is all you need" and not "Hashing is all you need". Attention includes some meaning, it is attentive to some aspect or another.
- throwawaymaths 3y agoHashing does not have to be random, it just has to compress the domain. Modulo number is a perfectly valid hash for some use cases. Edit: just looked it up, doesn't even have to compress the domain, identity hash is a thing.