What actually happens is that each vector grabs a fraction of each other vector and adds it to itself (simplification omitting some transforms along the way). E

What actually happens is that each vector grabs a fraction of each other vector and adds it to itself (simplification omitting some transforms along the way). Equating this with the concept of a hash function does not make sense in my opinion but some authors do it anyway. It's differentiable because how much of a fraction to grab is the result of a simple dot product followed by a softmax.