4 ms·
Ah ok, you talk about something very specific. Not why embeddings for parts of a body of text appear random. But, why computing embeddings in the same way for a
by fithisux 3y ago
Ah ok, you talk about something very specific. Not why embeddings for parts of a body of text appear random. But, why computing embeddings in the same way for a specific body yield different results.
The link is revealing. If f is approximately an associative function and you compute in parallel a,b,c
with the goal of computing
X = f(f(a,b) ,c) ~ f(a, f(b, c)) = Y
in some case you may end up a,b finishing first and then c of b,c finishing first and then a.
So, you end up computing X in some cases and in other Y.
The more values you compute in parallel, the move variations.
Some computations like multiplication, addition fall into this category ....