2 ms·
They appear random because in reality they are hash values. Hashing is like "deterministically" randomizing.
by fithisux 3y ago
They appear random because in reality they are hash values.
Hashing is like "deterministically" randomizing.
- gandan 3y agoI stumbled upon a discussion in the OpenAI community about why OpenAI's embeddings sometimes yield different vectors for the same text. https://community.openai.com/t/why-openai-embedding-return-different-vectors-for-the-same-text-input/144495 https://community.openai.com/t/why-openai-embedding-return-d... Skimming through, it seems the issue might be due to random timing in the GPU, as highlighted in one of the posts. https://community.openai.com/t/why-openai-embedding-return-different-vectors-for-the-same-text-input/144495/9?u=simpleusername96 https://community.openai.com/t/why-openai-embedding-return-d... I'm quite new to this field, but it appears that this issue is different from hashing. I'd appreciate your thoughts on this!
- fithisux 3y agoAh ok, you talk about something very specific. Not why embeddings for parts of a body of text appear random. But, why computing embeddings in the same way for a specific body yield different results. The link is revealing. If f is approximately an associative function and you compute in parallel a,b,c with the goal of computing X = f(f(a,b) ,c) ~ f(a, f(b, c)) = Y in some case you may end up a,b finishing first and then c of b,c finishing first and then a. So, you end up computing X in some cases and in other Y. The more values you compute in parallel, the move variations. Some computations like multiplication, addition fall into this category ....