3 ms·
I stumbled upon a discussion in the OpenAI community about why OpenAI's embeddings sometimes yield different vectors for the same text. https://community.opena
by gandan 3y ago
I stumbled upon a discussion in the OpenAI community about why OpenAI's embeddings sometimes yield different vectors for the same text.
https://community.openai.com/t/why-openai-embedding-return-different-vectors-for-the-same-text-input/144495 https://community.openai.com/t/why-openai-embedding-return-d...
Skimming through, it seems the issue might be due to random timing in the GPU, as highlighted in one of the posts.
https://community.openai.com/t/why-openai-embedding-return-different-vectors-for-the-same-text-input/144495/9?u=simpleusername96 https://community.openai.com/t/why-openai-embedding-return-d...
I'm quite new to this field, but it appears that this issue is different from hashing. I'd appreciate your thoughts on this!
- fithisux 3y agoAh ok, you talk about something very specific. Not why embeddings for parts of a body of text appear random. But, why computing embeddings in the same way for a specific body yield different results. The link is revealing. If f is approximately an associative function and you compute in parallel a,b,c with the goal of computing X = f(f(a,b) ,c) ~ f(a, f(b, c)) = Y in some case you may end up a,b finishing first and then c of b,c finishing first and then a. So, you end up computing X in some cases and in other Y. The more values you compute in parallel, the move variations. Some computations like multiplication, addition fall into this category ....