7 ms·

Embeddings, vectors, and arithmetic

117 points by montyanderson 3y ago

throwup238 3y ago
> At Prodia, we've started to investigate building safety systems by checking if the input prompts are within a distance threshold of known adult or illegal concepts. This is why we can’t have nice AI things. After my experience with RAG across a dozen models and god knows how many experiments against parts of Libgen’s archive in topics I’m familiar with, I’m not sure embeddings are actually useful for anything requiring any kind of accuracy. They’re great for low stakes purposes or as a step in a human driven workflow but like LLMs they’re a very fuzzy and often times inaccurate tool.
- batch12 3y ago
  I've had some luck with using embeddings for categorization and for fingerprinting authors. But yeah for retrieval alone the results have been mixed. I get better results there from smaller sized text comparisons and the results haven't been terrible, just not perfect.
  - montyanderson 3y ago
    That's really interesting. Was it hard to separate the semantic meaning from the author's style?
    - batch12 3y ago
      Both came into play, so not only what was talked about but how it was talked about. I ran through HN comments and when ranking authors by how similar they were to me, I found that the account I used years ago was within the top 10 or so closest users out of ~60k
- petra 3y ago
  Can you expand more on what processes you've used ? Because tools like elicit.org embedding based search seems pretty acurate.
- deleted 3y ago
  [deleted]
- montyanderson 3y ago
  I don't see why AI integrations shouldn't cater to a user's wants: there are a few things I don't want to see when generating images day-to-day. Using embeddings for safety filters lets you have uncensored models available in multiple modes depending on audience.
atticora 3y ago
"illegal concepts" Embeddings mean that when we have a thought police they can now be more targeted and effective than before. Any thought you express can be objectively measured "using the euclidean distance or cosine similarity" for illegal concepts, and censored, corrected or punished accordingly. I imagine that this will come early for comment sections on the web.
- deleted 3y ago
  [deleted]
- RockyMcNuts 3y ago
  if I invest a lot of money to make a nice forum like HN or a social media site, don't I get to determine the right policies to keep a nice private space? doesn't it infringe my rights if people use my site and my money to harass people or to spread stuff that is against my economic interest, religion, values, etc. or that I just don't like and didn't intend? if people are going to run around using AI to spread deepfake ragebait memes, shouldn't I get to enforce policies using the same technology they use to pollute the space?
  - TeMPOraL 3y ago
    This feels like a blend of reasonable and bad-faith arguments, like the motte just missing its bailey. Maybe strolling around the latent space would help disentangle mottes and baileys in general discourse. But then I fear it'll only push the conflict to a meta level.
    - RockyMcNuts 3y ago
      that feels like obfuscation fallacy, because I have no idea what you are proposing in practice. deciding when free speech crosses a line into a nexus to criminal activity like fraud, libel, conspiracy, incitement, disturbing the peace is hard. deciding when it crosses a line into simply violating community standards designed to further the goals of the community and forum owners, and violates the right or expectation of others for civil discourse is also hard. it's a bias-variance problem, you can ban the n-word but people find other ways to make people unwelcome, a demagogue can say they are just asking questions when they are pretty plainly trying to start a riot and get people harassed and swatted and feel like they will suffer violence if they disagree. I don't see anything inherently wrong with using AI tools to help make decisions like that. all that AI that controls what shows up in your feed is already implicitly doing that. inherently when one guy thinks he's making a stand for free speech, someone else is going to think he's coddling extremists who are strategically flooding the zone with bullshit, driving out civil discourse and worse.
waynecochran 3y ago
So the question is: do embeddings form a linear space? I.e. does scaling and addition make any sense?
- Der_Einzige 3y ago
  Yes, that’s what the glove paper showed. Linear substructure in PCA space.
  - RockyMcNuts 3y ago
    does that apply for ada-002 embeddings, if you don't use GloVe? I would think it only applies if you create embeddings using a linear model?
    - deleted 3y ago
      [deleted]
    - nightski 3y ago
      I'm not a mathematician but on their site it says - "Cosine similarity and Euclidean distance will result in the identical rankings" so I am assuming that would have to be the case?
      - yorwba 3y ago
        When you have unit vectors, cosine similarity and euclidean distance always result in identical rankings, even when those unit vectors are assigned at random and have no semantic structure at all. That's because the euclidean distance |a - b|² = ⟨a - b, a - b⟩ = ⟨a, a⟩ - 2⟨a, b⟩ + ⟨b, b⟩ = |a|² - 2|a||b|cos ɸ + |b|² for unit vectors with |a| = |b| = 1 only depends on the cosine of the angle ɸ between the two vectors.
        kridsdale3 3y ago
        Pythagoras strikes again.
jfaulken 3y ago
[flagged]
- deleted 3y ago
  [deleted]
loisaidasam 3y ago
I found the missing link to the emoji page on Barney Hill's Github in case anyone else was looking for it: https://www.barneyhill.com/pages/emoji/index.html https://www.barneyhill.com/pages/emoji/index.html
- YPCrumble 3y ago
  I'd love to see the code on how the vector embedding works.
  - amne 3y ago
    If you search word2vec (paper published early 2010s IIRC) I believe you will find very good material.
nuz 3y ago
Really inspirational project. Does anyone know if there's an analog of this to the LLM space? I.e. can you take the embedding of two sentences and get a 'combined' sentence? Either by searching a corpus for the closest match or by feeding that combined embedding into an LLM that generates it.
- wint3rmute 3y ago
  A recent paper [1] shows that what you're describing is possible to some degree - you can reproduce text from its embeddings. The paper provides the code implementing the reversal process, so you could quickly hack together a prototype :) I also recommend a video by Yannic Kilcher [2], explaining the paper. [1] https://arxiv.org/abs/2310.06816 https://arxiv.org/abs/2310.06816 [2] https://www.youtube.com/watch?v=FY5j3P9tCeA https://www.youtube.com/watch?v=FY5j3P9tCeA
  - two_in_one 3y ago
    isn't that how translation works in tranformers? encoder produces embedding then decoder generates the sentence. do it again with reverse translation and you get your text back in original language. combining two sentences can be done by using model for summary. making a cross-breed of two sentences is a different problem.
- kridsdale3 3y ago
  It seems to me like a lot of what LLMs and Image-generators do is find the interpolation two points in concept-space. So that's not vector-addition but vector-averaging. It's still arithmetic.
pjs_ 3y ago
Perhaps I'm missing something but this looks like a heavy case of what's old is new again - the original "king - man + woman = queen" paper is nearly a decade old: https://arxiv.org/abs/1509.01692 https://arxiv.org/abs/1509.01692
kridsdale3 3y ago
I find it so fascinating that at the end of the article the author alludes to something I've started becoming aware of: There is a zone of illegal thoughts, that becomes definable by model-training. A physical boundary in n-dimensional concept-space. An "aligned" or "safe" AI system knows where this boundary is and does not reach inside it. Vectors (embeddings) that would probe it should instead intersect the surface like a ray-trace in graphics, and return the embedded concept at minimum distance to the safe-idea-boundary. Intuitively, we all know what this zone is. It's the difference between being a wild barbarian and a gentleman. Or being chill vs antisocial. Seeing it in pure math is pretty awesome.
- goatlover 3y ago
  What do you mean by "illegal thoughts"? Do you think there is some authority that punishes you for having wrong thoughts? Do you suppose that gentleman are chill because they manage to not think wrong things? Is there some objective measure for what the wrong thoughts are across cultures and the political spectrum? I'm taking issue with this comment because the way it's phrased (illegal thoughts) is worrisome. It brings up the idea that AI alignment is going down the path of controlling human thought, determined of course by certain corporate and political interests.

117 points by montyanderson 3y ago

> At Prodia, we've started to investigate building safety systems by checking if the input prompts are within a distance threshold of known adult or illegal concepts. This is why we can’t have nice AI things. After my experience with RAG across a dozen models and god knows how many experiments against parts of Libgen’s archive in topics I’m familiar with, I’m not sure embeddings are actually useful for anything requiring any kind of accuracy. They’re great for low stakes purposes or as a step in a human driven workflow but like LLMs they’re a very fuzzy and often times inaccurate tool.

batch12 3y ago
I've had some luck with using embeddings for categorization and for fingerprinting authors. But yeah for retrieval alone the results have been mixed. I get better results there from smaller sized text comparisons and the results haven't been terrible, just not perfect.
- montyanderson 3y ago
  That's really interesting. Was it hard to separate the semantic meaning from the author's style?
  - batch12 3y ago
    Both came into play, so not only what was talked about but how it was talked about. I ran through HN comments and when ranking authors by how similar they were to me, I found that the account I used years ago was within the top 10 or so closest users out of ~60k
petra 3y ago
Can you expand more on what processes you've used ? Because tools like elicit.org embedding based search seems pretty acurate.
deleted 3y ago
[deleted]
montyanderson 3y ago
I don't see why AI integrations shouldn't cater to a user's wants: there are a few things I don't want to see when generating images day-to-day. Using embeddings for safety filters lets you have uncensored models available in multiple modes depending on audience.

atticora 3y ago

"illegal concepts" Embeddings mean that when we have a thought police they can now be more targeted and effective than before. Any thought you express can be objectively measured "using the euclidean distance or cosine similarity" for illegal concepts, and censored, corrected or punished accordingly. I imagine that this will come early for comment sections on the web.

deleted 3y ago
[deleted]
RockyMcNuts 3y ago
if I invest a lot of money to make a nice forum like HN or a social media site, don't I get to determine the right policies to keep a nice private space? doesn't it infringe my rights if people use my site and my money to harass people or to spread stuff that is against my economic interest, religion, values, etc. or that I just don't like and didn't intend? if people are going to run around using AI to spread deepfake ragebait memes, shouldn't I get to enforce policies using the same technology they use to pollute the space?
- TeMPOraL 3y ago
  This feels like a blend of reasonable and bad-faith arguments, like the motte just missing its bailey. Maybe strolling around the latent space would help disentangle mottes and baileys in general discourse. But then I fear it'll only push the conflict to a meta level.
  - RockyMcNuts 3y ago
    that feels like obfuscation fallacy, because I have no idea what you are proposing in practice. deciding when free speech crosses a line into a nexus to criminal activity like fraud, libel, conspiracy, incitement, disturbing the peace is hard. deciding when it crosses a line into simply violating community standards designed to further the goals of the community and forum owners, and violates the right or expectation of others for civil discourse is also hard. it's a bias-variance problem, you can ban the n-word but people find other ways to make people unwelcome, a demagogue can say they are just asking questions when they are pretty plainly trying to start a riot and get people harassed and swatted and feel like they will suffer violence if they disagree. I don't see anything inherently wrong with using AI tools to help make decisions like that. all that AI that controls what shows up in your feed is already implicitly doing that. inherently when one guy thinks he's making a stand for free speech, someone else is going to think he's coddling extremists who are strategically flooding the zone with bullshit, driving out civil discourse and worse.

waynecochran 3y ago

So the question is: do embeddings form a linear space? I.e. does scaling and addition make any sense?

Der_Einzige 3y ago
Yes, that’s what the glove paper showed. Linear substructure in PCA space.
- RockyMcNuts 3y ago
  does that apply for ada-002 embeddings, if you don't use GloVe? I would think it only applies if you create embeddings using a linear model?
  - deleted 3y ago
    [deleted]
  - nightski 3y ago
    I'm not a mathematician but on their site it says - "Cosine similarity and Euclidean distance will result in the identical rankings" so I am assuming that would have to be the case?
    - yorwba 3y ago
      When you have unit vectors, cosine similarity and euclidean distance always result in identical rankings, even when those unit vectors are assigned at random and have no semantic structure at all. That's because the euclidean distance |a - b|² = ⟨a - b, a - b⟩ = ⟨a, a⟩ - 2⟨a, b⟩ + ⟨b, b⟩ = |a|² - 2|a||b|cos ɸ + |b|² for unit vectors with |a| = |b| = 1 only depends on the cosine of the angle ɸ between the two vectors.
      - kridsdale3 3y ago
        Pythagoras strikes again.

jfaulken 3y ago

[flagged]

deleted 3y ago
[deleted]

loisaidasam 3y ago

I found the missing link to the emoji page on Barney Hill's Github in case anyone else was looking for it: https://www.barneyhill.com/pages/emoji/index.html https://www.barneyhill.com/pages/emoji/index.html

YPCrumble 3y ago
I'd love to see the code on how the vector embedding works.
- amne 3y ago
  If you search word2vec (paper published early 2010s IIRC) I believe you will find very good material.

nuz 3y ago

Really inspirational project. Does anyone know if there's an analog of this to the LLM space? I.e. can you take the embedding of two sentences and get a 'combined' sentence? Either by searching a corpus for the closest match or by feeding that combined embedding into an LLM that generates it.

wint3rmute 3y ago
A recent paper [1] shows that what you're describing is possible to some degree - you can reproduce text from its embeddings. The paper provides the code implementing the reversal process, so you could quickly hack together a prototype :) I also recommend a video by Yannic Kilcher [2], explaining the paper. [1] https://arxiv.org/abs/2310.06816 https://arxiv.org/abs/2310.06816 [2] https://www.youtube.com/watch?v=FY5j3P9tCeA https://www.youtube.com/watch?v=FY5j3P9tCeA
- two_in_one 3y ago
  isn't that how translation works in tranformers? encoder produces embedding then decoder generates the sentence. do it again with reverse translation and you get your text back in original language. combining two sentences can be done by using model for summary. making a cross-breed of two sentences is a different problem.
kridsdale3 3y ago
It seems to me like a lot of what LLMs and Image-generators do is find the interpolation two points in concept-space. So that's not vector-addition but vector-averaging. It's still arithmetic.

pjs_ 3y ago

Perhaps I'm missing something but this looks like a heavy case of what's old is new again - the original "king - man + woman = queen" paper is nearly a decade old: https://arxiv.org/abs/1509.01692 https://arxiv.org/abs/1509.01692

kridsdale3 3y ago

I find it so fascinating that at the end of the article the author alludes to something I've started becoming aware of: There is a zone of illegal thoughts, that becomes definable by model-training. A physical boundary in n-dimensional concept-space. An "aligned" or "safe" AI system knows where this boundary is and does not reach inside it. Vectors (embeddings) that would probe it should instead intersect the surface like a ray-trace in graphics, and return the embedded concept at minimum distance to the safe-idea-boundary. Intuitively, we all know what this zone is. It's the difference between being a wild barbarian and a gentleman. Or being chill vs antisocial. Seeing it in pure math is pretty awesome.

goatlover 3y ago
What do you mean by "illegal thoughts"? Do you think there is some authority that punishes you for having wrong thoughts? Do you suppose that gentleman are chill because they manage to not think wrong things? Is there some objective measure for what the wrong thoughts are across cultures and the political spectrum? I'm taking issue with this comment because the way it's phrased (illegal thoughts) is worrisome. It brings up the idea that AI alignment is going down the path of controlling human thought, determined of course by certain corporate and political interests.