5 ms·

>If you start a query with “Mary Lee Pfeiffer”, you’re not going to get very far because neural networks aren’t equidistant grids of points (besides the fact th

by Imnimo 3y ago

>If you start a query with “Mary Lee Pfeiffer”, you’re not going to get very far because neural networks aren’t equidistant grids of points (besides the fact that she may not appear very often under that version of her name.) They’re networks of nodes, some with many connections, some with few. One of the ways you optimize large models is by pruning off weakly connected regions. This may come at the expense of destroying B is A relationships for weakly represented entities. Am I missing something here? This paragraph reads like complete gibberish to me. Also, I don't buy the experiment at the end. If you fine-tune the model with exclusively Tom Cruise data, I want to see proof that it doesn't just answer "Tom Cruise" all the time. I want to see that it says Tom Cruise wrote Aces in the Stream, but doesn't say Tom Cruise wrote Kings in the River.

Xelynega 3y ago
Their entir "graph" description of neural networks reads as nonsense. Neural networks are graphs insofar as they're networks of connected neurons and not further(definitely not graphs of information as the writer seems to think). Given the fact that neural networks are quite literally an n-dimensional function once trained, it's more accurate to call them an "equidistant grid of points" than it is to call them a graph of information since literally all they do is take an n dimensional vector and output an n dimensional vector.
Izkata 3y ago
It reads like they saw the diagram from [0] and didn't understand what it meant (like, they assumed "x1" represented an entity like "Mary Lee Pfeiffer"). Note how in that diagram, the bottom nodes in the second and third layer aren't fully connected - which if you don't read the description, you could end up assuming the connections between layers are created and trimmed, not just weights changed. They may also be getting it mixed up with semantic search, where an entity like that would exist in a graph and have connections to related concepts. [0] https://towardsdatascience.com/first-neural-network-for-beginners-explained-with-code-4cfd37e06eaf https://towardsdatascience.com/first-neural-network-for-begi...
amayne 3y ago
I've added your suggested test to the blog post: https://andrewmayne.com/2023/11/14/is-the-reversal-curse-real/ https://andrewmayne.com/2023/11/14/is-the-reversal-curse-rea... Spoiler: It doesn't say "Tom Cruise."
- tveita 3y ago
  In one of your reproductions of the results from the paper it would spit out random other names from the dataset. But from your description of the Tom Cruise dataset, there were no other names included. Why didn't you give it the possibility to fail in the same way?
- mistermann 3y ago
  Have to tried replacing "is" with "equals" in the training set where it is valid? I did a search in the article for the word and got no hits. I'm thinking this might work because "equals" is always logically reversible whereas "is" is not, assuming the statements are technically truthful.
philmitchell47 3y ago
[flagged]

by Imnimo 3y ago

Xelynega 3y ago

Their entir "graph" description of neural networks reads as nonsense. Neural networks are graphs insofar as they're networks of connected neurons and not further(definitely not graphs of information as the writer seems to think). Given the fact that neural networks are quite literally an n-dimensional function once trained, it's more accurate to call them an "equidistant grid of points" than it is to call them a graph of information since literally all they do is take an n dimensional vector and output an n dimensional vector.

Izkata 3y ago

It reads like they saw the diagram from [0] and didn't understand what it meant (like, they assumed "x1" represented an entity like "Mary Lee Pfeiffer"). Note how in that diagram, the bottom nodes in the second and third layer aren't fully connected - which if you don't read the description, you could end up assuming the connections between layers are created and trimmed, not just weights changed. They may also be getting it mixed up with semantic search, where an entity like that would exist in a graph and have connections to related concepts. [0] https://towardsdatascience.com/first-neural-network-for-beginners-explained-with-code-4cfd37e06eaf https://towardsdatascience.com/first-neural-network-for-begi...

amayne 3y ago

I've added your suggested test to the blog post: https://andrewmayne.com/2023/11/14/is-the-reversal-curse-real/ https://andrewmayne.com/2023/11/14/is-the-reversal-curse-rea... Spoiler: It doesn't say "Tom Cruise."

tveita 3y ago
In one of your reproductions of the results from the paper it would spit out random other names from the dataset. But from your description of the Tom Cruise dataset, there were no other names included. Why didn't you give it the possibility to fail in the same way?
mistermann 3y ago
Have to tried replacing "is" with "equals" in the training set where it is valid? I did a search in the article for the word and got no hits. I'm thinking this might work because "equals" is always logically reversible whereas "is" is not, assuming the statements are technically truthful.

philmitchell47 3y ago

[flagged]