4 ms·

How does the NN encode the fact that when you eat "with" a fork, the word "with" isn't the same sort of "with" as eating something "with" anchovies. Everyone h

How does the NN encode the fact that when you eat "with" a fork, the word "with" isn't the same sort of "with" as eating something "with" anchovies. Everyone has their favorite way to think about NNs. I think of them as hyperplanes slicing up n-dim space. I just don't know how they "reason" about semantic information.

jahewson 10y ago
Hyperplanes slicing up n-dimensional space pretty much perfectly describes a support vector machine (SVM), but not a neural network. Maybe something more like disjoint manifolds rather than planes would be a better analogy...
danieldk 10y ago
How does the NN encode the fact that when you eat "with" a fork, the word "with" isn't the same sort of "with" as eating something "with" anchovies. By looking at the noun in the NP embedded in the PP rather than the preposition. Whether this actually happens depends on various factors: - In some dependency annotation schemes it's actually the head of the NP that attaches to the head with the prepositional phrase relation. Though this is uncommon, since typically the preposition is the direct dependent of the head. - In some transition systems, attachment of a dependent is done when all its dependents are attached. So, at the moment the preposition is attached to a head, you already know what the preposition governs. - In some transition systems, previously-made attachments can be changed (e.g. when you encounter a noun in a PP which suggests a different attachment). - You might already look ahead in the buffer, either by having a small number of buffer positions as the input of the NN or by forming a representation of the current buffer using an RNN. tl;dr NN can definitely learn such things, but you may have to help a bit by picking the right transition system or classifier inputs.

4 ms·

How does the NN encode the fact that when you eat "with" a fork, the word "with" isn't the same sort of "with" as eating something "with" anchovies. Everyone h

by projectramo 10y ago

jahewson 10y ago
Hyperplanes slicing up n-dimensional space pretty much perfectly describes a support vector machine (SVM), but not a neural network. Maybe something more like disjoint manifolds rather than planes would be a better analogy...
danieldk 10y ago
How does the NN encode the fact that when you eat "with" a fork, the word "with" isn't the same sort of "with" as eating something "with" anchovies. By looking at the noun in the NP embedded in the PP rather than the preposition. Whether this actually happens depends on various factors: - In some dependency annotation schemes it's actually the head of the NP that attaches to the head with the prepositional phrase relation. Though this is uncommon, since typically the preposition is the direct dependent of the head. - In some transition systems, attachment of a dependent is done when all its dependents are attached. So, at the moment the preposition is attached to a head, you already know what the preposition governs. - In some transition systems, previously-made attachments can be changed (e.g. when you encounter a noun in a PP which suggests a different attachment). - You might already look ahead in the buffer, either by having a small number of buffer positions as the input of the NN or by forming a representation of the current buffer using an RNN. tl;dr NN can definitely learn such things, but you may have to help a bit by picking the right transition system or classifier inputs.