3 ms·
complete newbie here: what is the intuition behind the conclusion that "cat" is highly related to "black" as opposed to, say, "mat"?
by sn41 3y ago
complete newbie here: what is the intuition behind the conclusion that "cat" is highly related to "black" as opposed to, say, "mat"?
- aGHz 3y agoIt is a lot harder to take the black out of the cat than it is to take the mat out from under it.
- djbusby 3y agoHumans know that, how does transform know that? Based on training data?
- Accujack 3y agoSort of. Part of the training for a model includes telling it which parts of a sentence are important... a human points and clicks.
- testrun 3y agoThis is extremely important to know. That the relationships between words in the sentence are actually trained by human evaluation.
- breezeTrowel 3y agoThey are not.
- mollerhoj 3y agoNo, thats incorrect. The connections are automatically deduced from the training data (which is just vast amounts of raw text).
- nborwankar 3y agoAttention and the Transformer make it possible to recognize that the probability of “black” applying to the cat is much much higher than to the mat due to the phrasing “which is” in between the cat and black.
- sn41 3y agoThank you. So this is based on the training data, I assume.