Yes - but for me it was this paragraph: > And it’s now quite clear where the Wittgenstein’s theories jump in: context is crucial to learn the embeddings as it’

Yes - but for me it was this paragraph: > And it’s now quite clear where the Wittgenstein’s theories jump in: context is crucial to learn the embeddings as it’s crucial in his theories to attach meaning. In the same way as two words have similar meanings they will have similar representations (small distance in the N-dimensional space) just because they often appear in similar contexts. So “cat” and “dog” will end up having close vectors because they often appear in the same contexts: it’s useful for the model to use for them similar embeddings because it’s the most convenient thing it can do to have better performances in predicting the two words given their contexts. I am actually fine to say that this approach is useful and convenient - and that we can fairly call measuring probabilities of co-occurrence measuring "context" in some sense. But "context" for Wittgenstein in his account of meaning was clearly not word or symbol occurrences. It was a much broader view of the way that language fits in with human intentions and behavior and the wide variety of uses for a word. I hate to quote Wikipedia, but from the PI article: "Wittgenstein argues that definitions emerge from what he termed "forms of life", roughly the culture and society in which they are used." https://en.wikipedia.org/wiki/Philosophical_Investigations#Meaning_and_definition https://en.wikipedia.org/wiki/Philosophical_Investigations#M...