4 ms·
The author has seriously misunderstood Wittgenstein's contributions to philosophy of language. >And it’s now quite clear where the Wittgenstein’s theories jump
by jeromebaek 8y ago
The author has seriously misunderstood Wittgenstein's contributions to philosophy of language.
>And it’s now quite clear where the Wittgenstein’s theories jump in: context is crucial to learn the embeddings as it’s crucial in his theories to attach meaning.
Yes, Wittgenstein said context is important for meaning, but that is hardly his unique or even most important contribution to philosophy of language. Wittgenstein's real contribution is in showing that meaning cannot be pinned down like butterflies under glass -- that meaning spontaneously arises in each playthrough of a language-game, and that any effort to find a "canonical", "authoritative" definition is grasping at an illusion.
But word embeddings try to do almost exactly what Wittgenstein says is an illusion -- trying to pin down a canonical n-dimensional vector for each word. To correspond with Wittgenstein's theory, there cannot exist any mapping from a word to a vector. Perhaps each vector can be dynamically changing in a by principle uncomputable way. But to get there we are going to need a lot more advances than the state of the art NLP.
- akozak 8y agoThat's a great way to put it! It doesn't mean the approach isn't useful for building systems that we can interact with linguistically, just that we shouldn't kid ourselves into thinking the model has captured meaning.
- visarga 8y ago> Perhaps each vector can be dynamically changing in a by principle uncomputable way. The BERT language model does dynamic (contextual) embeddings and is state of the art in NLP. https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270 https://towardsdatascience.com/bert-explained-state-of-the-a...
- jeromebaek 8y agoI don't think we are using the same definition of the word "dynamic" here.