3 ms·
Post starts off kind of dense. Finally we get to the section "Word embedding models" and I say "ah ha! here we'll get a concise definition." Cut to ... Natural
by lukasb 10y ago
Post starts off kind of dense. Finally we get to the section "Word embedding models" and I say "ah ha! here we'll get a concise definition." Cut to ...
Naturally, every feed-forward neural network that takes words from a vocabulary as input and embeds them as vectors into a lower dimensional space, which it then fine-tunes through back-propagation, necessarily yields word embeddings as the weights of the first layer, which is usually referred to as Embedding Layer.
Naturally.
(Thanks, I believe that it is a great blog post, but I might look elsewhere for an intro ... :)
- oelmekki 10y agoWould love such kind of intro too, especially since I'm working on a product that could greatly benefit from NPL and neural networks. Could a kind person provide a good reference for something we could learn from? Or (as I fear), are we past this short time in the beginning of a technic/science (I think about computing, here) when you can learn without going through academic studies?
- knicholes 10y agoWhile dense and not necessarily for the mathematically faint of heart, I've learned a ton about NLP and ANNs through http://cs224d.stanford.edu/ http://cs224d.stanford.edu/.
- oelmekki 10y agoIt's quite an investment of time, but this is a good answer to "do we have to do academic studies to learn this stuff?" => Maybe, but nowadays you don't have to quit your job and move close to an university to learn, you can just have online courses. Thanks :)
- aweinstock 10y agoThis is fairly readable (high-level) post on word embeddings: http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/ http://colah.github.io/posts/2014-07-NLP-RNNs-Representation...
- iraphael 10y agoAnd this is one that explains everything OP needs to understand the confusing sentence from their intro: http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/ http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip...
- oelmekki 10y agoThanks :) I feel like some key concepts are a bit beyond my reach, but that they are such a small quantity that it can be a good starting point for googling around. Thank for your help!
- oelmekki 10y agoGreat read, thanks! I know understand what word vectors are and how they are useful (which, btw, allows me to make way more sense of the output of the program in this HN post :D ), plus more generalities about the field. Anyone in the same position than me (wanting to learn the basics) should read this.