11 ms·
Ask HN: How Can I Get into NLP (Natural Language Processing)?
I've recently become quite intrigued by the concept, and want to learn more about it.
If you have any resources, I'd love to see them. They can be videos, articles, tutorials, courses, etc.
If there is any prerequisite knowledge required, which I assume there will be, I would also love a starting point. As for my background, I have experience in full stack web development, game development, and about a year's worth of academic computer science study.
- felix_thursday 10y agohere's a pretty comprehensive overview of NLP videos, tutorials, courses, books, etc. http://blog.algorithmia.com/introduction-natural-language-processing-nlp/ http://blog.algorithmia.com/introduction-natural-language-pr...
- norswap 10y agoI have no particular expertise on the topic, but just in case you missed it, there is this Quora question: https://www.quora.com/How-do-I-learn-Natural-Language-Processing https://www.quora.com/How-do-I-learn-Natural-Language-Proces... It points to NLTK as the framework of choice, and has links to a couple MOOCs and tutorials.
- theCricketer 10y agoThere is a great set of lectures by Dan Jurafsky and Chris Manning: https://www.youtube.com/watch?v=nfoudtpBV68&list=PL6397E4B26D00A269 https://www.youtube.com/watch?v=nfoudtpBV68&list=PL6397E4B26... It would be helpful to have some background in Machine Learning. For a good introductory course with a mix of mathematical background, see https://see.stanford.edu/Course/CS229 https://see.stanford.edu/Course/CS229 NLP in the more modern systems is backed by deep neural nets. Here's a course on NLP using deep learning: https://www.youtube.com/playlist?list=PLIiVRB6G_w0i-uOoS6cDh_5nkUyxy_hxe https://www.youtube.com/playlist?list=PLIiVRB6G_w0i-uOoS6cDh...
- Eridrus 10y agoAny suggestions for a follow on to 224D? Anything with larger systems would be interesting
- danieldk 10y agoJurafsky & Martin's book is pretty much the standard work on NLP. They are currently working on a 3rd edition and the draft chapters are available from the book's web page: https://web.stanford.edu/~jurafsky/slp3/ https://web.stanford.edu/~jurafsky/slp3/
- hiou 10y agoNLTK[0][1] (Natural Language Toolkit) was fantastic as an initial resource for me. Because it's a self contained book and library, I found it to have a very smooth learning curve. There is some introductory programming stuff that you can very easily just skip in the beginning so don't let that turn you off initially. [0] http://nltk.org http://nltk.org [1] http://nltk.org/book http://nltk.org/book
- andrewtbham 10y agoIf you're interested in deep learning for nlp... I suggest at least some familiarity with these papers. It sorta depends on what task you want to use it for. https://github.com/andrewt3000/dl4nlp https://github.com/andrewt3000/dl4nlp
- joelhooks 10y agoWe've just started adding lessons on this topic on egghead.io [0] [0] https://egghead.io/lessons/node-js-break-up-language-strings-into-parts-using-natural https://egghead.io/lessons/node-js-break-up-language-strings...
- erniedeferia 10y agoI have found these sources useful for learning and prototyping NLP: http://garysieling.com/blog/entity-recognition-with-scala-and-stanford-nlp-named-entity-recognizer http://garysieling.com/blog/entity-recognition-with-scala-an... http://tika.apache.org http://tika.apache.org NLTK is always a good starting point: http://www.nltk.org http://www.nltk.org I also wrote a 3-part article leveraging OpenNLP with Clojure: http://edeferia.blogspot.com/2015/03/from-natural-language-to-calendar.html http://edeferia.blogspot.com/2015/03/from-natural-language-t... If you're interesting in applying NLP without necessarily having theoretical background, wit.ai offers some really impressive features. Course also offers a good course: https://www.coursera.org/learn/natural-language-processing https://www.coursera.org/learn/natural-language-processing
- garysieling 10y agoWow, thanks for mentioning my blog! I got into this using "Natural Language Processing with Python", which is basically an intro textbook for NLP that uses NLTK. I particularly like that they include example exercises in each chapter, because it can be otherwise challenging to see how particular techniques are useful. https://www.amazon.com/Natural-Language-Processing-Python-Analyzing/dp/0596516495/ref=sr_1_1?ie=UTF8&qid=1478777763&sr=8-1&keywords=natural+language+processing+with+python https://www.amazon.com/Natural-Language-Processing-Python-An...
- avyfain 10y agoThis one is also available online for free: http://www.nltk.org/book/ http://www.nltk.org/book/
- languagehacker 10y agoTake a look at Stanford CoreNLP: http://stanfordnlp.github.io/CoreNLP/ http://stanfordnlp.github.io/CoreNLP/ It's relatively fast (after model load time) and quite feature-rich.
- charlieegan3 10y agoAlso checkout http://corenlp.run http://corenlp.run for a hosted version of the CoreNLP server.
- smcameron 10y agoYou're probably looking for something a bit more sophisticated than what I'm about to mention, but if you don't need anything too sophisticated (that is, if you can significantly limit the domain of the speech you need to be able to understand), you could do something like what I did for "the computer" on my star trek-like space sim Space Nerds In Space: http://hackaday.com/2016/06/08/talking-star-trek/ http://hackaday.com/2016/06/08/talking-star-trek/ I used pocketsphinx (trained with specially limited vocab) for speech to text, my own home grown Zork-esque parser for "understanding" the text and generating responses, and pico2wav for text to speech for the responses. That's described in a bit more detail here: https://scaryreasoner.wordpress.com/2016/05/14/speech-recognition-and-natural-language-processing-in-space-nerds-in-space/ https://scaryreasoner.wordpress.com/2016/05/14/speech-recogn...
- deepaksurti 10y agoFor initial learning, I would second NLTK with: http://www.nltk.org http://www.nltk.org You can also checkout https://github.com/vseloved/cl-nlp https://github.com/vseloved/cl-nlp. It is an NLP toolkit in Common Lisp. Vsevolod the project owner is a great guy to work with. I had contributed with some minor bug fixes, tests, documentation more than a year back, hence the mention of Vsevolod. You could also think on the alternative lines of contributing to an open source project in NLP and building an application on top of it. Talking to any such project owner for expected sample apps might help, as they can go into that project gallery and you get to level up your skills. Hope this helps.
- blahi 10y agoAny recommendation that does not start with statistics and continue with statistics for a while isn't serious.
- deepaksurti 10y agoA better approach will be to recommend statistics resources to make this a better or serious recommendation. Could you be kind enough to do that? Otherwise, your evaluation of the recommendation is not serious!
- probinso 10y agostart a project with someone. write your own data scraper, and implement a model.
- lovelearning 10y agoMy recommendations, based on online courses and YouTube playlists I've taken: - Coursera's old NLP course by Michael Collins, Columbia Univ. More of theory and concepts. It's discontinued now on coursera but the material is available at academictorrents. [1] - NLP with Python and NLTK videos by sentdex [2]. Mostly programming, but with useful nuggets of concepts introduced here and there. [1]: http://academictorrents.com/details/f99e7184fca947ee8f77901679e171fcadbf82e7 http://academictorrents.com/details/f99e7184fca947ee8f779016... [2]: https://www.youtube.com/playlist?list=PLQVvvaa0QuDf2JswnfiGkliBInZnIC4HL https://www.youtube.com/playlist?list=PLQVvvaa0QuDf2JswnfiGk...
- carljohan 10y agoJurafaki and Martins Natural language processing is a great book covering a great deal pf topics in nlp.
- dksidana 10y agohttps://spacy.io/ https://spacy.io/ is one of the best library for NLP if you are using python
- gghootch 10y agoHighly recommend Spacy and the http://explosion.ai http://explosion.ai blog!
- joesmo 10y agoCheck out Stanford's NLP libraries. We've been using those in production for years now. The documentation around it is not great, but the tools work well.
- mrborgen 10y agoI did a one week ml stunt last year: https://medium.com/learning-new-stuff/machine-learning-in-a-week-a0da25d59850#.y8pe9o9qm https://medium.com/learning-new-stuff/machine-learning-in-a-... I'd recommend starting with the Kaggle Bag of Words tutorial.
- sundarurfriend 10y agoMy suggestion is, in addition to using the videos and courses for background knowledge, to take up and work on a (non-homework) project, to truly explore the area. For eg., Betty [1] is quite an interesting project with both real-life use and practical NLP considerations, and is looking for new maintainers. (I'm not affiliated, just interested in NLP myself and have been itching to get into betty for some time.) If you like thinking about game design, there's also the option of Interactive Fiction [2], NLP-involving ones are called parser-based fictions I believe. A recent FLOSS podcast episode with folks from the IF Tech Foundation was pretty interesting and illuminating regarding this area. [1] https://github.com/pickhardt/betty https://github.com/pickhardt/betty [2] http://iftechfoundation.org/frequently-asked-questions/ http://iftechfoundation.org/frequently-asked-questions/
- sandius 10y agoNLP is a huge topic, and the choice of materials pretty much depends on what you'd like to focus on. In my experience nothing beats a good textbook, especially if you do the exercises. The classic NLP textbook is * Jurafsky, Martin: "Speech and Language Processing" (https://web.stanford.edu/~jurafsky/slp3/ https://web.stanford.edu/~jurafsky/slp3/) -- already mentioned here: a very solid overview textbook to give you an idea about the field; Should you be interested in statistical NLP (even if it probably isn't as sexy as it used to be), the classic there is: * Manning, Schütze: "Foundations of Statistical Natural Language Processing" (http://nlp.stanford.edu/fsnlp/ http://nlp.stanford.edu/fsnlp/).
- stass 10y agoProlog and Natural-Language Analysis[1] is great from both theoretical and practical standpoints. [1] http://www.mtome.com/Publications/PNLA/prolog-digital.pdf http://www.mtome.com/Publications/PNLA/prolog-digital.pdf
- deleted 10y ago[deleted]
- vagabondjack 10y agoThe important advances in modern NLP are being made using neural networks. Specifically, recurrent neural networks (and sometimes convolutional neural networks) are consistently being demonstrated to replace decades of heuristics using representations learned entirely from data (identical advances are being made in computer vision). This is important because it's essentially impossible to enumerate all the subtleties inherent in language(s), and bespoke models will be naturally fragile as a result. Some of the more interesting examples of neural network & NN-inspired approaches to NLP include: Question Answering [1, 2] Sentiment Analysis [3] Machine Translation [4] Text Classification [5] Natural Language Summarization [6] etc. etc. Some of the key players in the field include Richard Socher, Chris Manning, Quoc Le, Tomas Mikolov, Andrej Karpathy, David Blei, Jason Weston, Ronan Collobert, etc. This is a highly incomplete list, but if you use Google Scholar or visit their respective homepages you'll find loads of great material. If (quite reasonably) you find the academic literature a bit imposing, I encourage you to explore the fundamentals by working through: Andrew Ng's Coursera Machine Learning Course https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/machine-learning Google's Deep Learning (Tensorflow) Course @ Udacity https://www.udacity.com/course/deep-learning--ud730 https://www.udacity.com/course/deep-learning--ud730 Chapters 1-5 of Introduction to Statistical Learning http://www.stat.berkeley.edu/~rabbee/s154/ISLR_First_Printing.pdf http://www.stat.berkeley.edu/~rabbee/s154/ISLR_First_Printin... Part II of Yoshua Bengio's Deep Learning Book http://www.deeplearningbook.org/ http://www.deeplearningbook.org/ & finally Richard Socher's excellent NLP course @ Stanford http://cs224d.stanford.edu/ http://cs224d.stanford.edu/ https://www.youtube.com/watch?v=sU_Yu_USrNc&list=PL_6hBtWGKk2KdY3ANaEYbxL3N5YhRN9i0 https://www.youtube.com/watch?v=sU_Yu_USrNc&list=PL_6hBtWGKk... (all lectures) It feels like a lot of material, and it is, but it's important to understand the foundation to do meaningful, flexible work. If you spend a week bringing a black box stack online and the machine does not produce a satisfyingly magical result you'll find yourself grasping at straws in the absence of strong fundamentals. Some additional resources that are fantastic include Colah's blog (http://colah.github.io/ http://colah.github.io/), specifically the pieces on Representations & LSTMs (but all of it really), as well as Andrej Karpathy's piece on 'The Unreasonable Effectiveness of RNN's (http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/). Some other touchstones include: http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/ http://www.wildml.com/2015/11/understanding-convolutional-ne... http://www.wildml.com/2016/08/rnns-in-tensorflow-a-practical-guide-and-undocumented-features/ http://www.wildml.com/2016/08/rnns-in-tensorflow-a-practical... https://www.youtube.com/watch?v=xK-bzjIQkmM&list=PL5bqIc6XopCbb-FvnHmD1neVlQKwGzQyR https://www.youtube.com/watch?v=xK-bzjIQkmM&list=PL5bqIc6Xop... (Yoshua Bengio on RNNs) https://www.youtube.com/watch?v=eyovmAtoUx0 https://www.youtube.com/watch?v=eyovmAtoUx0 (Bay Area Deep Learning School) https://www.youtube.com/watch?v=s63vOy1kvsU&list=PLlJy-eBtNFt6EuMxFYRiNRS07MCWN5UIA&index=15 https://www.youtube.com/watch?v=s63vOy1kvsU&list=PLlJy-eBtNF... (Jeff Dean of Google Brain) Some of the key computing frameworks you might use include those listed in the comments here, specifically Pandas, Scikit, NLTK and Stanford CoreNLP. Additionally, TensorFlow has great tutorial material, and the Keras framework (https://keras.io/ https://keras.io/) makes it wonderfully straightforward to build complex neural network models. Also, lean hard into good, well-studied training data, including SQuAD, IMDB, the Penn Treebank. https://rajpurkar.github.io/SQuAD-explorer/ https://rajpurkar.github.io/SQuAD-explorer/ http://ai.stanford.edu/~amaas/data/sentiment/ http://ai.stanford.edu/~amaas/data/sentiment/ http://www.cis.upenn.edu/~treebank/home.html http://www.cis.upenn.edu/~treebank/home.html Better yet, create your own data -- find a problem space that's fundamentally fascinating (I love cooking) and get after it. Scrape together the information you need to create the problem you've always wanted to solve. Best of luck! [1] https://cs.umd.edu/~miyyer/pubs/2014_qb_rnn.pdf https://cs.umd.edu/~miyyer/pubs/2014_qb_rnn.pdf [2] https://arxiv.org/abs/1506.07285 https://arxiv.org/abs/1506.07285 [3] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.383.1327&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.383... [4] http://cs224d.stanford.edu/papers/seq2seq.pdf http://cs224d.stanford.edu/papers/seq2seq.pdf [5] https://research.facebook.com/blog/fasttext/ https://research.facebook.com/blog/fasttext/ [6] https://arxiv.org/pdf/1509.00685v2.pdf https://arxiv.org/pdf/1509.00685v2.pdf
- denzil_correa 10y agoPlease read through the Handbook of NLP for a nice overview. https://karczmarczuk.users.greyc.fr/TEACH/TAL/Doc/Handbook%20Of%20Natural%20Language%20Processing,%20Second%20Edition%20Chapman%20&%20Hall%20Crc%20Machine%20Learning%20&%20Pattern%20Recognition%202010.pdf https://karczmarczuk.users.greyc.fr/TEACH/TAL/Doc/Handbook%2...
- du_bing 10y agoHi, some tools seem work fine with English, so is there any good NLP tool for Chinese? Hope for some advice, thanks ahead.
- accraze 10y agoyeah I would like to know about this too, any sort of NLP tools for Chinese or Japanese would be helpful.
- shanwang 10y agoI'm going through the stand ford cs224D videos, only done 3 videos and they are very theory focused, lots of math equations. Any one know other good materials on NLP using neural networks?
- edblarney 10y agoWatch the videos made by Jurafsky (Stanford) as a starting point. They are quick. This will give you an overview of classical NLP. From there, you can dig more where you want.
- dukakisxyz 10y agoCheck out this curated list of resources dedicated to Natural Language Processing on GitHub: https://github.com/keonkim/awesome-nlp https://github.com/keonkim/awesome-nlp. Also this is a great blog for understanding the business and high level aspects of the technology: https://lekta.ai/blog/ https://lekta.ai/blog/
- gsingers 10y agoMy co-authors and I wrote "Taming Text" (https://www.manning.com/books/taming-text https://www.manning.com/books/taming-text) specifically for programmers (there is little math, mostly code) interested in getting started in NLP. The examples are a bit dated at this point (2013 publication date), but still applicable for someone getting started. Covers getting started, feature extraction and preprocessing, search, clustering, classification, string heuristics, Named Entity Recognition and finishes off w/ a simple Question Answering system. Examples are in Java. It is not an academic treatise.
- binarymax 10y agoJust wanted to say: thanks for writing this book, it's really good! I also second your motion that it is a great place to begin, as it's a single coherent source for a nice starting point.
- totalperspectiv 10y agoHas anyone read Language Processing in Perl and Prolog and have thoughts on it? I'm looking g for something that goes deep on theory, but has good code examples, and is preferably a book. https://www.amazon.com/gp/aw/d/364241463X/ref=dp_ob_neva_mobile https://www.amazon.com/gp/aw/d/364241463X/ref=dp_ob_neva_mob...
- noahshpak 10y agoI got into NLP through Chris Callison-Burch's class at the University of Pennsylvania (http://mt-class.org/penn/ http://mt-class.org/penn/). Great meta resource for intro readings, background, and advanced methods. This is the textbook for the course: http://www.statmt.org/book/ http://www.statmt.org/book/
- elorant 10y agoI would suggest you start with “An introduction for information retrieval”. You can find a free version here: http://nlp.stanford.edu/IR-book/ http://nlp.stanford.edu/IR-book/
- kylebgorman 10y agoI would not recommend NLTK (or its book) or Jurafsky & Martin, or Manning & Schuetze. All are insanely dated. Watch some Coursera lectures, check out a newer, non-academic, application-oriented text, or just build something.
- garysieling 10y agoDo you want to use NLP in a project, or to dig into the state of the art? The NLTK approach may be dated, but it is easier to approach as an engineer, especially if this is a hobby. It will give you a good introduction to problems in the space. The math heavy approaches may give better results long-term, but it will be a much longer time commitment, but this is probably more appropriate if you're trying to find a job. You can also do interesting things with a small dataset and the free plans of APIs like Watson. E.g., I'm working on a search engine for standalone lectures - https://www.findlectures.com https://www.findlectures.com.
- lifeisstillgood 10y agoto the mods: vagabondjack's comment seems sensible, informative and well thought out but seems to have been de-duped in error. Any chance of raising it out of grey-text territory?
- JSeymourATL 10y agoBuild up personal & professional contacts. Check out this group -- ACM Special Interest Group on Artificial Intelligence > https://sigai.acm.org/index.html https://sigai.acm.org/index.html
- probinso 10y agoStart by finding a linguist. You can find one at your local university. Let the linguist design your first project. It should be something that they don't know how to solve, but have wanted to know. Don't worry about if it is feasible. Go to local data meetups when you have enough exposure to form your first questions.