4 ms·
Is there an equivalent to MNIST for NLP? I've always wanted to play around in this space but I don't know a good, and simple, database to start with.
by arcanus 9y ago
Is there an equivalent to MNIST for NLP? I've always wanted to play around in this space but I don't know a good, and simple, database to start with.
- nl 9y agoDepends on what you want to try. NLTK has built in datasets. 20 Newsgroups is useful for trying lots of things.
- gumby 9y agoWell there's word2vec, which while it isn't quite the same (its whole point is the vector classification it already embodies), I think is actually the kind of think you were asking for.
- jventura 9y agoI worked with NLP for my research, and I used to build my corpora from wikipedia documents. Here's a tool that I've built to do it: https://github.com/joaoventura/WikiCorpusExtractor https://github.com/joaoventura/WikiCorpusExtractor
- jimsmart 9y agoThere are a few different datasets that might be of use, depending on what you're playing with:- - bAbI https://research.fb.com/downloads/babi/ https://research.fb.com/downloads/babi/ and https://github.com/facebook/bAbI-tasks https://github.com/facebook/bAbI-tasks - SQuAD https://rajpurkar.github.io/SQuAD-explorer/ https://rajpurkar.github.io/SQuAD-explorer/ - WebQuestions https://github.com/brmson/dataset-factoid-webquestions https://github.com/brmson/dataset-factoid-webquestions Edit: there's also a great list of datasets on the ParlAI project page https://github.com/facebookresearch/ParlAI https://github.com/facebookresearch/ParlAI