5 ms·

Tangential anecdote: Incidentally the article mentions that ImageNet was conceived through WordNet[0]. I actually downloaded WordNet again last night with the i

Tangential anecdote: Incidentally the article mentions that ImageNet was conceived through WordNet[0]. I actually downloaded WordNet again last night with the idea to relocate qualifying adjectives from certain noun phrases in a system to a separate list of qualifiers, ie. "adj1 adj2 noun" -> {"noun",["adj1","adj2"]} What I found was that WordNet has crap coverage, eg. large numbers of real adjectives were unlisted. In my ancient Chinese translation work (hobby), I have basically concluded that Wiktionary[1] is the best thing going, and also easiest to contribute corrections and background data to. So I thought "why not adapt English Wiktionary data[2] to produce an English adjective list?". This proved far more complicated than a quick hack, since the only way the data seemed to be supplied was as a huge XML file mixing Wiki and XML markup.[3] In the end I downloaded a Wiktionary-derived dataset known as dbnary[4] and parsed wordlists out of that, resulting in a satisfying list of 103,125 English adjectives. Total time invested, under 2 hours. [0] http://wordnet.princeton.edu/ http://wordnet.princeton.edu/ [1] http://en.wiktionary.org/ http://en.wiktionary.org/ [2] https://dumps.wikimedia.org/backup-index.html https://dumps.wikimedia.org/backup-index.html [3] https://dumps.wikimedia.org/enwiktionary/20170720/enwiktionary-20170720-pages-articles.xml.bz2 https://dumps.wikimedia.org/enwiktionary/20170720/enwiktiona... [4] http://kaiko.getalp.org/about-dbnary/ http://kaiko.getalp.org/about-dbnary/

PeterisP 9y ago
BabelNet http://babelnet.org/ http://babelnet.org/ or Open Multilingual Wordnet (http://compling.hss.ntu.edu.sg/omw/ http://compling.hss.ntu.edu.sg/omw/) might be interesting for you. However, the prime use case of WN is as an verified inventory of machine readable senses, providing structured information about the semantic attributes of the word by providing the semantic links to other words/senses. Wictionary will tell you that a word is an adjective, but it doesn't tell a machine what does that adjective mean, how it relates to other adjectives. E.g., the wiktionary entry on 'corgi' has a few sentences that mention that it's a type of dog, but it's not structured enough for this information to be used, and if you just extract it yourself without manual verification then it won't have high accuracy; WN, on the other hand, has an explicit structured link (hypernym) between corgi and dog. (http://wordnetweb.princeton.edu/perl/webwn?o2=&o0=1&o8=1&o1=1&o7=&o5=&o9=&o6=&o3=&o4=&s=corgi&i=2&h=100#c http://wordnetweb.princeton.edu/perl/webwn?o2=&o0=1&o8=1&o1=...)
- contingencies 9y ago
  Interesting though those features were not required for yesterday's application. Now getting on to something more syntax / grammar interpretation related (ie. to a basic level only, parse-the-structure-of-the-sentence - this part is a "verb phrase", that part is a "noun phrase", etc.) ... any idea what a good software library would be? I found https://spacy.io/ https://spacy.io/ + https://github.com/emilmont/pyStatParser https://github.com/emilmont/pyStatParser + http://spes.sourceforge.net/ http://spes.sourceforge.net/ + https://stackoverflow.com/questions/2699646/how-to-get-logical-parts-of-a-sentence-with-java/2703107#2703107 https://stackoverflow.com/questions/2699646/how-to-get-logic...
  - PeterisP 9y ago
    Spacy is an option; for syntactic parsing, you might also want to take a look at Stanford CoreNLP (https://stanfordnlp.github.io/CoreNLP/ https://stanfordnlp.github.io/CoreNLP/) or NLTK (http://www.nltk.org/ http://www.nltk.org/) or SyntaxNet (https://github.com/tensorflow/models/tree/master/syntaxnet https://github.com/tensorflow/models/tree/master/syntaxnet).

PeterisP 9y ago

BabelNet http://babelnet.org/ http://babelnet.org/ or Open Multilingual Wordnet (http://compling.hss.ntu.edu.sg/omw/ http://compling.hss.ntu.edu.sg/omw/) might be interesting for you. However, the prime use case of WN is as an verified inventory of machine readable senses, providing structured information about the semantic attributes of the word by providing the semantic links to other words/senses. Wictionary will tell you that a word is an adjective, but it doesn't tell a machine what does that adjective mean, how it relates to other adjectives. E.g., the wiktionary entry on 'corgi' has a few sentences that mention that it's a type of dog, but it's not structured enough for this information to be used, and if you just extract it yourself without manual verification then it won't have high accuracy; WN, on the other hand, has an explicit structured link (hypernym) between corgi and dog. (http://wordnetweb.princeton.edu/perl/webwn?o2=&o0=1&o8=1&o1=1&o7=&o5=&o9=&o6=&o3=&o4=&s=corgi&i=2&h=100#c http://wordnetweb.princeton.edu/perl/webwn?o2=&o0=1&o8=1&o1=...)

contingencies 9y ago
Interesting though those features were not required for yesterday's application. Now getting on to something more syntax / grammar interpretation related (ie. to a basic level only, parse-the-structure-of-the-sentence - this part is a "verb phrase", that part is a "noun phrase", etc.) ... any idea what a good software library would be? I found https://spacy.io/ https://spacy.io/ + https://github.com/emilmont/pyStatParser https://github.com/emilmont/pyStatParser + http://spes.sourceforge.net/ http://spes.sourceforge.net/ + https://stackoverflow.com/questions/2699646/how-to-get-logical-parts-of-a-sentence-with-java/2703107#2703107 https://stackoverflow.com/questions/2699646/how-to-get-logic...
- PeterisP 9y ago
  Spacy is an option; for syntactic parsing, you might also want to take a look at Stanford CoreNLP (https://stanfordnlp.github.io/CoreNLP/ https://stanfordnlp.github.io/CoreNLP/) or NLTK (http://www.nltk.org/ http://www.nltk.org/) or SyntaxNet (https://github.com/tensorflow/models/tree/master/syntaxnet https://github.com/tensorflow/models/tree/master/syntaxnet).