5 ms·

I would venture there's a way to make "concepts" and "entities" become linkable automatically based on existing articles, but that would mean a bit of engineeri

by eksith 13y ago

I would venture there's a way to make "concepts" and "entities" become linkable automatically based on existing articles, but that would mean a bit of engineering. I.E. A name, product or academic field. But then there's going to be a high number of links to articles that haven't been created yet or deleted/merged etc... in cases of lack of notability.

mindcrime 13y ago
I don't know if it has that exact feature or not, but Semantic Mediawiki has a lot of extensions to base Mediawiki that involve working with data at a semantic level. http://semantic-mediawiki.org/wiki/Help:Introduction_to_Semantic_MediaWiki http://semantic-mediawiki.org/wiki/Help:Introduction_to_Sema... If not, doing that should be possible using an NLP library that does NER. Along with heuristics, one could use the list of currently existing articles as a seed. Edit: Of course, if all you're trying to do is link to existing pages, then you don't use the set of existing pages as a seed, you just use them as the list. But if you're trying to extract "entities" that don't have WP pages yet, then you'd still want to fall back to other NER techniques, which include various heuristics and what-not. Whether or not there would be an value in that is an open question, I suppose.
- eksith 13y ago
  Good resource! And I think your idea of linking to the disambiguation page makes sense, but there may be a way to infer the correct article from the list of links based on the context of the text in the linking article.
  - mindcrime 13y ago
    Yeah, there's that as well. Also, if you're interested in that sort of thing, two other projects you might find interest are: http://stanbol.apache.org http://stanbol.apache.org and http://uima.apache.org http://uima.apache.org Both involve extracting semantic meaning from unstructured data. It's pretty cool stuff.
    - bergie 13y ago
      Here is a quick demo of Stanbol-provided Wikipedia annotations and disambiguation in a WYSIWYG editor: https://www.youtube.com/watch?v=zAMUpd6rb9k&feature=youtube_gdata_player https://www.youtube.com/watch?v=zAMUpd6rb9k&feature=yout...

by eksith 13y ago

mindcrime 13y ago

I don't know if it has that exact feature or not, but Semantic Mediawiki has a lot of extensions to base Mediawiki that involve working with data at a semantic level. http://semantic-mediawiki.org/wiki/Help:Introduction_to_Semantic_MediaWiki http://semantic-mediawiki.org/wiki/Help:Introduction_to_Sema... If not, doing that should be possible using an NLP library that does NER. Along with heuristics, one could use the list of currently existing articles as a seed. Edit: Of course, if all you're trying to do is link to existing pages, then you don't use the set of existing pages as a seed, you just use them as the list. But if you're trying to extract "entities" that don't have WP pages yet, then you'd still want to fall back to other NER techniques, which include various heuristics and what-not. Whether or not there would be an value in that is an open question, I suppose.

eksith 13y ago
Good resource! And I think your idea of linking to the disambiguation page makes sense, but there may be a way to infer the correct article from the list of links based on the context of the text in the linking article.
- mindcrime 13y ago
  Yeah, there's that as well. Also, if you're interested in that sort of thing, two other projects you might find interest are: http://stanbol.apache.org http://stanbol.apache.org and http://uima.apache.org http://uima.apache.org Both involve extracting semantic meaning from unstructured data. It's pretty cool stuff.
  - bergie 13y ago
    Here is a quick demo of Stanbol-provided Wikipedia annotations and disambiguation in a WYSIWYG editor: https://www.youtube.com/watch?v=zAMUpd6rb9k&feature=youtube_gdata_player https://www.youtube.com/watch?v=zAMUpd6rb9k&feature=yout...