8 ms·

For all the hype that surrounds this platform, the live demos are truly underwhelming...

skorgu 12y ago
Agreed. For kicks I dumped a bunch of dmr biographical material (Wired, NYT, his personal web page) into http://ibmlaser.mybluemix.net/siredemo.html http://ibmlaser.mybluemix.net/siredemo.html Some "PERSONS" it found, everything between '' is [sic]: 'C' 'Cobol' 'B begat C.' '”' (yes, a single fancy closing quote) 'engineer' '’ initials' '’70s' '“Jobs’ genius' Some of it is understandable given how annoying single-character identifiers are to parse and others are roughly reasonable in context. Even given some of those quirks I bet it would be useful if you had bajillions of pages (cough ediscovery cough) to search through instead of a few general-audience articles.
- shrikant 12y ago
  For anyone else who wants to know, 'dmr' in parent post is Dennis Ritchie. (I knew this, but didn't make the association right away, and it's quite hard to search online for 'dmr')
  - TillE 12y ago
    Searching for 'dmr person' does the trick with Google. Top result, and a sidebar of Dennis Ritchie.
  - pdabbadabba 12y ago
    You should've asked Watson. Then you would have learned the truth: dmr is 's wear daily.
- simias 12y ago
  Yeah it's pretty bad, even inputing a well written news article like this one: http://www.theguardian.com/fashion/2014/oct/08/return-of-john-galliano-more-than-matter-of-style http://www.theguardian.com/fashion/2014/oct/08/return-of-joh... ” PERSON its ORGANIZATION ’s Wear Daily PERSON shoulder pad DISEASE The rest is mostly correct (although random punctuation seems included at times) but it's nothing you couldn't get by word matching using a big dictionary. It reminded me to get tested for shoulder pads though, so there's that.
  - deleted 12y ago
    [deleted]
  - pesenti 12y ago
    We have a bug with quotes/punctuation that we are fixing now (that account for 1 & 3). "its" is not necessarily a mistake, it likely refers to the owner. "shoulder pad" is a mistake due to the probabilistic nature of the algorithms used.
  - parsabg 12y ago
    FWIW I ran the article through AYLIEN's Text API and here's what I got: http://aylien.com/text-api-demo?text=http%3A%2F%2Fwww.theguardian.com%2Ffashion%2F2014%2Foct%2F08%2Freturn-of-john-galliano-more-than-matter-of-style&tab=concepts&run=1 http://aylien.com/text-api-demo?text=http%3A%2F%2Fwww.thegua... (disclaimer: I'm the founder of AYLIEN)
- pesenti 12y ago
  We have found a bug with character normalization (which messes up the parsing of the punctuation). We'll fix this today.
- jonperl 12y ago
  Interesting, this service is a lot like http://www.alchemyapi.com/products/demo/alchemylanguage http://www.alchemyapi.com/products/demo/alchemylanguage
- pesenti 12y ago
  We fixed the bug with punctuation. Can you try again?
  - skorgu 12y ago
    Took me a while but I retried. It's definitely better, to the point that I have to stretch a bit to find really bad errors. "B begat C" still makes it in as do a few similar ones but considering how pathological a case a single-letter identifier is I can forgive that. I assume because the identifiers aren't unique that they're returned in the underlying api as text locations. Hmm now I want to throw EDGAR data at this.
TeMPOraL 12y ago
I played with language recognition demo and learned that Polish is apparently German. Being born in Galicia[0], I guess I shouldn't be surprised... [0] - http://en.wikipedia.org/wiki/Galicia_%28Eastern_Europe%29 http://en.wikipedia.org/wiki/Galicia_%28Eastern_Europe%29
- pesenti 12y ago
  Sorry Polish is not one of the supported language yet: Arabic; Chinese (Simplified); Chinese (Traditional); Cyrillic; Danish; Dutch; English; Farsi; Finnish; French; German; Greek; Hebrew; Hindi; Icelandic; Italian; Japanese; Korean; Norwegian (Bokmal); Norwegian (Nynorsk); Portuguese; Spanish; Swedish; Turkish; Urdu.
  - TeMPOraL 12y ago
    Oh right, I'm sorry. It was actually written on the demo page, but I somehow missed it.
  - johnward 12y ago
    I also imagine that distinguishing between many of these languages is not a trivial task for a computer to handle.
    - TillE 12y ago
      You'd get collisions if you're trying to determine the language of a single word, obviously. But a sentence or two should be no problem. You're just matching words in a dictionary. Different languages are mostly quite distinct. It would be a more interesting challenge to recognize and understand, say, German dialects.
      - a3_nm 12y ago
        In fact you can already get very good results, even without a dictionary, just with a distribution of the trigrams in your input languages (which is immediate to compute from a corpus). Just compute the distribution of the sample and return the language that has the closest one.

Agreed. For kicks I dumped a bunch of dmr biographical material (Wired, NYT, his personal web page) into http://ibmlaser.mybluemix.net/siredemo.html http://ibmlaser.mybluemix.net/siredemo.html Some "PERSONS" it found, everything between '' is [sic]: 'C' 'Cobol' 'B begat C.' '”' (yes, a single fancy closing quote) 'engineer' '’ initials' '’70s' '“Jobs’ genius' Some of it is understandable given how annoying single-character identifiers are to parse and others are roughly reasonable in context. Even given some of those quirks I bet it would be useful if you had bajillions of pages (cough ediscovery cough) to search through instead of a few general-audience articles.

shrikant 12y ago
For anyone else who wants to know, 'dmr' in parent post is Dennis Ritchie. (I knew this, but didn't make the association right away, and it's quite hard to search online for 'dmr')
- TillE 12y ago
  Searching for 'dmr person' does the trick with Google. Top result, and a sidebar of Dennis Ritchie.
- pdabbadabba 12y ago
  You should've asked Watson. Then you would have learned the truth: dmr is 's wear daily.
simias 12y ago
Yeah it's pretty bad, even inputing a well written news article like this one: http://www.theguardian.com/fashion/2014/oct/08/return-of-john-galliano-more-than-matter-of-style http://www.theguardian.com/fashion/2014/oct/08/return-of-joh... ” PERSON its ORGANIZATION ’s Wear Daily PERSON shoulder pad DISEASE The rest is mostly correct (although random punctuation seems included at times) but it's nothing you couldn't get by word matching using a big dictionary. It reminded me to get tested for shoulder pads though, so there's that.
- deleted 12y ago
  [deleted]
- pesenti 12y ago
  We have a bug with quotes/punctuation that we are fixing now (that account for 1 & 3). "its" is not necessarily a mistake, it likely refers to the owner. "shoulder pad" is a mistake due to the probabilistic nature of the algorithms used.
- parsabg 12y ago
  FWIW I ran the article through AYLIEN's Text API and here's what I got: http://aylien.com/text-api-demo?text=http%3A%2F%2Fwww.theguardian.com%2Ffashion%2F2014%2Foct%2F08%2Freturn-of-john-galliano-more-than-matter-of-style&tab=concepts&run=1 http://aylien.com/text-api-demo?text=http%3A%2F%2Fwww.thegua... (disclaimer: I'm the founder of AYLIEN)
pesenti 12y ago
We have found a bug with character normalization (which messes up the parsing of the punctuation). We'll fix this today.
jonperl 12y ago
Interesting, this service is a lot like http://www.alchemyapi.com/products/demo/alchemylanguage http://www.alchemyapi.com/products/demo/alchemylanguage
pesenti 12y ago
We fixed the bug with punctuation. Can you try again?
- skorgu 12y ago
  Took me a while but I retried. It's definitely better, to the point that I have to stretch a bit to find really bad errors. "B begat C" still makes it in as do a few similar ones but considering how pathological a case a single-letter identifier is I can forgive that. I assume because the identifiers aren't unique that they're returned in the underlying api as text locations. Hmm now I want to throw EDGAR data at this.

TeMPOraL 12y ago

I played with language recognition demo and learned that Polish is apparently German. Being born in Galicia[0], I guess I shouldn't be surprised... [0] - http://en.wikipedia.org/wiki/Galicia_%28Eastern_Europe%29 http://en.wikipedia.org/wiki/Galicia_%28Eastern_Europe%29

pesenti 12y ago
Sorry Polish is not one of the supported language yet: Arabic; Chinese (Simplified); Chinese (Traditional); Cyrillic; Danish; Dutch; English; Farsi; Finnish; French; German; Greek; Hebrew; Hindi; Icelandic; Italian; Japanese; Korean; Norwegian (Bokmal); Norwegian (Nynorsk); Portuguese; Spanish; Swedish; Turkish; Urdu.
- TeMPOraL 12y ago
  Oh right, I'm sorry. It was actually written on the demo page, but I somehow missed it.
- johnward 12y ago
  I also imagine that distinguishing between many of these languages is not a trivial task for a computer to handle.
  - TillE 12y ago
    You'd get collisions if you're trying to determine the language of a single word, obviously. But a sentence or two should be no problem. You're just matching words in a dictionary. Different languages are mostly quite distinct. It would be a more interesting challenge to recognize and understand, say, German dialects.
    - a3_nm 12y ago
      In fact you can already get very good results, even without a dictionary, just with a distribution of the trigrams in your input languages (which is immediate to compute from a corpus). Just compute the distribution of the sample and return the language that has the closest one.