8 ms·

Natural Language for Developers

183 points by Perados 10y ago

anilshanbhag 10y ago
Correct me if I am wrong, but this approach essentially recognizes entities and matches it to a rule. This is naive ! There are better approaches which work based on parse tree structure or use a PCFG to encode rules
- IshKebab 10y ago
  Can you link to these better approaches?
  - majewsky 10y ago
    PCFGs are about the first thing that I learned about in my Stastical Machine Translation course. It's like a regular context-free grammar (see Wikipedia if you're not familiar with these), but each production is assigned a probability, such that all productions from the same non-terminal sum up to probability 1. Then you can assign a probability to a parse tree by multiplying the probabilities of all productions that were used (the model assumes that all productions are mutually statistically independent). After a proper training run, you have your ambiguous grammar which, given an input sentence, will produce a set of parse trees, and assign a probability to each of them. You will typically choose the most probable one. The Wikipedia article on PCFG is surprisingly biased on RNA/protein analysis and does not go into much detail on how to train these PCFGs. But it's basically just an EM algorithm using inside-outside weights. See for example https://link.springer.com/chapter/10.1007/978-3-540-78291-9_7 https://link.springer.com/chapter/10.1007/978-3-540-78291-9_...
    - IshKebab 10y ago
      This requires defining matching patterns, like a word-based regex though, right? Houndify uses that approach because it is easier to integrate with speech recognition, but it is very cumbersome to use and maintain and doesn't allow learning new utterances. I don't think Wit.ai works as simply as you believe - it isn't merely matching keywords. Unfortunately I haven't been able to find out exactly how it does work. Alexa, and Nuance Mix work the same way as Wit.ai. In my experience it is the superior approach. Viv is also one to watch but they haven't really said the NLU works yet.
aibottle 10y ago
Would never use that. It's free of charge, so how is it financed? Right, its Facebook owned and just like every product of (Google, Facebook, Apple, whatever) if if cannot pay for it I won't use it, they eventually stop offering the service someday because they change their plans. Don't want to be forced to rewrite my app because of their change of mind.
- cyberferret 10y ago
  These are my fears too. I've been looking at this over the weekend as a possibility to embed a 'bot like feature into my web app, but I would hate my users to get used to it, then have Wit go away like Parse did. Much harder to develop an AI engine from scratch myself than an authentication/database engine.
  - aibottle 10y ago
    Exactly, Parse was a great example. Rather basic service and the outcry was huge. NLP is way harder and maybe even impossible for most developers. But I'm optimistic, that something like wit.ai could be done in an opensource community-driven environment. Projects like SpaCy[0] make NLP easier to use for more developers. [0] https://spacy.io/ https://spacy.io/
  - hellcow 10y ago
    Give Abot a look (https://github.com/itsabot/abot https://github.com/itsabot/abot). It's an MIT-licensed bot framework, and it should do everything you need. Would love your feedback as well--email's in my profile.
  - HillRat 10y ago
    As far as I've seen, this isn't anything you couldn't do with basic classification and NER passes in OpenNLP. Straightforward, even trivial, stuff as far as existing NLP toolsets go.
    - deleted 10y ago
      [deleted]
personjerry 10y ago
How does wit.ai make money?
- deet 10y ago
  Wit.ai has been around for a while now, and used to charge a monthly fee for some private models. It was then acquired by Facebook and made free for all uses.
  - personjerry 10y ago
    I guess it would really improve peoples' trust in wit.ai (and thus the number of users) if they would declare their plans for monetization or policy otherwise for the future. Otherwise developers might not want to see their work become invalid when the platform suddenly costs money, or something.
    - stephenr 10y ago
      It's owned by Facebook. What part of their monetisation plan (privacy whoring to sell ads) is not apparent to anyone who cares to look by now?
      - personjerry 10y ago
        That would be rather short-sighted. Different platforms should be monetized differently, and Facebook has shown that it is willing to do this, such as with WhatsApp, which isn't filled with ads: https://www.quora.com/How-does-WhatsApp-make-money-especially-in-India https://www.quora.com/How-does-WhatsApp-make-money-especiall... With wit.ai, I would be surprised if they decided "privacy whoring to sell ads" is the right way to go.
        stephenr 10y ago
        From that very page > Instead, the company said it will explore ways businesses can use WhatsApp to connect with individuals, and will introduce new ways for users to communicate with businesses and organisations that will pay the company to target relevant communications with customers. >> organisations that will pay the company to target relevant communications with customers So, their plan is more targeted advertising in the form of "offers" or "deals" etc.
JustinBlock 10y ago
It seems like Facebook built this so they can feed the data from people using their api to train their eventual messenger AI/assistant.
- jventura 10y ago
  I don't know why you are being downvoted, but this reminds me that thing about facebook/google/etc. that if there is no product you are the product. It also came to my mind that they may be after your users data for their own models..
  - majewsky 10y ago
    They "may be"? As far as I heard, Google is pretty clear about using spoken language input for their voice recognition API for precisely this purpose.
- dhaivatpandya 10y ago
  They were actually acquired by Facebook some time ago.
iheredia 10y ago
So, the site has this demo: https://wit.ai/getting-started https://wit.ai/getting-started First two queries it shows are "Turn on the lights" and "Open the door please!" showing how nice the response is. If you combine them in "Turn on the lights and open the door please!" then the reponse ignores the door part completely: { "_text": "Turn on the lights and open the door please!", "confidence": 0.94, "intent": "lights", "entities": { "on_off": [ { "value": "on" } ] } }
- dominotw 10y ago
  "wash the car" , "feed the cat" results in "intent": "tv_onoff", seems wrong for such simple example.
  - elcapitan 10y ago
    Sounds like the algorithm was trained by someone with heavy procrastination problems.
  - cyberferret 10y ago
    Similarly, when I input "make me a sandwich", I got "tv_onoff" as the intent. I am assuming however, that you can "train" the AI to suit your particular syntax depending on your app and bot's intended purposes. No doubt as more scenarios are fed into Wit's database, the parsing and intent selection will get better?!?
    - coroxout 10y ago
      I typed "Translate how much does that cost to French", which was one of the examples it showed me on the front page (under "Intent Parser", exactly as shown, no quotes), and it still gave me tv_onoff. And "Turn on the TV please" is interpreted as "Turn the TV on to channel 'please'". And don't invite any physicists round, because "Set the temperature to 293 K in my bedroom" results in a "temperature" value of 293000. Still, the docs do indeed say you can train it (for sentiment and yes/no queries, at least, and hopefully other things too).
  - 10y ago
dfischer 10y ago
I've been using Wit and enjoy it. The site could improve in terms of usability and performance. It definitely feels buggy at times. I'm not using their story framework though. Just their intents parser. Using wit entirely for the conversational layer doesn't seem like a good way to go about things. Overall I'm happy though. - I wish I could download my data set at will.
- dshah 10y ago
  I'm using wit.ai as well, and overall it's working well for me. But, the new story-based functionality is something I still haven't been able to get comfortable with -- and it still feels very early. So, that part I'm still implementing myself. And yes, I too wish they they would allow you to download your logs/model. Bugs me that they don't do that (and one of the reasons I'm also looking at api.ai).
  - dfischer 10y ago
    Yeah I took a look at api.ai but the reason I ultimately went down the wit.ai path is because of Facebook backing them. Seems like a nice wallet + data set to help.
- francis88 10y ago
  We will be releasing an import/export in the next 2 weeks
  - dfischer 10y ago
    Wonderful! Thanks!
- aaggarwal 10y ago
  I was using wit.ai for adding an NLP interface on an IoT product at a startup a few years ago. This was before wit.ai was acquired by Facebook. They have an impressive product and a great team. It was very smooth to work with their api, and with reasonable accuracy.
meeper16 10y ago
What we now need is a real AI engine.
rkwz 10y ago
I've used nlp-compromise.js for building an ecommerce shopping assistant bot. It's a bit rough around the edges but works most of the time. Demo: http://nlp-compromise.github.io/website/ http://nlp-compromise.github.io/website/ Github: https://github.com/nlp-compromise/nlp_compromise https://github.com/nlp-compromise/nlp_compromise
- jbnicolai 10y ago
  You seem to have commited a merge conflict in https://npmcdn.com/nlp-syllables@0.0.4/builds/nlp-syllables.js https://npmcdn.com/nlp-syllables@0.0.4/builds/nlp-syllables.... Library looks fantastic!
joshgel 10y ago
I have been using this with https://jasperproject.github.io/ https://jasperproject.github.io/ on a raspberry pi to make an open source amazon echo type device. It works pretty well for controlling basic things around the house.
freen 10y ago
In it's current implementation, Wit.ai is broken. I built a bot recently on it and it was one of the worst experiences I've ever had integrating into a third party API. By a long shot. 1) chat logic is declared on their buggy web platform, which is inconsistent if you use multiple tabs, and doesn't actually indicate current system status. I had a number of "bugs" whereby my "story" was correct, but attempts to test it failed and only after I gave up and let it be for a few hours did it start to work. Meaning stories that were broken would all of a sudden be marked green after a few hours. Flows that failed would start to work given some unclear quantity of time. Oh yeah, and the opposite too: things that worked while futzing around with their interface would potentially break after a few hours. Delightful. 2) Documentation is horrific. It's not clear from any of their examples how to build anything other than a toy to turn on and off lights. If you can figure out how to build a "yes-or-no" entity that follows two different paths from their "recipes" I will buy you a drink ($10 USD and for the first five people). Just provide proof and a BTC address. 3) Flow control is next to impossible to implement, and when you do, it's totally opaque as to why the bot actually chose one flow over the other. 4) can't currently version your stories, which means you can develop new features without breaking production. 6) their support is horrendous. I have four outstanding questions, all unanswered after, oh, two weeks? The combination of all of the above means it's INCREDIBLY brittle, hard to reason about and extremely time consuming to develop against. If Facebook wants people to build bots for their messenger platform, open source wit.ai, and then pay people to develop it. This current state of affairs is untenable, morally questionable for all of the reasons outlined elsewhere in the thread (which have gone unsatisfactorily answered by apparent Facebook employees who should know), and fundamentally broken.

183 points by Perados 10y ago

Correct me if I am wrong, but this approach essentially recognizes entities and matches it to a rule. This is naive ! There are better approaches which work based on parse tree structure or use a PCFG to encode rules

IshKebab 10y ago
Can you link to these better approaches?
- majewsky 10y ago
  PCFGs are about the first thing that I learned about in my Stastical Machine Translation course. It's like a regular context-free grammar (see Wikipedia if you're not familiar with these), but each production is assigned a probability, such that all productions from the same non-terminal sum up to probability 1. Then you can assign a probability to a parse tree by multiplying the probabilities of all productions that were used (the model assumes that all productions are mutually statistically independent). After a proper training run, you have your ambiguous grammar which, given an input sentence, will produce a set of parse trees, and assign a probability to each of them. You will typically choose the most probable one. The Wikipedia article on PCFG is surprisingly biased on RNA/protein analysis and does not go into much detail on how to train these PCFGs. But it's basically just an EM algorithm using inside-outside weights. See for example https://link.springer.com/chapter/10.1007/978-3-540-78291-9_7 https://link.springer.com/chapter/10.1007/978-3-540-78291-9_...
  - IshKebab 10y ago
    This requires defining matching patterns, like a word-based regex though, right? Houndify uses that approach because it is easier to integrate with speech recognition, but it is very cumbersome to use and maintain and doesn't allow learning new utterances. I don't think Wit.ai works as simply as you believe - it isn't merely matching keywords. Unfortunately I haven't been able to find out exactly how it does work. Alexa, and Nuance Mix work the same way as Wit.ai. In my experience it is the superior approach. Viv is also one to watch but they haven't really said the NLU works yet.

aibottle 10y ago

Would never use that. It's free of charge, so how is it financed? Right, its Facebook owned and just like every product of (Google, Facebook, Apple, whatever) if if cannot pay for it I won't use it, they eventually stop offering the service someday because they change their plans. Don't want to be forced to rewrite my app because of their change of mind.

cyberferret 10y ago
These are my fears too. I've been looking at this over the weekend as a possibility to embed a 'bot like feature into my web app, but I would hate my users to get used to it, then have Wit go away like Parse did. Much harder to develop an AI engine from scratch myself than an authentication/database engine.
- aibottle 10y ago
  Exactly, Parse was a great example. Rather basic service and the outcry was huge. NLP is way harder and maybe even impossible for most developers. But I'm optimistic, that something like wit.ai could be done in an opensource community-driven environment. Projects like SpaCy[0] make NLP easier to use for more developers. [0] https://spacy.io/ https://spacy.io/
- hellcow 10y ago
  Give Abot a look (https://github.com/itsabot/abot https://github.com/itsabot/abot). It's an MIT-licensed bot framework, and it should do everything you need. Would love your feedback as well--email's in my profile.
- HillRat 10y ago
  As far as I've seen, this isn't anything you couldn't do with basic classification and NER passes in OpenNLP. Straightforward, even trivial, stuff as far as existing NLP toolsets go.
  - deleted 10y ago
    [deleted]

personjerry 10y ago

How does wit.ai make money?

deet 10y ago
Wit.ai has been around for a while now, and used to charge a monthly fee for some private models. It was then acquired by Facebook and made free for all uses.
- personjerry 10y ago
  I guess it would really improve peoples' trust in wit.ai (and thus the number of users) if they would declare their plans for monetization or policy otherwise for the future. Otherwise developers might not want to see their work become invalid when the platform suddenly costs money, or something.
  - stephenr 10y ago
    It's owned by Facebook. What part of their monetisation plan (privacy whoring to sell ads) is not apparent to anyone who cares to look by now?
    - personjerry 10y ago
      That would be rather short-sighted. Different platforms should be monetized differently, and Facebook has shown that it is willing to do this, such as with WhatsApp, which isn't filled with ads: https://www.quora.com/How-does-WhatsApp-make-money-especially-in-India https://www.quora.com/How-does-WhatsApp-make-money-especiall... With wit.ai, I would be surprised if they decided "privacy whoring to sell ads" is the right way to go.
      - stephenr 10y ago
        From that very page > Instead, the company said it will explore ways businesses can use WhatsApp to connect with individuals, and will introduce new ways for users to communicate with businesses and organisations that will pay the company to target relevant communications with customers. >> organisations that will pay the company to target relevant communications with customers So, their plan is more targeted advertising in the form of "offers" or "deals" etc.

JustinBlock 10y ago

It seems like Facebook built this so they can feed the data from people using their api to train their eventual messenger AI/assistant.

jventura 10y ago
I don't know why you are being downvoted, but this reminds me that thing about facebook/google/etc. that if there is no product you are the product. It also came to my mind that they may be after your users data for their own models..
- majewsky 10y ago
  They "may be"? As far as I heard, Google is pretty clear about using spoken language input for their voice recognition API for precisely this purpose.
dhaivatpandya 10y ago
They were actually acquired by Facebook some time ago.

iheredia 10y ago

So, the site has this demo: https://wit.ai/getting-started https://wit.ai/getting-started First two queries it shows are "Turn on the lights" and "Open the door please!" showing how nice the response is. If you combine them in "Turn on the lights and open the door please!" then the reponse ignores the door part completely: { "_text": "Turn on the lights and open the door please!", "confidence": 0.94, "intent": "lights", "entities": { "on_off": [ { "value": "on" } ] } }

dominotw 10y ago
"wash the car" , "feed the cat" results in "intent": "tv_onoff", seems wrong for such simple example.
- elcapitan 10y ago
  Sounds like the algorithm was trained by someone with heavy procrastination problems.
- cyberferret 10y ago
  Similarly, when I input "make me a sandwich", I got "tv_onoff" as the intent. I am assuming however, that you can "train" the AI to suit your particular syntax depending on your app and bot's intended purposes. No doubt as more scenarios are fed into Wit's database, the parsing and intent selection will get better?!?
  - coroxout 10y ago
    I typed "Translate how much does that cost to French", which was one of the examples it showed me on the front page (under "Intent Parser", exactly as shown, no quotes), and it still gave me tv_onoff. And "Turn on the TV please" is interpreted as "Turn the TV on to channel 'please'". And don't invite any physicists round, because "Set the temperature to 293 K in my bedroom" results in a "temperature" value of 293000. Still, the docs do indeed say you can train it (for sentiment and yes/no queries, at least, and hopefully other things too).
- 10y ago

dfischer 10y ago

I've been using Wit and enjoy it. The site could improve in terms of usability and performance. It definitely feels buggy at times. I'm not using their story framework though. Just their intents parser. Using wit entirely for the conversational layer doesn't seem like a good way to go about things. Overall I'm happy though. - I wish I could download my data set at will.

dshah 10y ago
I'm using wit.ai as well, and overall it's working well for me. But, the new story-based functionality is something I still haven't been able to get comfortable with -- and it still feels very early. So, that part I'm still implementing myself. And yes, I too wish they they would allow you to download your logs/model. Bugs me that they don't do that (and one of the reasons I'm also looking at api.ai).
- dfischer 10y ago
  Yeah I took a look at api.ai but the reason I ultimately went down the wit.ai path is because of Facebook backing them. Seems like a nice wallet + data set to help.
francis88 10y ago
We will be releasing an import/export in the next 2 weeks
- dfischer 10y ago
  Wonderful! Thanks!
aaggarwal 10y ago
I was using wit.ai for adding an NLP interface on an IoT product at a startup a few years ago. This was before wit.ai was acquired by Facebook. They have an impressive product and a great team. It was very smooth to work with their api, and with reasonable accuracy.

meeper16 10y ago

What we now need is a real AI engine.

rkwz 10y ago

I've used nlp-compromise.js for building an ecommerce shopping assistant bot. It's a bit rough around the edges but works most of the time. Demo: http://nlp-compromise.github.io/website/ http://nlp-compromise.github.io/website/ Github: https://github.com/nlp-compromise/nlp_compromise https://github.com/nlp-compromise/nlp_compromise

jbnicolai 10y ago
You seem to have commited a merge conflict in https://npmcdn.com/nlp-syllables@0.0.4/builds/nlp-syllables.js https://npmcdn.com/nlp-syllables@0.0.4/builds/nlp-syllables.... Library looks fantastic!

joshgel 10y ago

I have been using this with https://jasperproject.github.io/ https://jasperproject.github.io/ on a raspberry pi to make an open source amazon echo type device. It works pretty well for controlling basic things around the house.

freen 10y ago

In it's current implementation, Wit.ai is broken. I built a bot recently on it and it was one of the worst experiences I've ever had integrating into a third party API. By a long shot. 1) chat logic is declared on their buggy web platform, which is inconsistent if you use multiple tabs, and doesn't actually indicate current system status. I had a number of "bugs" whereby my "story" was correct, but attempts to test it failed and only after I gave up and let it be for a few hours did it start to work. Meaning stories that were broken would all of a sudden be marked green after a few hours. Flows that failed would start to work given some unclear quantity of time. Oh yeah, and the opposite too: things that worked while futzing around with their interface would potentially break after a few hours. Delightful. 2) Documentation is horrific. It's not clear from any of their examples how to build anything other than a toy to turn on and off lights. If you can figure out how to build a "yes-or-no" entity that follows two different paths from their "recipes" I will buy you a drink ($10 USD and for the first five people). Just provide proof and a BTC address. 3) Flow control is next to impossible to implement, and when you do, it's totally opaque as to why the bot actually chose one flow over the other. 4) can't currently version your stories, which means you can develop new features without breaking production. 6) their support is horrendous. I have four outstanding questions, all unanswered after, oh, two weeks? The combination of all of the above means it's INCREDIBLY brittle, hard to reason about and extremely time consuming to develop against. If Facebook wants people to build bots for their messenger platform, open source wit.ai, and then pay people to develop it. This current state of affairs is untenable, morally questionable for all of the reasons outlined elsewhere in the thread (which have gone unsatisfactorily answered by apparent Facebook employees who should know), and fundamentally broken.