5 ms·

If you want to figure out what Watson can do and bypass all the marketing hype, you can just try all the services available at http://ibm.com/watsondeveloperclo

by pesenti 10y ago

If you want to figure out what Watson can do and bypass all the marketing hype, you can just try all the services available at http://ibm.com/watsondevelopercloud http://ibm.com/watsondevelopercloud. I won't argue that the PR goes often too far, and that's a big debate we have internally (I work for Watson). But it's a pity that most of the negative opinions expressed here come from people who haven't even bothered to try any of the services we put out there or read any of the scientific papers that have been published by our team.

ChicagoBoy11 10y ago
I have tried it -- but for me, at least, it reinforced the negative opinion that other have expressed here. Not because many of the services offered aren't good -- they are -- but because you can't help but note the enormous chasm between what I can actually do with Watson today to what you see on IBM ads. To me, the resentment is completely understandable. When Watson goes on Jeopardy -- an incredible technical and PR achievement -- the implication to the rest of the world is that ".... and now you can use it too!" The ads, too, convey that idea. But that isn't the case. Instead, I can access APIs for language translation, sentiment analysis, object recognition, etc... You guys are selling Watson as a new paradigm of AI never before seen by man -- that I can plug my business and my app into this and do things never-before conceived in the history of computing. The service you let me access is basically a bunch of algorithms for which there are countless alternatives -- even in places like Algorithmia. There's a huge disconnect.
- pesenti 10y ago
  You will indeed be able to do Jeopardy too, and hopefully much more (we found that answering factoid questions isn't the most critical function so we haven't prioritized it, but it's coming). Many of the services that are on the platform can do things that were inconceivable just 5 years ago (like having a general visual classifier handle earth imagery with high enough accuracy to be useful http://www.huffingtonpost.com/entry/ibm-watson-california-water_us_573f2e9ae4b045cc9a70c0f1 http://www.huffingtonpost.com/entry/ibm-watson-california-wa...). Yes, there are alternatives - which is the sign of a healthy market - but we keep improving ours to stay state-of-the-art and to introduce higher level functions.
  - tezka 10y ago
    Produce independently reviewed citation that you are state of the art, or even close to it, in any of these functionality.
    - pesenti 10y ago
      Sure here are two (you may have to follow some links to go to the actual publications): Speech to text: https://developer.ibm.com/watson/blog/2016/04/28/recent-advances-in-conversational-speech-recognition-2/ https://developer.ibm.com/watson/blog/2016/04/28/recent-adva... Emotion: https://developer.ibm.com/watson/blog/2016/02/29/another-step-closer-to-building-empathetic-systems/ https://developer.ibm.com/watson/blog/2016/02/29/another-ste...
      - dr_zoidberg 10y ago
        In my experience, Speech to text has been a huge disapointment. I do give you that in the country where I live (Argentina) we speak a different... "dialect" of Spanish that what you've probably used to train your engine. Even then, it's entirely useless for my country, whereas Google has zero problems.
        pesenti 10y ago
        Very fair point. Our STT service, even though it is based on a technology with the very best results for public benchmarks, is not as robust to accents at the moment - for the exact reason you mention, we didn't train it on all the dialects. But that will come with usage in the field. Google has a head start on this but with the usage of our services growing quickly we will be able to catch up quickly.
  - smitherfield 10y ago
    >we found that answering factoid questions isn't the most critical function so we haven't prioritized it, but it's coming This should be priority number one. As others ITT have been saying, your marketing and PR departments have been writing checks that the product can't cash. When people see that Watson can't do what it does on TV, you lose a sale, and that potential customer is now going to be skeptical of IBM's future marketing claims. It doesn't matter if "answering factoid questions" doesn't have much real-world value — it's first of all what you're marketing the product as, so it should have been done and omigod amazing yesterday, and second of all the reason why your very smart marketing and PR people are pushing that angle is because it's a demonstration that is immediately and intuitively impressive to technical and non-technical people alike. Getting a computer to "answer factoid questions" viscerally screams "major breakthrough!" in a way that "general visual classification of earth imagery" simply doesn't. It wows people at the sales meeting instead of boring them. IBM needs to make sales. You need to build up a userbase toot sweet before Google et al start moving into this sector in a major way, as indeed they already are. There's a limited pool of enterprise customers, they don't change vendors often, and you need to get as many as you can as fast as you can, before Google/Amazon/whoever does, or you're done.
- comex 10y ago
  For what it's worth, I tried the translation demo[1] on some Wikipedia text in Italian and French, along with Google Translate on the same text, and Watson seemed nearly on par but consistently slightly worse. (I also tried Bing, which seemed more even with Google.) I guess this is to be expected, given the relative lack of importance to IBM, but I was vaguely hoping that Watson could somehow afford to spend more CPU time on it or something (after all, their main business is selling access; Google and Bing also do that but mainly exist as a free tool for consumers) and get a better result. Oh well.
  - Bartweiss 10y ago
    I think this may boil down to dedicated intelligence. Google Translate is an entire project devoted solely to efficient, human-useful text translation, where Watson is a more general-purpose language processor. Certainly Watson must have dedicated translation logic, but I would be surprised if there are any narrow domains where Watson beats strong single-purpose competitors.
    - pesenti 10y ago
      We do have a team dedicated to Machine Translation (the same team that actually introduced the statistical approach - the "IBM Models" - in the 90s) but that team had been focused more on non-European languages. If you search the literature you will find that the IBM team is leading for languages like Arabic and Chinese but trailing for European language. We are working on that.
      - Bartweiss 10y ago
        This is a really interesting response, thank you (and thanks for answering so many questions in this thread). That result makes a lot of sense - Google has been working with the huge corpus of English-European translations (e.g. UN reports) and getting impressive results largely because there's so much data available with such direct equivalences. Just based on personal experimentation, they're much less capable with longer "translation distances" like phonogram<-->logogram and Indo-European<-->Sino-Tibetan. I didn't know IBM was leading on those long-distance translations, that's cool to hear.
tiredgrad 10y ago
I've tried Watson; I didn't find it particularly useful, but I did think that the visualisation and interface was superb. Most of my playing with it was focussed on the 'Explore' parts, not the 'Predict'/'Assemble'/'Social Media' functions. I was disappointed in the level of documentation for some stuff (you have to look quite hard to find a definition of 'summary' - on my data, it just sums the column, for example). I was also disappointed that you can't fit distributions or access the distributions that Watson fits, and at the lack of statistics (you can't get at skewness, kurtosis and so on).
guiambros 10y ago
Coincidentally I spent the last couple of hours playing with Watson's services (I was looking for a decent Speech-to-Text API for a toy project). Marketing aside, I must say that BlueMix's user interface is the worst. thing. ever. Buggy, extremely slow, fragmented, with cryptic error messages[1]. Took me 45+ minutes to reactivate my account (the old one was kindly deleted due to inactivity), and create the the TTS/STT services endpoints. Funny; after all the investment to develop Watson, you'd think UI would be the easiest part to get it right. On the positive side, the TTS and STT APIs are simply a pleasure to work with. The documentation is excellent, accuracy is pretty good, and the demos are spot on. Plus you have support for streaming audio through WebSocket for STT (which is a must for my project), and a few voices to chose from for TTS. [1] http://imgur.com/3s4KUPv http://imgur.com/3s4KUPv
- douche 10y ago
  Good lord, are the IBM websites bad... It's taken me almost a full day just to successfully download some DB2 or Sametime SDKs, by the time I'd drilled down through the labyrinth of their non-google-indexed download site, had to generate an account, wait for that to actually be created, then snake my way back down to the download I wanted in the first place,
- wrp 10y ago
  Can someone point me to description of Watson's sentence realization API? I guess it could be called the semantics-to-text part. Realizers like FUF/SURGE, KPML/Nigel, and RealPro are quite complex to use. SIMPLENLG and OpenCCG are simpler, but the results I saw were not impressive. The demos of Watson are quite good, so I'm really interested to see how it performs on operations more complex than delivering factoids.
- pesenti 10y ago
  We are very aware of the Bluemix usability issues and are working with that team to address them. Did you try the new Bluemix by any chance? Thanks for the kind comments on our APIs. We are really trying hard to make them usable.
  - 10y ago
IshKebab 10y ago
I tried their voice recognition. It was actually objectively the worst one of the 5 or 6 I tried (the best was Google by a long way, although I didn't try Bing and I couldn't try Baidu becuase you need a Chinese phone number).
- Bartweiss 10y ago
  Did you try Dragon speech-to-text or similar? I haven't paid for recognition services, but I'd love to know how the dedicated ones compare to Google's offering (which I find usable, but inconvenient).
- pesenti 10y ago
  You should reach out to us or ask questions on the forum. Our experience is that people experiencing very bad accuracy may not have passed the right parameters (e.g., you need to make sure to use the right audio model). We are aware of the usability issue and are trying to address it. In general Watson's STT is slightly better than competition for long audio (like phone conversations), slightly worse for short utterances (like search queries). That's due to a bias in training set.
cs2818 10y ago
I just wanted to say thank you for making these tools easily accessible and mostly free to try. I work in academic research and learned long ago to ignore any PR related to AI/ML products. The ability to easily play with and test the abilities of Watson services is definitely helpful and enjoyable for us.
YeGoblynQueenne 10y ago
>> I work for Watson Hi. Do you happen to do any work with its parts that are written in Prolog?

by pesenti 10y ago

ChicagoBoy11 10y ago

I have tried it -- but for me, at least, it reinforced the negative opinion that other have expressed here. Not because many of the services offered aren't good -- they are -- but because you can't help but note the enormous chasm between what I can actually do with Watson today to what you see on IBM ads. To me, the resentment is completely understandable. When Watson goes on Jeopardy -- an incredible technical and PR achievement -- the implication to the rest of the world is that ".... and now you can use it too!" The ads, too, convey that idea. But that isn't the case. Instead, I can access APIs for language translation, sentiment analysis, object recognition, etc... You guys are selling Watson as a new paradigm of AI never before seen by man -- that I can plug my business and my app into this and do things never-before conceived in the history of computing. The service you let me access is basically a bunch of algorithms for which there are countless alternatives -- even in places like Algorithmia. There's a huge disconnect.

pesenti 10y ago
You will indeed be able to do Jeopardy too, and hopefully much more (we found that answering factoid questions isn't the most critical function so we haven't prioritized it, but it's coming). Many of the services that are on the platform can do things that were inconceivable just 5 years ago (like having a general visual classifier handle earth imagery with high enough accuracy to be useful http://www.huffingtonpost.com/entry/ibm-watson-california-water_us_573f2e9ae4b045cc9a70c0f1 http://www.huffingtonpost.com/entry/ibm-watson-california-wa...). Yes, there are alternatives - which is the sign of a healthy market - but we keep improving ours to stay state-of-the-art and to introduce higher level functions.
- tezka 10y ago
  Produce independently reviewed citation that you are state of the art, or even close to it, in any of these functionality.
  - pesenti 10y ago
    Sure here are two (you may have to follow some links to go to the actual publications): Speech to text: https://developer.ibm.com/watson/blog/2016/04/28/recent-advances-in-conversational-speech-recognition-2/ https://developer.ibm.com/watson/blog/2016/04/28/recent-adva... Emotion: https://developer.ibm.com/watson/blog/2016/02/29/another-step-closer-to-building-empathetic-systems/ https://developer.ibm.com/watson/blog/2016/02/29/another-ste...
    - dr_zoidberg 10y ago
      In my experience, Speech to text has been a huge disapointment. I do give you that in the country where I live (Argentina) we speak a different... "dialect" of Spanish that what you've probably used to train your engine. Even then, it's entirely useless for my country, whereas Google has zero problems.
      - pesenti 10y ago
        Very fair point. Our STT service, even though it is based on a technology with the very best results for public benchmarks, is not as robust to accents at the moment - for the exact reason you mention, we didn't train it on all the dialects. But that will come with usage in the field. Google has a head start on this but with the usage of our services growing quickly we will be able to catch up quickly.
- smitherfield 10y ago
  >we found that answering factoid questions isn't the most critical function so we haven't prioritized it, but it's coming This should be priority number one. As others ITT have been saying, your marketing and PR departments have been writing checks that the product can't cash. When people see that Watson can't do what it does on TV, you lose a sale, and that potential customer is now going to be skeptical of IBM's future marketing claims. It doesn't matter if "answering factoid questions" doesn't have much real-world value — it's first of all what you're marketing the product as, so it should have been done and omigod amazing yesterday, and second of all the reason why your very smart marketing and PR people are pushing that angle is because it's a demonstration that is immediately and intuitively impressive to technical and non-technical people alike. Getting a computer to "answer factoid questions" viscerally screams "major breakthrough!" in a way that "general visual classification of earth imagery" simply doesn't. It wows people at the sales meeting instead of boring them. IBM needs to make sales. You need to build up a userbase toot sweet before Google et al start moving into this sector in a major way, as indeed they already are. There's a limited pool of enterprise customers, they don't change vendors often, and you need to get as many as you can as fast as you can, before Google/Amazon/whoever does, or you're done.
comex 10y ago
For what it's worth, I tried the translation demo[1] on some Wikipedia text in Italian and French, along with Google Translate on the same text, and Watson seemed nearly on par but consistently slightly worse. (I also tried Bing, which seemed more even with Google.) I guess this is to be expected, given the relative lack of importance to IBM, but I was vaguely hoping that Watson could somehow afford to spend more CPU time on it or something (after all, their main business is selling access; Google and Bing also do that but mainly exist as a free tool for consumers) and get a better result. Oh well.
- Bartweiss 10y ago
  I think this may boil down to dedicated intelligence. Google Translate is an entire project devoted solely to efficient, human-useful text translation, where Watson is a more general-purpose language processor. Certainly Watson must have dedicated translation logic, but I would be surprised if there are any narrow domains where Watson beats strong single-purpose competitors.
  - pesenti 10y ago
    We do have a team dedicated to Machine Translation (the same team that actually introduced the statistical approach - the "IBM Models" - in the 90s) but that team had been focused more on non-European languages. If you search the literature you will find that the IBM team is leading for languages like Arabic and Chinese but trailing for European language. We are working on that.
    - Bartweiss 10y ago
      This is a really interesting response, thank you (and thanks for answering so many questions in this thread). That result makes a lot of sense - Google has been working with the huge corpus of English-European translations (e.g. UN reports) and getting impressive results largely because there's so much data available with such direct equivalences. Just based on personal experimentation, they're much less capable with longer "translation distances" like phonogram<-->logogram and Indo-European<-->Sino-Tibetan. I didn't know IBM was leading on those long-distance translations, that's cool to hear.

tiredgrad 10y ago

I've tried Watson; I didn't find it particularly useful, but I did think that the visualisation and interface was superb. Most of my playing with it was focussed on the 'Explore' parts, not the 'Predict'/'Assemble'/'Social Media' functions. I was disappointed in the level of documentation for some stuff (you have to look quite hard to find a definition of 'summary' - on my data, it just sums the column, for example). I was also disappointed that you can't fit distributions or access the distributions that Watson fits, and at the lack of statistics (you can't get at skewness, kurtosis and so on).

guiambros 10y ago

Coincidentally I spent the last couple of hours playing with Watson's services (I was looking for a decent Speech-to-Text API for a toy project). Marketing aside, I must say that BlueMix's user interface is the worst. thing. ever. Buggy, extremely slow, fragmented, with cryptic error messages[1]. Took me 45+ minutes to reactivate my account (the old one was kindly deleted due to inactivity), and create the the TTS/STT services endpoints. Funny; after all the investment to develop Watson, you'd think UI would be the easiest part to get it right. On the positive side, the TTS and STT APIs are simply a pleasure to work with. The documentation is excellent, accuracy is pretty good, and the demos are spot on. Plus you have support for streaming audio through WebSocket for STT (which is a must for my project), and a few voices to chose from for TTS. [1] http://imgur.com/3s4KUPv http://imgur.com/3s4KUPv

douche 10y ago
Good lord, are the IBM websites bad... It's taken me almost a full day just to successfully download some DB2 or Sametime SDKs, by the time I'd drilled down through the labyrinth of their non-google-indexed download site, had to generate an account, wait for that to actually be created, then snake my way back down to the download I wanted in the first place,
wrp 10y ago
Can someone point me to description of Watson's sentence realization API? I guess it could be called the semantics-to-text part. Realizers like FUF/SURGE, KPML/Nigel, and RealPro are quite complex to use. SIMPLENLG and OpenCCG are simpler, but the results I saw were not impressive. The demos of Watson are quite good, so I'm really interested to see how it performs on operations more complex than delivering factoids.
pesenti 10y ago
We are very aware of the Bluemix usability issues and are working with that team to address them. Did you try the new Bluemix by any chance? Thanks for the kind comments on our APIs. We are really trying hard to make them usable.
- 10y ago

IshKebab 10y ago

I tried their voice recognition. It was actually objectively the worst one of the 5 or 6 I tried (the best was Google by a long way, although I didn't try Bing and I couldn't try Baidu becuase you need a Chinese phone number).

Bartweiss 10y ago
Did you try Dragon speech-to-text or similar? I haven't paid for recognition services, but I'd love to know how the dedicated ones compare to Google's offering (which I find usable, but inconvenient).
pesenti 10y ago
You should reach out to us or ask questions on the forum. Our experience is that people experiencing very bad accuracy may not have passed the right parameters (e.g., you need to make sure to use the right audio model). We are aware of the usability issue and are trying to address it. In general Watson's STT is slightly better than competition for long audio (like phone conversations), slightly worse for short utterances (like search queries). That's due to a bias in training set.

cs2818 10y ago

I just wanted to say thank you for making these tools easily accessible and mostly free to try. I work in academic research and learned long ago to ignore any PR related to AI/ML products. The ability to easily play with and test the abilities of Watson services is definitely helpful and enjoyable for us.

YeGoblynQueenne 10y ago

>> I work for Watson Hi. Do you happen to do any work with its parts that are written in Prolog?