15 ms·

IBM halting sales of Watson AI tool for drug discovery

257 points by hprotagonist 8y ago

sambroner 8y ago
I can’t read most of that article because it’s behind a paywall, but wow, is Watson just the largest attempt to sell vaporware of all time?? They had a super bowl ad! A few friends from university were hired into the Watson team as hardly technical PMs. I have never heard any of them describe what Watson is or does.
- bunderbunder 8y ago
  I'm not sure I'd call it vaporware. I've not used it myself, but the more objective reports I've seen indicate that Watson is very good at what it does. The problem seems more to be what it does - accurately interpreting and then answering queries written in natural language - isn't actually that useful. For example, for drug discovery: I'm guessing, off the cuff, that there are other drug discovery tools out there, and that, while they don't allow you to frame your searches in terms of natural language queries, that's probably not actually a problem in practice. Because the querying methods they've developed are presumably highly tuned to their real task, which is enabling a skilled and knowledgeable practitioner to specify what they're looking for with great precision. It's sort of like programming languages: The ones that are designed to be the closest to natural language (e.g., AppleScript) do have a gentle learning curve for absolute beginners, but they also turn out to be some of the very worst languages for trying to do any sort of serious work.
  - sodosopa 8y ago
    Which is why data scientists use python instead of AppleScript.
- sodosopa 8y ago
  I’m running on prem instances of Watson and it’s far from vaporware but Watson has become more of a brand name. For the Watson product I use, It’s more of a wrapper to do AI and ML work, notebooks, containers and related tools for data science.
  - elliekelly 8y ago
    Same here. And so far I have to say I'm quite happy with it. Is it the best tool for ML? Probably not. But it's certainly the easiest I've found so far and it works for what I'm using it for.
- donkeyd 8y ago
  I've commented this before, but Watson is nothing but branding for anything IBM does that is somewhat intelligent. There's pretty much no common technology between the different Watson products, so there's no 'core Watson' like many people believe. Some products with Watson branding are great and industry leading. Others suck. The latter tend to appear in headlines and severely impact the brand they established with Jeopardy and their ads.
  - apohn 8y ago
    I suspect what happened was that Watson started out as a single product or focused suite of products. I'm just guessing, but then as the marketing induced hype started and growth/results were not on target , they realized there was a lot of interest in Watson so IBM pivoted and started rolling everything under the Watson brand - products, consulting, cloud stuff, etc. Basically that way they could say Watson was a "success" and internally/externally it would be opaque as to where exactly the profits and losses were. I say all this because I'm at a place now where this is happening. Hugh company with a supposedly game changing product that is mediocre (at best). Massive marketing campaign for "Product X" that is 100% buzzwords and 200% BS. After constant missed revenue targets and product disappointments internally and externally, company is clearly pivoting (but not saying so) to rolling everything under "Brand X."
    - nintendo95 8y ago
      I worked there (Watson Health) a few years back. All that happened is IBM. That's right. They bought us (a small start-up) and this is what they did in a year to us: 1. replaced managers with their own who didn't get anything, kind of old date executives taken away from mainframe 2. banned remote work (this costed them a few brilliant engineers) 3. opened one huge open space full of noise and chatter, I mean sales team next to the development team, etc (this cost them even more headcount among experienced developers) 4. For months we did nothing! There were whole teams doing nothing for months on an end. I wrote zero lines of code in over 6 month period. Why? Because management didn't know which direction should be taken... and they kept hiring too! They kept hiring new developers when the ones already at place had absolutely nothing to do! This is from my (simple developer) perspective. Not sure how it looked in sales, among executives, etc. But that was very weird environment. I had great Manager who asked me to learn react and take courses in react (mind you it was two years back!) As we "might want to do something in react in the future". So I basically spent my last six months there learning React, aka preparing to the job interviews... they even got us paid courses and all. I mean... IBM. And once they fired me (these were lay-offs, thousands affected, many of them just hired in past year, like myself) I was paid severance pay too. I went there worked a year, last 6 months was learning for job interviews... fantastic pay too. IBM is crazy.
- gowld 8y ago
  This is what IBM has always done. 20 years ago they pretended that their Deep Blue chess engine was in all their IT products. It's similar to how DeepMind AlphaGo is PR for Google's AI effort, but IBM takes it to comical extremes.
  - wpasc 8y ago
    I agree with you that IBM is full of shit, Google too to a degree, but Google has used AI in a variety of their products even down to data center cooling processes. I don't think IBM and Google are comparable in this respect.
  - ne0sapi3n 8y ago
    > 20 years ago they pretended that their Deep Blue chess engine was in all their IT products [citation needed] IBM did not do this, though funding the Deep Thought team from CMU was cheaper than a Super Bowl commercial and brought more durable effect.
- notfromhere 8y ago
  Watson is the branding term for an army of outsourced consultants (mostly in India) plugging away at the raw data. Watson is not very good, and largely doesn't exist outside of slick marketing campaigns.
- dekhn 8y ago
  there are large amounts of interesting things in watson but it was still overmarketed for what it can do.
- VeninVidiaVicii 8y ago
  That's really interesting. I work in the emergency room, and pretty much our entire approach up patient care is algorithm-driven. As soon as the patient checks in, their demographics (age, sex, etc) and vitals are fed into a mysterious program, which suggests an acuity level, and basically drives the whole course of treatment. The patient is required to be tested for a variety of disease processes based on the AI-generated differential diagnoses. Honestly, I thought/still think medicine is headed toward this, which is why I decided to go into research.
  - darrin 8y ago
    What program is that? Is that something Epic/Cerner/whatever does, or some add-on?
    - VeninVidiaVicii 8y ago
      It's MedHost with a lot of modifications specific to our hospital.
imroot 8y ago
I have a friend who is an OB/GYN Oncologist in Indianapolis. Her hospital had IBM Watson Health on campus last year, and she mentioned to me in passing last month that they finally had kicked IBM off of their (learning) university's campus. When I asked her why, she said: "Often times, Watson would recommend courses of treatment that would be completely incorrect, if not detrimental -- or even sometimes lethal -- to the patient. It became more of a hassle than a learning tool."
- devoply 8y ago
  It's good we still have doctors to provide oversight for this sort of thing. Imagine a future where machines take over because they are thought to be smarter than doctors and then start killing people and their mistakes are written off because the people were already sick. Like an Elizabeth Holmes AI.
  - euler_ 8y ago
    Does it make a difference if they preform better than human docs? The same thing happens now with human docs.
    - ethbro 8y ago
      The difference is the expectation of fallibility with human docs. Which is why checks are built into the system. I have zero trust in IBM to market their ML products correctly so that proper checks are maintained. Especially since technically, explainability is still an active area of ML research.
      - freehunter 8y ago
        >I have zero trust in IBM to market their ML products correctly so that proper checks are maintained. That's a problem, yes, but it's not a new one. Vendor management has been around for decades, centuries, millennia maybe. If I contract out part of my job, it's still my responsibility to make sure the contractors are doing their job right. "But their marketing said..." or "but their sales guys said..." is not an excuse and everyone knows that. Doctors noticing that Watson is wrong is expected. Doctors missing the fact that Waston is wrong is a failure of that doctor and the doctor who didn't check the results is the responsible party. The checks don't come from Watson, the checks come from humans who oversee Watson. If Watson is wrong often enough that it's hindering the doctors, then kicking it out is the right call. But there can never be an argument of "Watson got the diagnosis wrong and that's why the patient died" because ultimately IBM is still just a vendor and Watson is still just a contractor.
deleted 8y ago
[deleted]
deleted 8y ago
[deleted]
StreamBright 8y ago
No surprise here. IBM has stopped producing usable engineering solutions long time ago.
wpasc 8y ago
I sincerely hope IBM's failures do not deter further progress in trying to innovate in healthcare technology. I believe healthcare vitally needs GOOD/USEFUL technology and AI applications because medical progress is not advancing quickly enough by several metrics. I hope IBM continues to fail and becomes a case study in the failure of massive bureaucracy.
- PedroBatista 8y ago
  IBM will not fail, Oracle will fail first than IBM and that's unlikely to happen. You see, they are NOT tech companies. They have something you would call "tech", they sell something that looks like "tech" but they really solve problems people with access to large amounts of money have. In other words, they produce cushy and warm beds for C-level execs to have a good night of sleep, and everyone needs some bed to sleep every night.. Technology is a risky and volatile business and IBM has more than 100 years in business.
  - pm90 8y ago
    Spot on. IBM really does capitalize on the adage of nobody being fired for buying IBM. Their biggest strength has been in their marketing rather than actual technology. Note that, being a large org, they do have teams doing excellent work too. IBM research, some of the cloud, design etc. teams seem to be doing good stuff. But most of it is still overpromising and underdelivering while charging clients a lot of cash.
    - exelius 8y ago
      This is less true than it used to be. Oracle/SAP are largely recognized as multi-billion dollar technology quagmires these days; integration has become so much easier that cobbling together a bunch of SaaS solutions to replace a centralized ERP is a viable approach. Likewise, IBM has become known as a vaporware vendor. They combine the worst parts of management consulting with an offshore / outsourced development process, so I guess it’s not a surprise they ended up like all the other companies that sell business tech solutions (Infosys, Wipro, etc). They’re often mentioned in the same breath these days.
tolstoy77 8y ago
a lot of these AI companies products are really terrible. Has anyone ever tried the AI API models from clarifai ? Just so unaccurate. It seems like a scam. I've also had a really bad experience with watson's speech to text apis.
- IshKebab 8y ago
  Yeah their speech recognition API was by far the worst of the ~5 I tested a few years ago. Like, almost as bad as Sphinx.
- linuxftw 8y ago
  I'm pretty sure it's all a big scam, at least the crap that comes out of IBM.
  - tolstoy77 8y ago
    +1
- pmart123 8y ago
  Watson’s is awful. Google’s default models are by far the best followed by Microsoft at a distance. Then, many of them are so bad that it is laughable.
  - gbersac 8y ago
    Do you have any insight about AWS IA ?
    - pmart123 8y ago
      Haven’t tried it actually yet.
    - pmart123 8y ago
      I tested most of the solutions a year or two ago before AWS rolled out Amazon Comprehend. I found that Google’s API combined with some code and open source packages ended up being additive for entity recognition, classification, and semantic text. Unsurprisingly, Google’s custom search API’s are also worlds better than the other places, as is it’s Maps API. I haven’t tried any of the image classification API’s, but I won’t be surprised if Google won here too.
haolez 8y ago
Slightly unrelated, but I hope IBM doesn't ruin Red Hat and CoreOS. I would love to see the CoreOS tools to gain more adoption.
- jniedrauer 8y ago
  I'm a long time Fedora user. I love the sane defaults. It's just a solid workhorse. But I've been feeling pretty uneasy about using it lately...
  - jsight 8y ago
    Alternatively, a lot more IBMers will be able to use Fedora and it may improve even more. I don't see them having a strong reason to do any damange there.
    - aquaticsunset 8y ago
      Maybe? IBM is mainly a Windows and Mac house for dev machines, with Ubuntu being a very, very small portion of workstations. I’m not sure the acquisition will create a mass exodus from the status quo there.
abdulhaq 8y ago
I suggest we take each AI doctor and place it in a self-driving car. The AI doctor will have a speaker to talk to the AI driver. Then take all these self driving cars with their AI drivers and passengers and get them all to drive together in circles on a race track. The last car not to crash gets to survive and is taken to the Whitehouse where it can advise Trump on what Mexicans and Muslims really think.
- codinger 8y ago
  ummm https://cdn.vox-cdn.com/uploads/chorus_asset/file/6238309/Screen_Shot_2016-03-24_at_10.46.22_AM.0.png https://cdn.vox-cdn.com/uploads/chorus_asset/file/6238309/Sc...
foobiekr 8y ago
Slowly, I am noticing that people are starting to question the abilities of machine learning systems and, in particular, seeing a lot of the transparent AI-washing that is now going on for systems that aren't AI at all. This is one of the most positive changes in the tech industry, honestly.
- Barrin92 8y ago
  Machine learning and neural net systems seem to be the inverse of the "uncanny valley", call it the "miraculous hill" if you will. They're just good enough to, with insane amount of computing resources, churn out economically viable results, but in my opinion they're the biggest setback in actually developing intelligence or cognition in a long time. There's very little insight to be gained from what they pick up, and they function purely in a stochastic sense. Even the best ML algorithm has no ability to reason at a high level or produce counterfactuals, it works purely by correlation and still, in those 1% edge cases, will be as helpless as anything else. That might be good enough when selling advertisements, but in automated cars and healthcare treatment, this sort of failure is not an option.
  - hodgesrm 8y ago
    ML and neural nets work surprisingly well on a lot of boring but important use cases. Optimizing online ad placement might be humdrum but it's the foundation of a $100B market in the US alone. [0] In my opinion the real hype is that ML has become so popular that new practitioners tend to forget other analytic techniques like SQL data warehouses. Interestingly these are starting to absorb ML capabilities like logistic regression, which are now accessible through SQL and can benefit from MPP and vectorwise query execute in DBMS types like ClickHouse, Vertica, and Google BigQuery. [1] https://www.forbes.com/sites/danafeldman/2018/03/28/u-s-tv-ad-spend-drops-as-digital-ad-spend-climbs-to-107b-in-2018/#31c701a77aa6 https://www.forbes.com/sites/danafeldman/2018/03/28/u-s-tv-a... Disclaimer: I work on ClickHouse.
- daenz 8y ago
  https://i.imgur.com/FXi6NXb.jpg https://i.imgur.com/FXi6NXb.jpg According to the trend, we're approaching another AI Winter
cromwellian 8y ago
Basically, Triumph of Sales over Engineering.
notyourday 8y ago
Watson's applications outside of a quick search of existing datasets are vaporware. I'm amazed not a single customer has gone public with it when so many of them are whispering about it on LinkedIn.
distant_hat 8y ago
Slightly over a year back, we were contacted by IBM to try out Watson under some program for startups. Starting out, they asked us to give them a try. We gave them a NLP related task and they were confident they would be able to do a good job of it. After multiple meetings over the next 2 months, they barely produced any useful output. In the meantime, an intern with us got a pretty decent solution using just Python. Our management did get invited to lunches and meetings for selling them some more products. During one of such invitations, since our CEO couldn't make it, I was sent. I was surprised by how shoddy their presentation was given that I thought they might not be doing a good job on the problem we gave them but they should be slick on the selling side. Their presentation seemed like it was out of 2005 or something.
- opportune 8y ago
  Wow, I have been in this exact situation as an intern
- c0mpute 8y ago
  What do you think of your CEO now :)?
  - distant_hat 8y ago
    Well, he was pretty good, he let us take the decisions completely so we never paid anything for it other than in time. Moreover, we weren't flush with cash or anything so they wouldn't have made much anyway.
- sodosopa 8y ago
  Python is included in the most recent versions of Watson Studio ML. Were Jupyter notebooks included when you originally tried the tasks? I’m curious why the included python didn’t work and the standalone did.
  - beefcafe 8y ago
    That's like saying book A is in Spanish and book B is also in Spanish, so why did you like book B better? Python is just a language, what you do with it is what matters. Sounds like they built something completely distinct from Watson and it did the job.
bilater 8y ago
'And the trough is coming...' - Gartner
batmansmk 8y ago
Watson is more a marketing term than a technical connection between products. I invite you to test "Watson Tone Analyzer" https://tone-analyzer-demo.ng.bluemix.net/ https://tone-analyzer-demo.ng.bluemix.net/ : - "I like this product." => "this is an analytical opinion with neutral emotion." - "I like it" => "Tentative, 50% happy answer." - "It's not a bad product." => "analytical". - "It's not a bad tool" => "joyful answer".
- darkpuma 8y ago
  - "Spiders are not insects." => "fear" Exactly what I expected, and also flat wrong. - "There's a monster under my bed." => "joy" Fascinating.
  - chillacy 8y ago
    You'd think they'd pair it up with a naive bayes classifier at least.
- hodgesrm 8y ago
  Thanks. I tried it myself with text examples similar to yours. The answers are a bit bewildering.
- tiagobraw 8y ago
  I tried it with the first phrase on the page "This service uses linguistic analysis to detect joy, fear, sadness, anger, analytical, confident and tentative tones found in text." response: Fear: A response to impending danger. It is a survival mechanism that is a reaction to some negative stimulus. It may be a mild caution or an extreme phobia.
- CSMastermind 8y ago
  > Watson is more a marketing term than a technical connection between products. As far as I can tell there's absolutely no connection between the products being labeled as 'Watson' and often times very little machine learning taking place either. I've interviewed several candidates from IBM over the last ~3 years and almost all of them worked on something with 'Watson' in the name. When we whiteboarded out the architecture of what they worked on it was mostly automation with _maybe_ a touch of machine learning thrown in by a module written by someone else. Very few of them were able to pass the technical interview.
deleted 8y ago
[deleted]
sonnyblarney 8y ago
For reference, here is the IBM Watson VP hustling it [1] on Charlie Rose. Funny in retrospect. Especially that haircut ... [1] https://charlierose.com/videos/29530 https://charlierose.com/videos/29530
ct520 8y ago
How is Watson not the butt of jokes in the tech industry by now? Seriously, never heard anything that didn’t end up being smoke and mirrors.
DebtDeflation 8y ago
Good. Focus on the use cases that actually work, namely a chatbot for handling customer service questions and an enterprise search engine.
- aquaticsunset 8y ago
  Can’t tell if sarcasm, but IBM’s had massive headache and trouble in this segment too. Good customer service agents are expensive. Bad ones are cheaper. But both still have a far higher success rate than conversational bots do. And a hybrid approach is, again, harder than one would initially assume (you know... why it’s painful and infuriating to press fifteen numbers on the phone to get to a real person? Same concept)
duxup 8y ago
There have been a couple articles that a lot of issues with Watson was how it was sold. Watson was being sold as sort of a magic drop in solution that could do oh so many things. In reality Watson took a great deal of work to carefully structure and process data, evaluation of the output, and more thoughtful of approaches to further refine the information and evaluate outputs. It also required a fair amount of involvement with the individual customer's staff. And there was always the possibility that because each use case was different... it simply wouldn't work out. To some extent it seems like it should have been sold as a journey... not cookie cuter solution for things that Watson had never encountered before. How do you sell that, I don't know, if I knew I'd probabbly be a pretty good salesman.
wessorh 8y ago
Even Watson wasn't a general AI. Hearing all the anecdotes from folks that tried to use it.. Narrow AI has the attribute that it can't be ported from one field to another. It might win at Jepordy but sucks at Oncology and Drug Discovery. All AI isn't the same. We are general AI and we can solve problems in asymmetrically bad situations. Narrow AI is not portable
opsiprogram 8y ago
I work with a group using machine learning for drug discovery (I'm not a biologist/chemist) but the bio people around me loved to talk poorly about Watson's drug discovery tool. Lots of focus on the algorithms in the comments here, but from what I could glean generally they lacked domain experts when developing the datasets... we spend 90% just finding the best data... and even then it's tricky. I think they may have had lower standards for the input into the system... garbage in garbage out.
throw2016 8y ago
It seems just like crypto the 'ai' hype cycle is coming to an end with zero achievements to show for itself beyond puffery and premature speculation about radiologists, drivers and others losing their jobs. And its not just hype, like crypto the ai hype stepped well beyond the line to outright fraud and deception with tech folks trying to pass off backward looking pattern matching as 'intelligence' and hope no one notices. Every single commentator here knows there is nothing in computing or software engineering today that will allow one to 'create' an 'ai' as the world understands the term yet no one questioned pushing intentionally deceptive communication. This end result of dystopian scaremongering amounting to nothing is there is now zero credibility and extreme suspicion of problematic in-built bias. At the minimum there must be some standards for machine learning solutions to be thoroughly transparent, open to verification and exhaustively tested for racist and sexist bias before any rollout, for anyone who cares about the impact of their work in the real world.
- acdc4life 8y ago
  There is no "ai", it's the same bull crap we had in the 80s. We haven't had true innovation in this space for a very long time, the current hype was caused by GPU, cloud, and abundance of data for training.
ackfoo 8y ago
Dude, kindly do not post things that are behind a paywall. Is the web already not crap enough for you so you felt compelled to make it just that bit crappier?
bawana 8y ago
And now Google is pushing DeepMind Healthcare in England?! Do they not learn from these examples? Anyone who uses EMR knows very well the beauty of copy-paste and this single little convenience is creating more disinformation than the Mt Everest piles of illegible scrawl that used to be medical records. At least in those days, a HUMAN had to write something. And how did everybody forget this little gem from the 1960s: 'GIGO' = garbage in , garbage out. I dont care how many layers your RNN has, it will NEVER get accurate data. Sick people are not reliable sources of info and it takes a HUMAN to strip off all the emotional distortion to get the facts.

257 points by hprotagonist 8y ago

I can’t read most of that article because it’s behind a paywall, but wow, is Watson just the largest attempt to sell vaporware of all time?? They had a super bowl ad! A few friends from university were hired into the Watson team as hardly technical PMs. I have never heard any of them describe what Watson is or does.

bunderbunder 8y ago
I'm not sure I'd call it vaporware. I've not used it myself, but the more objective reports I've seen indicate that Watson is very good at what it does. The problem seems more to be what it does - accurately interpreting and then answering queries written in natural language - isn't actually that useful. For example, for drug discovery: I'm guessing, off the cuff, that there are other drug discovery tools out there, and that, while they don't allow you to frame your searches in terms of natural language queries, that's probably not actually a problem in practice. Because the querying methods they've developed are presumably highly tuned to their real task, which is enabling a skilled and knowledgeable practitioner to specify what they're looking for with great precision. It's sort of like programming languages: The ones that are designed to be the closest to natural language (e.g., AppleScript) do have a gentle learning curve for absolute beginners, but they also turn out to be some of the very worst languages for trying to do any sort of serious work.
- sodosopa 8y ago
  Which is why data scientists use python instead of AppleScript.
sodosopa 8y ago
I’m running on prem instances of Watson and it’s far from vaporware but Watson has become more of a brand name. For the Watson product I use, It’s more of a wrapper to do AI and ML work, notebooks, containers and related tools for data science.
- elliekelly 8y ago
  Same here. And so far I have to say I'm quite happy with it. Is it the best tool for ML? Probably not. But it's certainly the easiest I've found so far and it works for what I'm using it for.
donkeyd 8y ago
I've commented this before, but Watson is nothing but branding for anything IBM does that is somewhat intelligent. There's pretty much no common technology between the different Watson products, so there's no 'core Watson' like many people believe. Some products with Watson branding are great and industry leading. Others suck. The latter tend to appear in headlines and severely impact the brand they established with Jeopardy and their ads.
- apohn 8y ago
  I suspect what happened was that Watson started out as a single product or focused suite of products. I'm just guessing, but then as the marketing induced hype started and growth/results were not on target , they realized there was a lot of interest in Watson so IBM pivoted and started rolling everything under the Watson brand - products, consulting, cloud stuff, etc. Basically that way they could say Watson was a "success" and internally/externally it would be opaque as to where exactly the profits and losses were. I say all this because I'm at a place now where this is happening. Hugh company with a supposedly game changing product that is mediocre (at best). Massive marketing campaign for "Product X" that is 100% buzzwords and 200% BS. After constant missed revenue targets and product disappointments internally and externally, company is clearly pivoting (but not saying so) to rolling everything under "Brand X."
  - nintendo95 8y ago
    I worked there (Watson Health) a few years back. All that happened is IBM. That's right. They bought us (a small start-up) and this is what they did in a year to us: 1. replaced managers with their own who didn't get anything, kind of old date executives taken away from mainframe 2. banned remote work (this costed them a few brilliant engineers) 3. opened one huge open space full of noise and chatter, I mean sales team next to the development team, etc (this cost them even more headcount among experienced developers) 4. For months we did nothing! There were whole teams doing nothing for months on an end. I wrote zero lines of code in over 6 month period. Why? Because management didn't know which direction should be taken... and they kept hiring too! They kept hiring new developers when the ones already at place had absolutely nothing to do! This is from my (simple developer) perspective. Not sure how it looked in sales, among executives, etc. But that was very weird environment. I had great Manager who asked me to learn react and take courses in react (mind you it was two years back!) As we "might want to do something in react in the future". So I basically spent my last six months there learning React, aka preparing to the job interviews... they even got us paid courses and all. I mean... IBM. And once they fired me (these were lay-offs, thousands affected, many of them just hired in past year, like myself) I was paid severance pay too. I went there worked a year, last 6 months was learning for job interviews... fantastic pay too. IBM is crazy.
gowld 8y ago
This is what IBM has always done. 20 years ago they pretended that their Deep Blue chess engine was in all their IT products. It's similar to how DeepMind AlphaGo is PR for Google's AI effort, but IBM takes it to comical extremes.
- wpasc 8y ago
  I agree with you that IBM is full of shit, Google too to a degree, but Google has used AI in a variety of their products even down to data center cooling processes. I don't think IBM and Google are comparable in this respect.
- ne0sapi3n 8y ago
  > 20 years ago they pretended that their Deep Blue chess engine was in all their IT products [citation needed] IBM did not do this, though funding the Deep Thought team from CMU was cheaper than a Super Bowl commercial and brought more durable effect.
notfromhere 8y ago
Watson is the branding term for an army of outsourced consultants (mostly in India) plugging away at the raw data. Watson is not very good, and largely doesn't exist outside of slick marketing campaigns.
dekhn 8y ago
there are large amounts of interesting things in watson but it was still overmarketed for what it can do.
VeninVidiaVicii 8y ago
That's really interesting. I work in the emergency room, and pretty much our entire approach up patient care is algorithm-driven. As soon as the patient checks in, their demographics (age, sex, etc) and vitals are fed into a mysterious program, which suggests an acuity level, and basically drives the whole course of treatment. The patient is required to be tested for a variety of disease processes based on the AI-generated differential diagnoses. Honestly, I thought/still think medicine is headed toward this, which is why I decided to go into research.
- darrin 8y ago
  What program is that? Is that something Epic/Cerner/whatever does, or some add-on?
  - VeninVidiaVicii 8y ago
    It's MedHost with a lot of modifications specific to our hospital.

imroot 8y ago

I have a friend who is an OB/GYN Oncologist in Indianapolis. Her hospital had IBM Watson Health on campus last year, and she mentioned to me in passing last month that they finally had kicked IBM off of their (learning) university's campus. When I asked her why, she said: "Often times, Watson would recommend courses of treatment that would be completely incorrect, if not detrimental -- or even sometimes lethal -- to the patient. It became more of a hassle than a learning tool."

devoply 8y ago
It's good we still have doctors to provide oversight for this sort of thing. Imagine a future where machines take over because they are thought to be smarter than doctors and then start killing people and their mistakes are written off because the people were already sick. Like an Elizabeth Holmes AI.
- euler_ 8y ago
  Does it make a difference if they preform better than human docs? The same thing happens now with human docs.
  - ethbro 8y ago
    The difference is the expectation of fallibility with human docs. Which is why checks are built into the system. I have zero trust in IBM to market their ML products correctly so that proper checks are maintained. Especially since technically, explainability is still an active area of ML research.
    - freehunter 8y ago
      >I have zero trust in IBM to market their ML products correctly so that proper checks are maintained. That's a problem, yes, but it's not a new one. Vendor management has been around for decades, centuries, millennia maybe. If I contract out part of my job, it's still my responsibility to make sure the contractors are doing their job right. "But their marketing said..." or "but their sales guys said..." is not an excuse and everyone knows that. Doctors noticing that Watson is wrong is expected. Doctors missing the fact that Waston is wrong is a failure of that doctor and the doctor who didn't check the results is the responsible party. The checks don't come from Watson, the checks come from humans who oversee Watson. If Watson is wrong often enough that it's hindering the doctors, then kicking it out is the right call. But there can never be an argument of "Watson got the diagnosis wrong and that's why the patient died" because ultimately IBM is still just a vendor and Watson is still just a contractor.

deleted 8y ago

[deleted]

deleted 8y ago

[deleted]

StreamBright 8y ago

No surprise here. IBM has stopped producing usable engineering solutions long time ago.

wpasc 8y ago

I sincerely hope IBM's failures do not deter further progress in trying to innovate in healthcare technology. I believe healthcare vitally needs GOOD/USEFUL technology and AI applications because medical progress is not advancing quickly enough by several metrics. I hope IBM continues to fail and becomes a case study in the failure of massive bureaucracy.

PedroBatista 8y ago
IBM will not fail, Oracle will fail first than IBM and that's unlikely to happen. You see, they are NOT tech companies. They have something you would call "tech", they sell something that looks like "tech" but they really solve problems people with access to large amounts of money have. In other words, they produce cushy and warm beds for C-level execs to have a good night of sleep, and everyone needs some bed to sleep every night.. Technology is a risky and volatile business and IBM has more than 100 years in business.
- pm90 8y ago
  Spot on. IBM really does capitalize on the adage of nobody being fired for buying IBM. Their biggest strength has been in their marketing rather than actual technology. Note that, being a large org, they do have teams doing excellent work too. IBM research, some of the cloud, design etc. teams seem to be doing good stuff. But most of it is still overpromising and underdelivering while charging clients a lot of cash.
  - exelius 8y ago
    This is less true than it used to be. Oracle/SAP are largely recognized as multi-billion dollar technology quagmires these days; integration has become so much easier that cobbling together a bunch of SaaS solutions to replace a centralized ERP is a viable approach. Likewise, IBM has become known as a vaporware vendor. They combine the worst parts of management consulting with an offshore / outsourced development process, so I guess it’s not a surprise they ended up like all the other companies that sell business tech solutions (Infosys, Wipro, etc). They’re often mentioned in the same breath these days.

tolstoy77 8y ago

a lot of these AI companies products are really terrible. Has anyone ever tried the AI API models from clarifai ? Just so unaccurate. It seems like a scam. I've also had a really bad experience with watson's speech to text apis.

IshKebab 8y ago
Yeah their speech recognition API was by far the worst of the ~5 I tested a few years ago. Like, almost as bad as Sphinx.
linuxftw 8y ago
I'm pretty sure it's all a big scam, at least the crap that comes out of IBM.
- tolstoy77 8y ago
  +1
pmart123 8y ago
Watson’s is awful. Google’s default models are by far the best followed by Microsoft at a distance. Then, many of them are so bad that it is laughable.
- gbersac 8y ago
  Do you have any insight about AWS IA ?
  - pmart123 8y ago
    Haven’t tried it actually yet.
  - pmart123 8y ago
    I tested most of the solutions a year or two ago before AWS rolled out Amazon Comprehend. I found that Google’s API combined with some code and open source packages ended up being additive for entity recognition, classification, and semantic text. Unsurprisingly, Google’s custom search API’s are also worlds better than the other places, as is it’s Maps API. I haven’t tried any of the image classification API’s, but I won’t be surprised if Google won here too.

haolez 8y ago

Slightly unrelated, but I hope IBM doesn't ruin Red Hat and CoreOS. I would love to see the CoreOS tools to gain more adoption.

jniedrauer 8y ago
I'm a long time Fedora user. I love the sane defaults. It's just a solid workhorse. But I've been feeling pretty uneasy about using it lately...
- jsight 8y ago
  Alternatively, a lot more IBMers will be able to use Fedora and it may improve even more. I don't see them having a strong reason to do any damange there.
  - aquaticsunset 8y ago
    Maybe? IBM is mainly a Windows and Mac house for dev machines, with Ubuntu being a very, very small portion of workstations. I’m not sure the acquisition will create a mass exodus from the status quo there.

abdulhaq 8y ago

I suggest we take each AI doctor and place it in a self-driving car. The AI doctor will have a speaker to talk to the AI driver. Then take all these self driving cars with their AI drivers and passengers and get them all to drive together in circles on a race track. The last car not to crash gets to survive and is taken to the Whitehouse where it can advise Trump on what Mexicans and Muslims really think.

codinger 8y ago
ummm https://cdn.vox-cdn.com/uploads/chorus_asset/file/6238309/Screen_Shot_2016-03-24_at_10.46.22_AM.0.png https://cdn.vox-cdn.com/uploads/chorus_asset/file/6238309/Sc...

foobiekr 8y ago

Slowly, I am noticing that people are starting to question the abilities of machine learning systems and, in particular, seeing a lot of the transparent AI-washing that is now going on for systems that aren't AI at all. This is one of the most positive changes in the tech industry, honestly.

Barrin92 8y ago
Machine learning and neural net systems seem to be the inverse of the "uncanny valley", call it the "miraculous hill" if you will. They're just good enough to, with insane amount of computing resources, churn out economically viable results, but in my opinion they're the biggest setback in actually developing intelligence or cognition in a long time. There's very little insight to be gained from what they pick up, and they function purely in a stochastic sense. Even the best ML algorithm has no ability to reason at a high level or produce counterfactuals, it works purely by correlation and still, in those 1% edge cases, will be as helpless as anything else. That might be good enough when selling advertisements, but in automated cars and healthcare treatment, this sort of failure is not an option.
- hodgesrm 8y ago
  ML and neural nets work surprisingly well on a lot of boring but important use cases. Optimizing online ad placement might be humdrum but it's the foundation of a $100B market in the US alone. [0] In my opinion the real hype is that ML has become so popular that new practitioners tend to forget other analytic techniques like SQL data warehouses. Interestingly these are starting to absorb ML capabilities like logistic regression, which are now accessible through SQL and can benefit from MPP and vectorwise query execute in DBMS types like ClickHouse, Vertica, and Google BigQuery. [1] https://www.forbes.com/sites/danafeldman/2018/03/28/u-s-tv-ad-spend-drops-as-digital-ad-spend-climbs-to-107b-in-2018/#31c701a77aa6 https://www.forbes.com/sites/danafeldman/2018/03/28/u-s-tv-a... Disclaimer: I work on ClickHouse.
daenz 8y ago
https://i.imgur.com/FXi6NXb.jpg https://i.imgur.com/FXi6NXb.jpg According to the trend, we're approaching another AI Winter

cromwellian 8y ago

Basically, Triumph of Sales over Engineering.

notyourday 8y ago

Watson's applications outside of a quick search of existing datasets are vaporware. I'm amazed not a single customer has gone public with it when so many of them are whispering about it on LinkedIn.

distant_hat 8y ago

Slightly over a year back, we were contacted by IBM to try out Watson under some program for startups. Starting out, they asked us to give them a try. We gave them a NLP related task and they were confident they would be able to do a good job of it. After multiple meetings over the next 2 months, they barely produced any useful output. In the meantime, an intern with us got a pretty decent solution using just Python. Our management did get invited to lunches and meetings for selling them some more products. During one of such invitations, since our CEO couldn't make it, I was sent. I was surprised by how shoddy their presentation was given that I thought they might not be doing a good job on the problem we gave them but they should be slick on the selling side. Their presentation seemed like it was out of 2005 or something.

opportune 8y ago
Wow, I have been in this exact situation as an intern
c0mpute 8y ago
What do you think of your CEO now :)?
- distant_hat 8y ago
  Well, he was pretty good, he let us take the decisions completely so we never paid anything for it other than in time. Moreover, we weren't flush with cash or anything so they wouldn't have made much anyway.
sodosopa 8y ago
Python is included in the most recent versions of Watson Studio ML. Were Jupyter notebooks included when you originally tried the tasks? I’m curious why the included python didn’t work and the standalone did.
- beefcafe 8y ago
  That's like saying book A is in Spanish and book B is also in Spanish, so why did you like book B better? Python is just a language, what you do with it is what matters. Sounds like they built something completely distinct from Watson and it did the job.

bilater 8y ago

'And the trough is coming...' - Gartner

batmansmk 8y ago

Watson is more a marketing term than a technical connection between products. I invite you to test "Watson Tone Analyzer" https://tone-analyzer-demo.ng.bluemix.net/ https://tone-analyzer-demo.ng.bluemix.net/ : - "I like this product." => "this is an analytical opinion with neutral emotion." - "I like it" => "Tentative, 50% happy answer." - "It's not a bad product." => "analytical". - "It's not a bad tool" => "joyful answer".

darkpuma 8y ago
- "Spiders are not insects." => "fear" Exactly what I expected, and also flat wrong. - "There's a monster under my bed." => "joy" Fascinating.
- chillacy 8y ago
  You'd think they'd pair it up with a naive bayes classifier at least.
hodgesrm 8y ago
Thanks. I tried it myself with text examples similar to yours. The answers are a bit bewildering.
tiagobraw 8y ago
I tried it with the first phrase on the page "This service uses linguistic analysis to detect joy, fear, sadness, anger, analytical, confident and tentative tones found in text." response: Fear: A response to impending danger. It is a survival mechanism that is a reaction to some negative stimulus. It may be a mild caution or an extreme phobia.
CSMastermind 8y ago
> Watson is more a marketing term than a technical connection between products. As far as I can tell there's absolutely no connection between the products being labeled as 'Watson' and often times very little machine learning taking place either. I've interviewed several candidates from IBM over the last ~3 years and almost all of them worked on something with 'Watson' in the name. When we whiteboarded out the architecture of what they worked on it was mostly automation with _maybe_ a touch of machine learning thrown in by a module written by someone else. Very few of them were able to pass the technical interview.

deleted 8y ago

[deleted]

sonnyblarney 8y ago

For reference, here is the IBM Watson VP hustling it [1] on Charlie Rose. Funny in retrospect. Especially that haircut ... [1] https://charlierose.com/videos/29530 https://charlierose.com/videos/29530

ct520 8y ago

How is Watson not the butt of jokes in the tech industry by now? Seriously, never heard anything that didn’t end up being smoke and mirrors.

DebtDeflation 8y ago

Good. Focus on the use cases that actually work, namely a chatbot for handling customer service questions and an enterprise search engine.

aquaticsunset 8y ago
Can’t tell if sarcasm, but IBM’s had massive headache and trouble in this segment too. Good customer service agents are expensive. Bad ones are cheaper. But both still have a far higher success rate than conversational bots do. And a hybrid approach is, again, harder than one would initially assume (you know... why it’s painful and infuriating to press fifteen numbers on the phone to get to a real person? Same concept)

duxup 8y ago

There have been a couple articles that a lot of issues with Watson was how it was sold. Watson was being sold as sort of a magic drop in solution that could do oh so many things. In reality Watson took a great deal of work to carefully structure and process data, evaluation of the output, and more thoughtful of approaches to further refine the information and evaluate outputs. It also required a fair amount of involvement with the individual customer's staff. And there was always the possibility that because each use case was different... it simply wouldn't work out. To some extent it seems like it should have been sold as a journey... not cookie cuter solution for things that Watson had never encountered before. How do you sell that, I don't know, if I knew I'd probabbly be a pretty good salesman.

wessorh 8y ago

Even Watson wasn't a general AI. Hearing all the anecdotes from folks that tried to use it.. Narrow AI has the attribute that it can't be ported from one field to another. It might win at Jepordy but sucks at Oncology and Drug Discovery. All AI isn't the same. We are general AI and we can solve problems in asymmetrically bad situations. Narrow AI is not portable

opsiprogram 8y ago

I work with a group using machine learning for drug discovery (I'm not a biologist/chemist) but the bio people around me loved to talk poorly about Watson's drug discovery tool. Lots of focus on the algorithms in the comments here, but from what I could glean generally they lacked domain experts when developing the datasets... we spend 90% just finding the best data... and even then it's tricky. I think they may have had lower standards for the input into the system... garbage in garbage out.

throw2016 8y ago

It seems just like crypto the 'ai' hype cycle is coming to an end with zero achievements to show for itself beyond puffery and premature speculation about radiologists, drivers and others losing their jobs. And its not just hype, like crypto the ai hype stepped well beyond the line to outright fraud and deception with tech folks trying to pass off backward looking pattern matching as 'intelligence' and hope no one notices. Every single commentator here knows there is nothing in computing or software engineering today that will allow one to 'create' an 'ai' as the world understands the term yet no one questioned pushing intentionally deceptive communication. This end result of dystopian scaremongering amounting to nothing is there is now zero credibility and extreme suspicion of problematic in-built bias. At the minimum there must be some standards for machine learning solutions to be thoroughly transparent, open to verification and exhaustively tested for racist and sexist bias before any rollout, for anyone who cares about the impact of their work in the real world.

acdc4life 8y ago
There is no "ai", it's the same bull crap we had in the 80s. We haven't had true innovation in this space for a very long time, the current hype was caused by GPU, cloud, and abundance of data for training.

ackfoo 8y ago

Dude, kindly do not post things that are behind a paywall. Is the web already not crap enough for you so you felt compelled to make it just that bit crappier?

bawana 8y ago

And now Google is pushing DeepMind Healthcare in England?! Do they not learn from these examples? Anyone who uses EMR knows very well the beauty of copy-paste and this single little convenience is creating more disinformation than the Mt Everest piles of illegible scrawl that used to be medical records. At least in those days, a HUMAN had to write something. And how did everybody forget this little gem from the 1960s: 'GIGO' = garbage in , garbage out. I dont care how many layers your RNN has, it will NEVER get accurate data. Sick people are not reliable sources of info and it takes a HUMAN to strip off all the emotional distortion to get the facts.