35 ms·

A rising sentiment that IBM’s Watson can’t deliver on its promises

503 points by artsandsci 9y ago

oculusthrift 9y ago
the articles content is more just background about watson than having to do with the the title. as a side note: it just sounds wrong to have the words "hating on" in an article title.
throwaway_ibm 9y ago
I know someone who is intimately involved with IBM Watson, they are highly educated and constantly diss the system. Calling it, 'Just a large database'. If Watson was a true breakthrough, it should be gaining marketshare throughout it's specialities but it's not. Google is leading the industry with DeepMind; Facebook and Microsoft aren't far behind. I'd encourage others to be very skeptical of the PR that IBM is pushing about their Watson problem. disclosure: I haven't read the article but wanted to share a related story.
peteretep 9y ago
A couple of years ago I was given a project that was essentially "Evaluate Watson APIs to see if there's anything there we could make use of", and came away with the distinct impression that it was largely smoke and mirrors, and there was very little that was either effective or interesting there.
- drvdevd 9y ago
  Could you elaborate a bit? I'm not affiliated with IBM or anything, just curious what you were looking for and what was missing.
  - dr_zoidberg 9y ago
    I'll give you my experience. Manager asked me if we could use the Watson Speech-to-text API a couple of years ago, because he was impressed with the ads he'd seen. I worked for a week. Turns out the "dialect" of spanish that is spoken in Argentina (and this was tested against porteños, not even the accents from the provinces!) is utterly unsupported. Even when speaking slowly, it'd throw a random set of words with even more random looking probabilities. Manager was expecting that after a week we'd have something akin to Googles "hello google!" at hand, through Watson. It took a couple of coworkers to come up with even more impressive broken examples to finally convince him that the technology was bad.
    - _jal 9y ago
      An interesting example of negative-sum advertising. Everyone involved lost money. I wonder how often that happens.
      - drvdevd 9y ago
        I hadn't even considered that point of view ... quite interesting. EDIT: I have to say, I bet this happens quite often in fact, especially with software.
    - drvdevd 9y ago
      Interesting! Thanks for the concrete example. It's bad enough trying to convince managers as to the specifics of why some product doesn't work for your project.. even worse when said product has great marketing.
- maxxxxx 9y ago
  Our IT department has been talking about Watson for a while but I never could get anything out of them what it's really supposed to do. From what I can tell it's a very slick marketing term with some mediocre tech behind it.
  - gaius 9y ago
    It sounds like Autonomy. Or Palantir.
  - dguaraglia 9y ago
    As far as I understand (and I have only passing knowledge on this, as we've been evaluating machine learning technologies for our startup) IBM Watson is just a set of APIs you can use to build your own solution, and the APIs themselves are not top-of-the-line (Google's are.) The other day I saw a TV ad for Watson for the first time: the ad showed a bunch of aeronautical engineers who instead of having to take all kinds of measurements of airflow around a hull or who knows what else (the TV was muted) now sat behind a single computer that would do the whole thing for them. It was such bullshit, I couldn't believe my eyes. Imagine Amazon promoting SES as this amazing tool that'd automatically send marketing emails and create a marketing funnel and drive user acquisition cost down by a factor of 100X, without you having to do anything. That's the level of vaporware they are selling.
    - RHSman2 9y ago
      Very few out of the box machine learning services work other than speech-to-text by the worlds greatest company in that field (google). It's not a case of roll your own but find companies that can help you deliver s solution to your specific problem. (That is my business) This is the absolute bullshit of Watson. What it could be is a series of very good datasets and then being able to build models (based on recipes) and deploy them. It's so unbelievably clunky though. And Bluemix? Urgh
- chrisco255 9y ago
  My team and I were able to build a pretty good chatbot with Watson two years ago. We used Google for speech-to-text, however, as we found IBM's to be behind in that area. We had to use a Node.js proxy with a couple libs to parse out some natural language features (like time)...but overall it worked pretty well. At the time, it was the only open system I knew of to write things like this. Today, of course, there's the Alexa API from Amazon and Microsoft's LUIS, which are probably vastly superior for this sort of thing.
- iampims 9y ago
  Their text to speech API is pretty decent and was the cheapest available until Amazon Polly.
ams6110 9y ago
What? A brand name which is just a word meaning "IBM Enterprise Products and Services" doesn't really live up to the marketing hype? I can't imagine such a thing.
batmansmk 9y ago
You can try by yourself. https://alchemy-language-demo.mybluemix.net/ https://alchemy-language-demo.mybluemix.net/ Imagine analyzing product reviews to determine if it was positive or negative. Type "I like it", and see the inaccurate targeted sentiment (neutral sentiment instead of positive).
- guyfawkes303 9y ago
  I tried this, and while 'targeted sentiment' may be neutral, 'document emotion' has Joy at 0.880435 vs the rest at 0. If you change your example from 'I like it' to 'I like puppies', the 'targeted sentiment' changes from neutral to 'puppies positive 0.550431'. While the first example you found does seem odd, the rest seems to work quite well. Maybe it just has problems attaching 'it' to a 'target'.
  - foldr 9y ago
    It has trouble with negation outside the relevant clause. So it is not fooled by e.g. "I don't hate it" vs. "I don't like it", but the first of the following two sentences is rated more negative than the second: I don't seem to hate it. I don't seem to like it. Edit: In fact it can even be fooled just by increasing the linear distance between a positive term and negation. The following is rated +0.6! I don't in any way shape or form like it.
- tensor 9y ago
  Google has a similar API: https://cloud.google.com/natural-language/ https://cloud.google.com/natural-language/
  - oceanswave 9y ago
    Microsoft too: https://azure.microsoft.com/en-us/services/cognitive-services/text-analytics/ https://azure.microsoft.com/en-us/services/cognitive-service...
- te_chris 9y ago
  Alchemy was an acquisition.
dboreham 9y ago
Bundle up for the second AI Winter...
- sjg007 9y ago
  I think we have a good 10 years of engineering production systems based on current tech.
- bitL 9y ago
  Third actually. But it's just Watson; Deep Learning and its derivations are here to stay as they are already state-of-art and close to humans.
  - z3phyr 9y ago
    They are not close to humans at all. They are good for very specific tasks only. Edit: Spelling
    - bitL 9y ago
      You are right - for specific tasks. E.g. in computer vision they are able to surpass humans for static image recognition. For image segmentation they are getting close already. For generating art they IMO have surpassed some "artists" already ;-) If you think about it, even with 90% accuracy they are worth it as they could almost automate some cognitively-difficult tasks and for those instances that they can't do it properly, a human can intervene. So instead of 10 humans watching over something you end up with a single one. It would take some time until all we have in the research is in production and that will change a lot of things. After that there might be another AI winter (who knows if we can do general intelligence?), but with current pipeline it might take a while to reach plateau.
      - deleted 9y ago
        [deleted]
  - tannhaeuser 9y ago
    What's the second AI winter then? I only know about the 1986-1990 era, where "AI" was used in a broad sense covering everything from Prolog, expert systems, LISP, heuristic search algorithms, fuzzy logic to ML, neural networks and their generalizations. I've still got great shiny magazines from that time, though.
dpflan 9y ago
I like how IBM does very elaborate marketing ploys to hype their wares: like Deep Blue competing against Kasparov and Watson competing against Jennings to showcase IBM's engineering prowess. But it does sell the idea pretty well I think, but perhaps the idea is too grand/far ahead of the present.
- forgot-my-pw 9y ago
  Deep blue was just a brute force machine. Perhaps Watson isn't much different, just slightly smarter.
  - sgt101 9y ago
    Watson that won Jeopardy was a fantastic advance in information retrieval. Google introduced the knowledge graph a year later, but domain (jeopardy) specific watson was terrific. The sad thing is that they got captured by PR rather than sense; a focus on customer service and my bet is that they could have disrupted Oracle/Seibel out of 75% of accounts. IBM would be ridin' high! BTW. This is still on the table for IBM if they are prepared to invest 18mths and focus gung ho on Oracle. In 18mths it won't be as "other players" will have moved, like Sauron, to hit this market.
  - zone411 9y ago
    Saying something like this does a big disservice to IBM's real accomplishment. Code a brute force chess engine, add some alpha-beta pruning, transposition table, or even some of your heuristics and let us know how it does. Just because it's not an adversarial RNN or whatever, it doesn't mean it's brute force.
tangue 9y ago
Crédit Mutuel (a french bank) has adopted Watson [0] and it's not encouraging : it was supposed to help answering emails, : they had to describe manually the concepts in emails and create topics in which looks a lot like decision-trees (and reminds me of this 1985 ad for Texas Instrument's Lisp AI https://www.scientificamerican.com/media/inline/blog/File/Dewdney_Mandelbrot.pdf https://www.scientificamerican.com/media/inline/blog/File/De... scroll to see the ad) Indeed the whole thing looks like a database with basic AI as a sales argument... [0 - in french] http://www.silicon.fr/credit-mutuel-non-ia-watson-magique-173889.html http://www.silicon.fr/credit-mutuel-non-ia-watson-magique-17...
- deleted 9y ago
  [deleted]
- Spooky23 9y ago
  Totally. I observed a support chatbot project that didn't quite meet up with the vision. Even based on the technical marketing claims the consultants were coming in with, it was a product less capable than the IBM Prolog-based expert systems that I built out 15-20 years ago to triage and correlate network events in a large WAN infrastructure.... and that was a product with a whole slew of major implementation and operational problems!
  - fapjacks 9y ago
    Hey just wondering: Have you written about your exploits before? If so, where can I read about them? Sounds interesting.
    - seanp2k2 9y ago
      It's not just interesting, it's C-O-M-C-A-S-T-I-C!
    - Spooky23 9y ago
      I haven't other than on HN, but I've been lucky in that I've had the opportunity to participate in some awesome and not-awesome stuff. Maybe I will start!
moomin 9y ago
Completely off topic, but didn't IBM have a system called Watson in the 1990s that was used by the police? Try as I might I can't find a reference for it anywhere.
- rjsw 9y ago
  Maybe you are thinking of the product from Harlequin [1]. [1] https://en.wikipedia.org/wiki/Xanalys https://en.wikipedia.org/wiki/Xanalys
  - moomin 9y ago
    I think you're right... thanks.
- sgt101 9y ago
  Scotland Yard had a system called Holmes https://en.wikipedia.org/wiki/HOLMES_2 https://en.wikipedia.org/wiki/HOLMES_2
- Retra 9y ago
  Watson is the name of the founder of IBM, so there's a pretty good chance his name was attached to various things over the years.
currymj 9y ago
it's just a brand name at this point, which they attach to any machine learning they develop or acquire, and they should stop trying to sell it as a distinct technology.
strict9 9y ago
Many years ago when I worked for a company that decided our existing ecommerce app was too terrible to fix and would be too much effort to rebuild, we talked to a number of vendors, including IBM. The marketing materials and salespeople made a compelling case, but deeper dives into the app itself and the support engineers behind it convinced even the most enthusiastic internal cheerleaders to look elsewhere. In recent years as news articles heralding the future of Watson for various industries (including healthcare and supply chain), I predicted a similar path. An amazing product in a very narrow environment designed specifically for marketing and selling purposes, and not very adaptable. FTA: “And everybody’s very happy to claim to work with Watson,” Perlich said. “So I think right now Watson is monetizing primarily on the brand perception.” This is painfully obvious, as this has been IBM for a very long time.
iamleppert 9y ago
The real crime is in using cancer kids to sell your product. I mean, who even does that? Even if you could cure cancer for kids, I find it incredibly tacky to go around making commercials about how you can cure cancer for kids, which aren't targeted at those who actually are in the position to use the technology, and its used to market to other tangential industries where the real money is. It's just despicable and you can tell right there its smoke and mirrors. There's a special place in hell for anyone working at IBM or involved in the Watson project who is supporting this thing. It's damaging legitimate deep learning/machine learning industry and generally making a fool out of IBM, AND giving children with cancer false hope....just so IBM can try and stay relevant and make money?
- orf 9y ago
  > The real crime is in using cancer kids to sell your product. I mean, who even does that?? Charities?
  - oblio 9y ago
    Ah, yes, the noblest of giants, the most charitable of charities, Big Blue.
  - ryanmarsh 9y ago
    Can confirm Source: former president of children's cancer research foundation In my defense we were actually trying to make a difference in the world not sell software.
    - radarsat1 9y ago
      > we were actually trying to make a difference in the world not sell software. I mean, to be fair, I don't think selling software and trying to make a difference in the world are at odds with each other.
- davidreiss 9y ago
  Kids are used by scams/charities to lure in people for all kinds of things. The latest one is the cars for kids scam which used kids to get donations and who knows what they did with it. http://www.nbcnewyork.com/investigations/Kars-for-Kids-Cars-4-Kids-Real-Estate-Investments-Investigation-140955003.html http://www.nbcnewyork.com/investigations/Kars-for-Kids-Cars-...
ceedan 9y ago
Does IBM itself even "use" Watson?
- marvel_boy 9y ago
  Of course not. Watson is just a marketing trick to charge for consultants. I worked there.
  - ceedan 9y ago
    This has always been my assumption. If Watson did anything close to how it's advertised, IBM would have a big competitive advantage by actually using it.
outside1234 9y ago
You don't say! This is IBM consulting ware? Who would have guessed!
notfromhere 9y ago
The dirty secret is that IBM Watson is just a brand for their army of data consultants, and their consultants aren't very good. In my experience working for a competitor in this space, IBM Watson was widely agreed to be smoke and mirrors without much going on
- ibm_throwaway 9y ago
  Former IBM Watson data scientist here. At first I was annoyed by your comment, but then I realized how true it is and sulked instead. The good consultants are leaving as soon as they hit the end of their first promotion cycle. The duds stick around and assume the upper-level manager roles. The group is in bad shape now and poised to get worse in the coming years.
  - ethbro 9y ago
    As someone who's worked in that space, I'll repost the short version of a comment I made before: it's an economics problem. Services Profit = Bid - Employee Cost + Upsales Bids are only so flexible. Employee cost is much easier to minimize. So inevitably, all consultancies trend towards providing the cheapest employees that will satisfy the customer. And if the customer isn't constantly vigilant, the quality can get pretty dire indeed (BAs who just learned to "code" yesterday).
    - gaius 9y ago
      Funny you should mention BA, since they have had a series of high profile IT meltdowns since outsourcing...
      - ethbro 9y ago
        Sorry, in my use *Business Analyst (aka business majors who want to get into "that computer thing") BPO / offshoring is a whole other can of rotten worms.
        9y ago
speeder 9y ago
I actually love the idea of Watson being used for healthcare... Sadly I think it is being used wrong... IBM is focusing on using Watson to cure very specific diseases, like certain types of cancer. I think a far better use for Watson would be to do initial diagnosis, for example my life got massively delayed because I got hypothyroidism as teenager, but only using internet data I could self-diagnose and self-treat (because doctors are still unwilling to help, not trusting data, and before someone come berate me for self-treatment, it is working...) as adult I could finally get my life 'started' (hypothyroidism affect physical and mental development, and slows down metabolism and the brain) During my quest I met many, many, many people on internet, that had self-diagnosed with something using the internet as a tool. All of us would have been diagnosed properly if Watson was being used on the doctors office, using its data crunching capabilities and symptoms as input to find out what problem we had. (in my case: I have Hashimoto's disease)
- vslira 9y ago
  Why would professionals support tech that makes their labor less scarce? Unfortunately this change won't come from doctors
  - pwthornton 9y ago
    Doctors might fear it, but healthcare administrators would certainly love for a tool to help make diagnosis faster, more accurate and less expensive.
    - sgt101 9y ago
      If it worked in that way, which it currently doesn't (see comments about question answering systems and Jeopardy) in terms of the state of the art, we're... maybe 30 years off that, like as in we're 30 years off fusion. Think about the human condition, think about human biology. Be clear- these are systems beyond current science... and doctors deal with them every day!
      - roywiggins 9y ago
slackingoff2017 9y ago
IBM is a dying giant, I've seen it languishing for years. Their massive screw up was a decade ago when they decided shareholder value was more important than having good engineers. They've since gutted their R&D departments and all that's left are duds and underpaid undereducated consultants rented from places like Accenture. The only good thing to come out of IBM in years is their Hyperscan regex library and unsurprisingly they don't market it at all or build practical applications with it
- bsg75 9y ago
  The same Hyperscan as here? https://github.com/01org/hyperscan https://github.com/01org/hyperscan Project homepage suggests this is an Intel project.
  - thealfreds 9y ago
    Also the LICENSE > Hyperscan is licensed under the BSD License. > Copyright (c) 2015, Intel Corporation
- KingMob 9y ago
  The Hyperscan website (https://01.org/hyperscan https://01.org/hyperscan) says it was built by Sensory Networks Inc, and then bought by and released as open source by Intel. No mention of IBM. Are you thinking of something else?
  - slackingoff2017 9y ago
    Oops, yeah. They haven't produced anything then :)
    - p1esk 9y ago
      Jeopardy, TrueNorth?
- WalterBright 9y ago
  In the 80's IBM was an unstoppable juggernaut. In the 90's it was Microsoft. Then Apple. Now Google/Amazon. Conventional wisdom is that, in a free market, a single corporation will grow to rule everything. But that never seems to pan out. Part of the reason is that the larger an organization gets, the more inefficient and bureaucratic it gets, the less it is able to adapt. See "The Innovator's Dilemma", a book everyone interested in business should read.
potatoman2 9y ago
Did they really need to stick an insult at Trump in there?
- optimuspaul 9y ago
  yes
- colejohnson66 9y ago
  As much as I hate bringing politics into places it wasn't, this wasn't from the author; It was from a quote > "IBM Watson is the Donald Trump of the AI industry—outlandish claims > that aren’t backed by credible data," said Oren Etzioni, CEO of the > Allen Institute for AI and former computer science professor. > "Everyone—journalists included—know[s] that the emperor has no clothes, > but most are reluctant to say so."
- deleted 9y ago
  [deleted]
etiam 9y ago
It's tempting to start whispering winter is coming, but I think one may reasonably hope that the current fashions at large have enough nuance to differentiate between this particular marketing gimmick and the broader developments in ML. Personally I'd be happy to see the paragraphs/minutes at the beginning of far too many interviews about "intelligent" machines exchanged, from straightening out the misconception that Watson is an example of this new hot "Deep Learning" thing and one of the pinnacles of achievement in the field, for some type of more valuable type of commentary from leading researchers.
PaulHoule 9y ago
This was my opinion when they started running these ads. My opinion has actually softened a little. Some of the cognitive services they are offering today are not half bad; also I can say their salespeople are doing a gangbusters job in places.
crsv 9y ago
Replace IBM's Watson with anything branded with "AI" right now and themes in the article still hold up.
- bsg75 9y ago
  It is likely that AI has not matured to the level its being marketed at currently? And that the over hyping of it today will negatively impact its adoption when ready for general use?
  - macintux 9y ago
    Seems like AI has been over-hyped for decades, just rarely with someone as prominent as IBM leading the charge.
  - scottlocklin 9y ago
    It happened in the 80s. https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter It will happen again. The use of the phrase "artificial intelligence" itself ought to be forbidden until such a thing actually exists in the corporeal world.
    - Barrin92 9y ago
      >It will happen again maybe an 'AI recession' is plausible but a large scale loss of interest is unlikely given that AI / ML / deep learning (pick your favourite word) is already too embedded in commercial application. Like always the hype in the field is strong, but this time we actually have the data growth we need to produce palpable results (and already have).
riku_iki 9y ago
Any first firings for choosing IBM?..
deleted 9y ago
[deleted]
et2o 9y ago
I saw a very humorous twitter exchange between a bioinformatician and IBM Watson's twitter account. The scientist asked them to provide any peer-reviewed ML publications and the best they could do was an abstract at a regional conference no-one has heard of. And it was a terrible abstract. It's completely marketing. IBM still has a good name among people who don't know much about technology. They're trading on this and the current saturation of 'machine learning' in the popular press.
- sgt101 9y ago
  That's mad as IBM have loads of peer reviewed ML publications. 3 minutes of searching : http://researcher.watson.ibm.com/researcher/view_group_pubs.php?grp=5443 http://researcher.watson.ibm.com/researcher/view_group_pubs.... http://researcher.watson.ibm.com/researcher/view_group_pubs.php?grp=4384 http://researcher.watson.ibm.com/researcher/view_group_pubs.... c'mon
  - et2o 9y ago
    I should have added: healthcare. No shit IBM research has publications, they have their own journal. Also, 15 publications in 3 years is pathetic. c'mon
    - sgt101 9y ago
      Well, the operational people will struggle to publish because it won't be on their scorecard, so effectively they won't be paid for it. Additionally many of them won't have any skills relevant to creating an academic publication and no nice professor to help them do it. So if IBM are going to publish it will be their lab. In terms of publication rates, in universities it's a mill. All academics are measured on publication impact and because of the arbitrary and self reinforcing nature of impact the way to win is to sprinkle as many papers around as you can get your graduate student factory as you possibly can. The question is what is the actual utility of minting out the same paper with nine different authors?
blueyes 9y ago
IBM has almost zero credibility in deep learning and AI. They haven't hired anyone of note. They haven't produced any novel or influential research in the field in years. And yet they air these cheesy Dylan ads and the rubes fall for it. Watson is a Theranos-scale fraud, and it's finally coming out.
- pas 9y ago
  Was the Jeopardy think just a ruse? Was it something easy (something any ML/DL team could have thrown together in a year?)
  - martingoodson 9y ago
    No it was a hard engineering challenge. But it solved a very specific problem that has no relevance to most business use-cases.
    - ethbro 9y ago
      Also, at effectively unlimited cost. Are there any numbers how much they spent on making it work?
      - sgt101 9y ago
        30 people (ish) for 3 years + 10 million on hardware? The brilliance of the team... priceless. The cost of everything, the value of nothing : this is what will be written on the tombstone of capitalism.
- Jach 9y ago
  http://jeffjonas.typepad.com/about.html http://jeffjonas.typepad.com/about.html However he's no longer at IBM...
tCfD 9y ago
Obvious fix is for IBM to put Watson on a blockchain /s
- samfriedman 9y ago
  You jest, but they're not above "throw a blockchain on it" levels of buzzword-chasing: https://www.ibm.com/us-en/marketplace/z14 https://www.ibm.com/us-en/marketplace/z14
diego_moita 9y ago
> "In the data-science community the sense is that whatever Watson can do, you can probably get as freeware somewhere, or possibly build yourself with your own knowledge" Any suggestions about the freeware?
- sgt101 9y ago
  MLlib Tensorflow Lime There you go.
  - nl 9y ago
    MLib The Watson of machine learning libraries. As someone who has used MLib and raised a number of bugs and feature requests: Use Spark for feature engineering and then Scikit for ML.
simonh 9y ago
This just goes to show just how tragically far away we are from even beginning to build the rudiments of a strong general purpose AI. For all the fantastic achievements of systems like Watson and Alphago, and they are amazing achievements, they are radically optimised special purpose systems fine tuned to solving one extremely specific and narrow problem, and that problem only. Watson is a case study in this, but I know Google has big plans for applying the tech behind Alphago in medicine. I wish them every success, but I'm concerned they will hit similar specialisation issues.
- FeepingCreature 9y ago
  I would argue this only shows how tragically far away IBM is.
- sgt101 9y ago
  Medicine - the biggest prize, the biggest trap. Every complication, huge, mind blowing rewards. Maybe, if Google were to go 100% they could break the beast... but it's a 100bn bet.
  - simonh 9y ago
    Untill you actually have a viable, testable plan to implement it almost allege money you throw at it is pretty much certain to be wasted. The Manhattan project is often given as an example, but that project was based on solid theoretical models and repeatable experiments. They knew exactly what they were doing and how they were going to do it. The problem in AI at the moment is mainly developing talent and expertise, not capital funding. I'm all for advanced research into technologies. However we need to advance that research on as many different fronts as possible in a flexible way simply because we don't know how we are going to solve the problem of general AI. Guessing on an approach and spending billions on it would be a complete waste. We're simply not at a point where that would do any good.
    - sgt101 9y ago
      Agree, they should have focused on much more tractable domains before Medicine. I think that there's an business case issue here; MBA's see this as the mega win, the research staff can't say "it won't work" (because you are immediately flagged as an obstacle to progress and the execs either cease to listen because they don't want to hear, or you get bullied with an avalanche of attacks on your team or sacked).
ChuckMcM 9y ago
When I worked at IBM I expressed concern that the television commercials depicting a HAL9000 level interactive dialog system were dangerously overselling what Watson could do. The challenge, as I saw it, was that no matter how good the tools and products that were used to help companies with data analysis to improve their operations were, when they realize they can't talk to a cube and joke with it about misusing colloquial phrases their disappointment overshadows all the 'good' stuff it was doing for them. No relationship works well if it starts with a lie and as this article shows, people do take those ads at face value and assume there really is a talking AI inside of IBM. Then they are hugely disappointed when they find out it doesn't exist.
- ethbro 9y ago
  Over-promising has killed AI progress before (expert systems). So history's just repeating itself. People want to believe. They'll pay for their belief. Then when projects start to fail (~5 years after commercialization) they form the opinion that it was never going to work and they'll never do it again. ... then we wait another decade, repeat.
  - olewhalehunter 9y ago
    There's this trend of Expert Systems being commented on as a failure and starting some sort of AI winter, yet to my knowledge they were among the first commercially viable digital animation tools, the first high speed trading platforms, and are still being used successfully in military planning. Where's the evidence that they were a failure or that they were oversold?
    - sgt101 9y ago
      Well, there was this place called "The Turing Institute" which was in Glasgow. It was run by a very nice man called Don Michie. I met Don a handful of times, and he was extremely kind and generous to me. He worked at Bletchley Park and died aged 91 having hit a bridge at 120mph going the wrong way up the M11 at 2am sitting next to his wife who had defected to Russia thirty years before. Don was driving, at the time he had been blind for six months. Anyway, The Turing Institute existed because of Don's drive to commercialise Expert Systems. Don was a driver in the Alvey project which was a grand project to solve issues around Expert Systems in response to the Japanese Fifth Generation project. A man called Lighthill then rumbled the Expert System movement. If you want to watch a demolition of it see the following video : https://www.youtube.com/watch?v=yReDbeY7ZMU https://www.youtube.com/watch?v=yReDbeY7ZMU and the other 5 of them (you will find them in the sidebar!) Fun fact. In video 6 you can watch Christopher Strachey have a pop at AI. Christopher didn't work at Bletchley, he was too young, but he did work at Manchester with Alan Turing and they created computer music and love poetry in collaboration before Turing killed himself. Strachey went on to be the first Professor of Computer Science at Oxford, he was a scion of a great family of artists and poets. But enough of this. The Turing Institute was an attempt to say "we are right, look", but in fact it failed. Don was clear why, when you hit 40k rules knowledge bases became completely intractable. It was impossible to make improvements or changes. This was christened the Knowledge Acquisition bottleneck, and Don and others turned to Machine Learning (he liked to call it Behavioral Cloning) to solve it. So : 1. It was done to death by peer review. 2. It died commercially. 3. It's main protagonists decided to do something diffferent.
deleted 9y ago
[deleted]
d--b 9y ago
watson's mistake is to have gone the chat bot route. promising a natural language input for all underlying problems simply discredits everything else...
scottlocklin 9y ago
Yeah, well, "duh." What boggles my mind is people will read this, nod sadly, and continue not to notice that a whole bunch of what they think they know about machine learning, autonomous vehicles and so on is also marketing department hype.
- mattnewton 9y ago
  I don't know if that's fair. To take self driving cars as an example, I've seen cars driving themselves around mountain View and Waymo isn't promising much more than that. Watson had ads with an intelligent voice but they were selling a bunch of programmers to make expert systems like it was the 80's but in java this time. I think by comparison those other fields are promising less and delivering more. Even if there is still a gap between what people think will happen in the next 10 years and what we can likely deliver in the next 10, it's nowhere like what is described in the article.
  - scottlocklin 9y ago
    C'mon man! Google allows one to check on their promises and their progress. For example, from 2012: http://www.computerworld.com/article/2491635/vertical-it/self-driving-cars-a-reality-for--ordinary-people--within-5-years--says-google-s-sergey-b.html http://www.computerworld.com/article/2491635/vertical-it/sel... "Such "autonomous vehicles" will be a reality for "ordinary people" in less than five years, Google co-founder Sergey Brin said Tuesday." How's that working out for you Sergey? Self driving tech hasn't moved appreciably since the 90s. Computers are better. More people are trying it. Reality is, it's just collision avoidance with GPS. They rely on maps; what happens when the maps change? What happens when a kid runs out in front of one of these things?
    - mattnewton 9y ago
      Sergey isn't taking people's money based on that promise. As an engineer I don't beat people up about realizing how hard problems are and not releasing crappy products. Missing estimates happens. It's another thing entirely to try and pass off a product as something it isn't to a buyer.
Probooks 9y ago
Problem is deeper (and simpler). IBM does not look for clients, but rather victims. We clients end up being caught in an internal upsales fight. Nobody cares which is the best solution IBM as a whole can offer to you (their own people do not even know all their available tools!), but rather how much suboptimal stuff each salesman can load onto you. I'm on my way out of IBM...
dislikes_IBM 9y ago
IBM has a toxic culture. They are the vendor lock-in Gods. Every company I've ever worked for has cringed at the mention of IBM, never suggested them as a new solution, and always regretted whatever if anything they locked themselves into. They are the only company that charges you to sample their API's. They are the absolute worst, an infection that needs to be cured.
- johngarrison 9y ago
  Sometimes I wish I was wealthy enough to buy out horrible companies, just so I could have the power to dismantle them and end the suffering.
  - Method-X 9y ago
    You wouldn't be wealthy for very long ;)
- smugworth 9y ago
  I'm working on a team for a platform built on open-source software which actively replaced a previous IBM solution, and there are more services from other vendors being phased out in favor of internally developed tooling. Seems like a good long-term investment, in an era where people are constantly criticizing CEOs for short-term cash-grabs. Why shackle yourself to a vendor if you plan to be around for awhile.
ghostly_s 9y ago
I overheard a good-'ol-boy businessman at a hotel bar a few months back. He bore an eerie likeness to Bosworth from Halt and Catch Fire, and was telling a younger gentleman about a project he worked on. "...so Watson comes in and they Algorithm the whole thing..." I'm pretty sure he thought Watson was a person.
ExactoKnight 9y ago
Watson's Natural Language Classifier, in particular its categorization API, is actually pretty impressive...
jjm 9y ago
They had so much time to contribute but instead chose marketing and pushed into areas where they didn't really have a handle on yet. As in management didn't understand. I mean all the datasets, dozens of libraries, stunning NN demos and training sets, TPUs (multiple versions at that!) all could've come out of the company. Think if keras and tensor flow were from IBM. Or all those cars now running Nvidia Jetson, or mega datacenters running NV100s or Google TPUs. Shoot they even had a chance to enhance PowerPC ICs for NNs. Alas but nope.
filereaper 9y ago
I'm quite late to this thread, but I worked on Watson very briefly (not on the core development, but overall system performance improvements). I think there's a major misunderstanding of Watson which isn't helped by IBM's Marketing efforts. IBM Marketing has been slapping the "Cognitive" label on everything and is creating unrealistic expectations. The Jeopardy playing Watson (DeepQA pipeline) was a landmark success at Information Retrieval, its architecture is built largely on Apache UIMA and Lucene with proprietary code for scaling out (performance) and filtering & ranking. I'm not an expert on IR so I won't comment further. This is very different from Neural Nets that are all the rage in ML today. I'd like to point the following links from David Ferrucci [1] the original architect of Watson and this technical publication at aaai.org [2]. The DeepQA pipeline wasn't fluff, the intention was to take this question-answer pipeline and apply it to other verticals such as Law and Medicine, essentially replace the Jeopardy playing Watson's corpus of Wikipedia, Britannica etc... with Legal and Medical equivalents. Given its runaway PR success, the Watson brand was applied to many other areas which haven't been successful but I'd like to point out what the original product was here. [1] https://www.singularityweblog.com/david-ferrucci-on-singularity-1-on-1-pursue-the-big-challenges/ https://www.singularityweblog.com/david-ferrucci-on-singular... [2] https://www.aaai.org/Magazine/Watson/watson.php https://www.aaai.org/Magazine/Watson/watson.php
- nl 9y ago
  This is true. I've read all the Watson Special Edition papers numerous times[1]. The original Watson work was very good, and adding neural-network based models as alternative question parsers and answer rankers is a clear way forward. It's a pity IBM decided to use the Watson name for everthing they do in ML now. [1] http://brenocon.com/watson_special_issue/ http://brenocon.com/watson_special_issue/
- denzil_correa 9y ago
  > The Jeopardy playing Watson (DeepQA pipeline) was a landmark success at Information Retrieval, its architecture is built largely on Apache UIMA and Lucene with proprietary code for scaling out (performance) and filtering & ranking. I'm not an expert on IR so I won't comment further. This is very different from Neural Nets that are all the rage in ML today. The question I would like to ask is that if the architecture is built on UIMA and Lucence, what exactly did "Watson" do?
laichzeit0 9y ago
I had problems with Watson to the effect that not even the documentation matches reality. There are some fairly basic things missing from their NERC offering. I can tell you that the functionality that is missing is so basic (e.g. negation) that without a doubt, no one in IBM has ever used this offering in practise beyond a toy example. The idea that IBM Watson is some uniform AI in a box with a bunch of REST API's to "expose" its intelligence seems to be the sales pitch. It's not. It's just a bunch of acquired products (you can see this when e.g. Watson Knowledge Studio breaks and you see the Python scripts that glues everything together in the backend) that are poorly integrated, probably because the left hand has no idea what the right hand is doing. Caveat emptor! https://stackoverflow.com/questions/44796501/ibm-watson-knowledge-studio-entities-with-role-attribute-and-extracting-it-fro https://stackoverflow.com/questions/44796501/ibm-watson-know... https://stackoverflow.com/questions/44800879/ibm-watson-knowledge-studio-annotating-negative-negated-mentions https://stackoverflow.com/questions/44800879/ibm-watson-know...
chisleu 9y ago
I'm late but have something to add. Until last week I was on a 6 month contract as a senior DevOps engineer for IBM/Watson. I was responsible for one of the huge real-time data ingestion pipelines that Watson receives. I left to work elsewhere in spite of being offered an excellent position. (If you guys are reading this, hi.) I went to IBM not expecting much more than working as a cog in a lumbering giant. Watson is the fastest growing part of IBM. If IBM has all of those eggs in one basket, it is the Watson basket. There were lots of jokes about cognitive in the office pool. That said, it was by far one of the best managed companies I've seen. They have some fantastic data engineers and scientists. They are backing most of the open source projects related to AI and next generation tech. Spark, VoltDB... The ads might seem sensational, but the concept of a black box that orders preemptive maintenance for an elevator isn't far fetched... More over, Watson had so many current customers because it is valuable. The technical advisors that but products don't put faith in ads any more than we do.
megamindbrian 9y ago
Everything to do with IBM is too expensive for the average user.
JimsWonderWorld 9y ago
The IBM Watson win on jeopardy was completely made up. IBM does not have an AI that can answer natural language questions now and they also did not have that 7 years ago. IBM'ers are delusional.

503 points by artsandsci 9y ago

the articles content is more just background about watson than having to do with the the title. as a side note: it just sounds wrong to have the words "hating on" in an article title.

throwaway_ibm 9y ago

I know someone who is intimately involved with IBM Watson, they are highly educated and constantly diss the system. Calling it, 'Just a large database'. If Watson was a true breakthrough, it should be gaining marketshare throughout it's specialities but it's not. Google is leading the industry with DeepMind; Facebook and Microsoft aren't far behind. I'd encourage others to be very skeptical of the PR that IBM is pushing about their Watson problem. disclosure: I haven't read the article but wanted to share a related story.

peteretep 9y ago

A couple of years ago I was given a project that was essentially "Evaluate Watson APIs to see if there's anything there we could make use of", and came away with the distinct impression that it was largely smoke and mirrors, and there was very little that was either effective or interesting there.

drvdevd 9y ago
Could you elaborate a bit? I'm not affiliated with IBM or anything, just curious what you were looking for and what was missing.
- dr_zoidberg 9y ago
  I'll give you my experience. Manager asked me if we could use the Watson Speech-to-text API a couple of years ago, because he was impressed with the ads he'd seen. I worked for a week. Turns out the "dialect" of spanish that is spoken in Argentina (and this was tested against porteños, not even the accents from the provinces!) is utterly unsupported. Even when speaking slowly, it'd throw a random set of words with even more random looking probabilities. Manager was expecting that after a week we'd have something akin to Googles "hello google!" at hand, through Watson. It took a couple of coworkers to come up with even more impressive broken examples to finally convince him that the technology was bad.
  - _jal 9y ago
    An interesting example of negative-sum advertising. Everyone involved lost money. I wonder how often that happens.
    - drvdevd 9y ago
      I hadn't even considered that point of view ... quite interesting. EDIT: I have to say, I bet this happens quite often in fact, especially with software.
  - drvdevd 9y ago
    Interesting! Thanks for the concrete example. It's bad enough trying to convince managers as to the specifics of why some product doesn't work for your project.. even worse when said product has great marketing.
maxxxxx 9y ago
Our IT department has been talking about Watson for a while but I never could get anything out of them what it's really supposed to do. From what I can tell it's a very slick marketing term with some mediocre tech behind it.
- gaius 9y ago
  It sounds like Autonomy. Or Palantir.
- dguaraglia 9y ago
  As far as I understand (and I have only passing knowledge on this, as we've been evaluating machine learning technologies for our startup) IBM Watson is just a set of APIs you can use to build your own solution, and the APIs themselves are not top-of-the-line (Google's are.) The other day I saw a TV ad for Watson for the first time: the ad showed a bunch of aeronautical engineers who instead of having to take all kinds of measurements of airflow around a hull or who knows what else (the TV was muted) now sat behind a single computer that would do the whole thing for them. It was such bullshit, I couldn't believe my eyes. Imagine Amazon promoting SES as this amazing tool that'd automatically send marketing emails and create a marketing funnel and drive user acquisition cost down by a factor of 100X, without you having to do anything. That's the level of vaporware they are selling.
  - RHSman2 9y ago
    Very few out of the box machine learning services work other than speech-to-text by the worlds greatest company in that field (google). It's not a case of roll your own but find companies that can help you deliver s solution to your specific problem. (That is my business) This is the absolute bullshit of Watson. What it could be is a series of very good datasets and then being able to build models (based on recipes) and deploy them. It's so unbelievably clunky though. And Bluemix? Urgh
chrisco255 9y ago
My team and I were able to build a pretty good chatbot with Watson two years ago. We used Google for speech-to-text, however, as we found IBM's to be behind in that area. We had to use a Node.js proxy with a couple libs to parse out some natural language features (like time)...but overall it worked pretty well. At the time, it was the only open system I knew of to write things like this. Today, of course, there's the Alexa API from Amazon and Microsoft's LUIS, which are probably vastly superior for this sort of thing.
iampims 9y ago
Their text to speech API is pretty decent and was the cheapest available until Amazon Polly.

ams6110 9y ago

What? A brand name which is just a word meaning "IBM Enterprise Products and Services" doesn't really live up to the marketing hype? I can't imagine such a thing.

batmansmk 9y ago

You can try by yourself. https://alchemy-language-demo.mybluemix.net/ https://alchemy-language-demo.mybluemix.net/ Imagine analyzing product reviews to determine if it was positive or negative. Type "I like it", and see the inaccurate targeted sentiment (neutral sentiment instead of positive).

guyfawkes303 9y ago
I tried this, and while 'targeted sentiment' may be neutral, 'document emotion' has Joy at 0.880435 vs the rest at 0. If you change your example from 'I like it' to 'I like puppies', the 'targeted sentiment' changes from neutral to 'puppies positive 0.550431'. While the first example you found does seem odd, the rest seems to work quite well. Maybe it just has problems attaching 'it' to a 'target'.
- foldr 9y ago
  It has trouble with negation outside the relevant clause. So it is not fooled by e.g. "I don't hate it" vs. "I don't like it", but the first of the following two sentences is rated more negative than the second: I don't seem to hate it. I don't seem to like it. Edit: In fact it can even be fooled just by increasing the linear distance between a positive term and negation. The following is rated +0.6! I don't in any way shape or form like it.
tensor 9y ago
Google has a similar API: https://cloud.google.com/natural-language/ https://cloud.google.com/natural-language/
- oceanswave 9y ago
  Microsoft too: https://azure.microsoft.com/en-us/services/cognitive-services/text-analytics/ https://azure.microsoft.com/en-us/services/cognitive-service...
te_chris 9y ago
Alchemy was an acquisition.

dboreham 9y ago

Bundle up for the second AI Winter...

sjg007 9y ago
I think we have a good 10 years of engineering production systems based on current tech.
bitL 9y ago
Third actually. But it's just Watson; Deep Learning and its derivations are here to stay as they are already state-of-art and close to humans.
- z3phyr 9y ago
  They are not close to humans at all. They are good for very specific tasks only. Edit: Spelling
  - bitL 9y ago
    You are right - for specific tasks. E.g. in computer vision they are able to surpass humans for static image recognition. For image segmentation they are getting close already. For generating art they IMO have surpassed some "artists" already ;-) If you think about it, even with 90% accuracy they are worth it as they could almost automate some cognitively-difficult tasks and for those instances that they can't do it properly, a human can intervene. So instead of 10 humans watching over something you end up with a single one. It would take some time until all we have in the research is in production and that will change a lot of things. After that there might be another AI winter (who knows if we can do general intelligence?), but with current pipeline it might take a while to reach plateau.
    - deleted 9y ago
      [deleted]
- tannhaeuser 9y ago
  What's the second AI winter then? I only know about the 1986-1990 era, where "AI" was used in a broad sense covering everything from Prolog, expert systems, LISP, heuristic search algorithms, fuzzy logic to ML, neural networks and their generalizations. I've still got great shiny magazines from that time, though.

dpflan 9y ago

I like how IBM does very elaborate marketing ploys to hype their wares: like Deep Blue competing against Kasparov and Watson competing against Jennings to showcase IBM's engineering prowess. But it does sell the idea pretty well I think, but perhaps the idea is too grand/far ahead of the present.

forgot-my-pw 9y ago
Deep blue was just a brute force machine. Perhaps Watson isn't much different, just slightly smarter.
- sgt101 9y ago
  Watson that won Jeopardy was a fantastic advance in information retrieval. Google introduced the knowledge graph a year later, but domain (jeopardy) specific watson was terrific. The sad thing is that they got captured by PR rather than sense; a focus on customer service and my bet is that they could have disrupted Oracle/Seibel out of 75% of accounts. IBM would be ridin' high! BTW. This is still on the table for IBM if they are prepared to invest 18mths and focus gung ho on Oracle. In 18mths it won't be as "other players" will have moved, like Sauron, to hit this market.
- zone411 9y ago
  Saying something like this does a big disservice to IBM's real accomplishment. Code a brute force chess engine, add some alpha-beta pruning, transposition table, or even some of your heuristics and let us know how it does. Just because it's not an adversarial RNN or whatever, it doesn't mean it's brute force.

tangue 9y ago

Crédit Mutuel (a french bank) has adopted Watson [0] and it's not encouraging : it was supposed to help answering emails, : they had to describe manually the concepts in emails and create topics in which looks a lot like decision-trees (and reminds me of this 1985 ad for Texas Instrument's Lisp AI https://www.scientificamerican.com/media/inline/blog/File/Dewdney_Mandelbrot.pdf https://www.scientificamerican.com/media/inline/blog/File/De... scroll to see the ad) Indeed the whole thing looks like a database with basic AI as a sales argument... [0 - in french] http://www.silicon.fr/credit-mutuel-non-ia-watson-magique-173889.html http://www.silicon.fr/credit-mutuel-non-ia-watson-magique-17...

deleted 9y ago
[deleted]
Spooky23 9y ago
Totally. I observed a support chatbot project that didn't quite meet up with the vision. Even based on the technical marketing claims the consultants were coming in with, it was a product less capable than the IBM Prolog-based expert systems that I built out 15-20 years ago to triage and correlate network events in a large WAN infrastructure.... and that was a product with a whole slew of major implementation and operational problems!
- fapjacks 9y ago
  Hey just wondering: Have you written about your exploits before? If so, where can I read about them? Sounds interesting.
  - seanp2k2 9y ago
    It's not just interesting, it's C-O-M-C-A-S-T-I-C!
  - Spooky23 9y ago
    I haven't other than on HN, but I've been lucky in that I've had the opportunity to participate in some awesome and not-awesome stuff. Maybe I will start!

moomin 9y ago

Completely off topic, but didn't IBM have a system called Watson in the 1990s that was used by the police? Try as I might I can't find a reference for it anywhere.

rjsw 9y ago
Maybe you are thinking of the product from Harlequin [1]. [1] https://en.wikipedia.org/wiki/Xanalys https://en.wikipedia.org/wiki/Xanalys
- moomin 9y ago
  I think you're right... thanks.
sgt101 9y ago
Scotland Yard had a system called Holmes https://en.wikipedia.org/wiki/HOLMES_2 https://en.wikipedia.org/wiki/HOLMES_2
Retra 9y ago
Watson is the name of the founder of IBM, so there's a pretty good chance his name was attached to various things over the years.

currymj 9y ago

it's just a brand name at this point, which they attach to any machine learning they develop or acquire, and they should stop trying to sell it as a distinct technology.

strict9 9y ago

Many years ago when I worked for a company that decided our existing ecommerce app was too terrible to fix and would be too much effort to rebuild, we talked to a number of vendors, including IBM. The marketing materials and salespeople made a compelling case, but deeper dives into the app itself and the support engineers behind it convinced even the most enthusiastic internal cheerleaders to look elsewhere. In recent years as news articles heralding the future of Watson for various industries (including healthcare and supply chain), I predicted a similar path. An amazing product in a very narrow environment designed specifically for marketing and selling purposes, and not very adaptable. FTA: “And everybody’s very happy to claim to work with Watson,” Perlich said. “So I think right now Watson is monetizing primarily on the brand perception.” This is painfully obvious, as this has been IBM for a very long time.

iamleppert 9y ago

The real crime is in using cancer kids to sell your product. I mean, who even does that? Even if you could cure cancer for kids, I find it incredibly tacky to go around making commercials about how you can cure cancer for kids, which aren't targeted at those who actually are in the position to use the technology, and its used to market to other tangential industries where the real money is. It's just despicable and you can tell right there its smoke and mirrors. There's a special place in hell for anyone working at IBM or involved in the Watson project who is supporting this thing. It's damaging legitimate deep learning/machine learning industry and generally making a fool out of IBM, AND giving children with cancer false hope....just so IBM can try and stay relevant and make money?

orf 9y ago
> The real crime is in using cancer kids to sell your product. I mean, who even does that?? Charities?
- oblio 9y ago
  Ah, yes, the noblest of giants, the most charitable of charities, Big Blue.
- ryanmarsh 9y ago
  Can confirm Source: former president of children's cancer research foundation In my defense we were actually trying to make a difference in the world not sell software.
  - radarsat1 9y ago
    > we were actually trying to make a difference in the world not sell software. I mean, to be fair, I don't think selling software and trying to make a difference in the world are at odds with each other.
davidreiss 9y ago
Kids are used by scams/charities to lure in people for all kinds of things. The latest one is the cars for kids scam which used kids to get donations and who knows what they did with it. http://www.nbcnewyork.com/investigations/Kars-for-Kids-Cars-4-Kids-Real-Estate-Investments-Investigation-140955003.html http://www.nbcnewyork.com/investigations/Kars-for-Kids-Cars-...

ceedan 9y ago

Does IBM itself even "use" Watson?

marvel_boy 9y ago
Of course not. Watson is just a marketing trick to charge for consultants. I worked there.
- ceedan 9y ago
  This has always been my assumption. If Watson did anything close to how it's advertised, IBM would have a big competitive advantage by actually using it.

outside1234 9y ago

You don't say! This is IBM consulting ware? Who would have guessed!

notfromhere 9y ago

The dirty secret is that IBM Watson is just a brand for their army of data consultants, and their consultants aren't very good. In my experience working for a competitor in this space, IBM Watson was widely agreed to be smoke and mirrors without much going on

ibm_throwaway 9y ago
Former IBM Watson data scientist here. At first I was annoyed by your comment, but then I realized how true it is and sulked instead. The good consultants are leaving as soon as they hit the end of their first promotion cycle. The duds stick around and assume the upper-level manager roles. The group is in bad shape now and poised to get worse in the coming years.
- ethbro 9y ago
  As someone who's worked in that space, I'll repost the short version of a comment I made before: it's an economics problem. Services Profit = Bid - Employee Cost + Upsales Bids are only so flexible. Employee cost is much easier to minimize. So inevitably, all consultancies trend towards providing the cheapest employees that will satisfy the customer. And if the customer isn't constantly vigilant, the quality can get pretty dire indeed (BAs who just learned to "code" yesterday).
  - gaius 9y ago
    Funny you should mention BA, since they have had a series of high profile IT meltdowns since outsourcing...
    - ethbro 9y ago
      Sorry, in my use *Business Analyst (aka business majors who want to get into "that computer thing") BPO / offshoring is a whole other can of rotten worms.
      - 9y ago

speeder 9y ago

I actually love the idea of Watson being used for healthcare... Sadly I think it is being used wrong... IBM is focusing on using Watson to cure very specific diseases, like certain types of cancer. I think a far better use for Watson would be to do initial diagnosis, for example my life got massively delayed because I got hypothyroidism as teenager, but only using internet data I could self-diagnose and self-treat (because doctors are still unwilling to help, not trusting data, and before someone come berate me for self-treatment, it is working...) as adult I could finally get my life 'started' (hypothyroidism affect physical and mental development, and slows down metabolism and the brain) During my quest I met many, many, many people on internet, that had self-diagnosed with something using the internet as a tool. All of us would have been diagnosed properly if Watson was being used on the doctors office, using its data crunching capabilities and symptoms as input to find out what problem we had. (in my case: I have Hashimoto's disease)

vslira 9y ago
Why would professionals support tech that makes their labor less scarce? Unfortunately this change won't come from doctors
- pwthornton 9y ago
  Doctors might fear it, but healthcare administrators would certainly love for a tool to help make diagnosis faster, more accurate and less expensive.
  - sgt101 9y ago
    If it worked in that way, which it currently doesn't (see comments about question answering systems and Jeopardy) in terms of the state of the art, we're... maybe 30 years off that, like as in we're 30 years off fusion. Think about the human condition, think about human biology. Be clear- these are systems beyond current science... and doctors deal with them every day!
    - roywiggins 9y ago

slackingoff2017 9y ago

IBM is a dying giant, I've seen it languishing for years. Their massive screw up was a decade ago when they decided shareholder value was more important than having good engineers. They've since gutted their R&D departments and all that's left are duds and underpaid undereducated consultants rented from places like Accenture. The only good thing to come out of IBM in years is their Hyperscan regex library and unsurprisingly they don't market it at all or build practical applications with it

bsg75 9y ago
The same Hyperscan as here? https://github.com/01org/hyperscan https://github.com/01org/hyperscan Project homepage suggests this is an Intel project.
- thealfreds 9y ago
  Also the LICENSE > Hyperscan is licensed under the BSD License. > Copyright (c) 2015, Intel Corporation
KingMob 9y ago
The Hyperscan website (https://01.org/hyperscan https://01.org/hyperscan) says it was built by Sensory Networks Inc, and then bought by and released as open source by Intel. No mention of IBM. Are you thinking of something else?
- slackingoff2017 9y ago
  Oops, yeah. They haven't produced anything then :)
  - p1esk 9y ago
    Jeopardy, TrueNorth?
WalterBright 9y ago
In the 80's IBM was an unstoppable juggernaut. In the 90's it was Microsoft. Then Apple. Now Google/Amazon. Conventional wisdom is that, in a free market, a single corporation will grow to rule everything. But that never seems to pan out. Part of the reason is that the larger an organization gets, the more inefficient and bureaucratic it gets, the less it is able to adapt. See "The Innovator's Dilemma", a book everyone interested in business should read.

potatoman2 9y ago

Did they really need to stick an insult at Trump in there?

optimuspaul 9y ago
yes
colejohnson66 9y ago
As much as I hate bringing politics into places it wasn't, this wasn't from the author; It was from a quote > "IBM Watson is the Donald Trump of the AI industry—outlandish claims > that aren’t backed by credible data," said Oren Etzioni, CEO of the > Allen Institute for AI and former computer science professor. > "Everyone—journalists included—know[s] that the emperor has no clothes, > but most are reluctant to say so."
deleted 9y ago
[deleted]

etiam 9y ago

It's tempting to start whispering winter is coming, but I think one may reasonably hope that the current fashions at large have enough nuance to differentiate between this particular marketing gimmick and the broader developments in ML. Personally I'd be happy to see the paragraphs/minutes at the beginning of far too many interviews about "intelligent" machines exchanged, from straightening out the misconception that Watson is an example of this new hot "Deep Learning" thing and one of the pinnacles of achievement in the field, for some type of more valuable type of commentary from leading researchers.

PaulHoule 9y ago

This was my opinion when they started running these ads. My opinion has actually softened a little. Some of the cognitive services they are offering today are not half bad; also I can say their salespeople are doing a gangbusters job in places.

crsv 9y ago

Replace IBM's Watson with anything branded with "AI" right now and themes in the article still hold up.

bsg75 9y ago
It is likely that AI has not matured to the level its being marketed at currently? And that the over hyping of it today will negatively impact its adoption when ready for general use?
- macintux 9y ago
  Seems like AI has been over-hyped for decades, just rarely with someone as prominent as IBM leading the charge.
- scottlocklin 9y ago
  It happened in the 80s. https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter It will happen again. The use of the phrase "artificial intelligence" itself ought to be forbidden until such a thing actually exists in the corporeal world.
  - Barrin92 9y ago
    >It will happen again maybe an 'AI recession' is plausible but a large scale loss of interest is unlikely given that AI / ML / deep learning (pick your favourite word) is already too embedded in commercial application. Like always the hype in the field is strong, but this time we actually have the data growth we need to produce palpable results (and already have).

riku_iki 9y ago

Any first firings for choosing IBM?..

deleted 9y ago

[deleted]

et2o 9y ago

I saw a very humorous twitter exchange between a bioinformatician and IBM Watson's twitter account. The scientist asked them to provide any peer-reviewed ML publications and the best they could do was an abstract at a regional conference no-one has heard of. And it was a terrible abstract. It's completely marketing. IBM still has a good name among people who don't know much about technology. They're trading on this and the current saturation of 'machine learning' in the popular press.

sgt101 9y ago
That's mad as IBM have loads of peer reviewed ML publications. 3 minutes of searching : http://researcher.watson.ibm.com/researcher/view_group_pubs.php?grp=5443 http://researcher.watson.ibm.com/researcher/view_group_pubs.... http://researcher.watson.ibm.com/researcher/view_group_pubs.php?grp=4384 http://researcher.watson.ibm.com/researcher/view_group_pubs.... c'mon
- et2o 9y ago
  I should have added: healthcare. No shit IBM research has publications, they have their own journal. Also, 15 publications in 3 years is pathetic. c'mon
  - sgt101 9y ago
    Well, the operational people will struggle to publish because it won't be on their scorecard, so effectively they won't be paid for it. Additionally many of them won't have any skills relevant to creating an academic publication and no nice professor to help them do it. So if IBM are going to publish it will be their lab. In terms of publication rates, in universities it's a mill. All academics are measured on publication impact and because of the arbitrary and self reinforcing nature of impact the way to win is to sprinkle as many papers around as you can get your graduate student factory as you possibly can. The question is what is the actual utility of minting out the same paper with nine different authors?

blueyes 9y ago

IBM has almost zero credibility in deep learning and AI. They haven't hired anyone of note. They haven't produced any novel or influential research in the field in years. And yet they air these cheesy Dylan ads and the rubes fall for it. Watson is a Theranos-scale fraud, and it's finally coming out.

pas 9y ago
Was the Jeopardy think just a ruse? Was it something easy (something any ML/DL team could have thrown together in a year?)
- martingoodson 9y ago
  No it was a hard engineering challenge. But it solved a very specific problem that has no relevance to most business use-cases.
  - ethbro 9y ago
    Also, at effectively unlimited cost. Are there any numbers how much they spent on making it work?
    - sgt101 9y ago
      30 people (ish) for 3 years + 10 million on hardware? The brilliance of the team... priceless. The cost of everything, the value of nothing : this is what will be written on the tombstone of capitalism.
Jach 9y ago
http://jeffjonas.typepad.com/about.html http://jeffjonas.typepad.com/about.html However he's no longer at IBM...

tCfD 9y ago

Obvious fix is for IBM to put Watson on a blockchain /s

samfriedman 9y ago
You jest, but they're not above "throw a blockchain on it" levels of buzzword-chasing: https://www.ibm.com/us-en/marketplace/z14 https://www.ibm.com/us-en/marketplace/z14

diego_moita 9y ago

> "In the data-science community the sense is that whatever Watson can do, you can probably get as freeware somewhere, or possibly build yourself with your own knowledge" Any suggestions about the freeware?

sgt101 9y ago
MLlib Tensorflow Lime There you go.
- nl 9y ago
  MLib The Watson of machine learning libraries. As someone who has used MLib and raised a number of bugs and feature requests: Use Spark for feature engineering and then Scikit for ML.

simonh 9y ago

This just goes to show just how tragically far away we are from even beginning to build the rudiments of a strong general purpose AI. For all the fantastic achievements of systems like Watson and Alphago, and they are amazing achievements, they are radically optimised special purpose systems fine tuned to solving one extremely specific and narrow problem, and that problem only. Watson is a case study in this, but I know Google has big plans for applying the tech behind Alphago in medicine. I wish them every success, but I'm concerned they will hit similar specialisation issues.

FeepingCreature 9y ago
I would argue this only shows how tragically far away IBM is.
sgt101 9y ago
Medicine - the biggest prize, the biggest trap. Every complication, huge, mind blowing rewards. Maybe, if Google were to go 100% they could break the beast... but it's a 100bn bet.
- simonh 9y ago
  Untill you actually have a viable, testable plan to implement it almost allege money you throw at it is pretty much certain to be wasted. The Manhattan project is often given as an example, but that project was based on solid theoretical models and repeatable experiments. They knew exactly what they were doing and how they were going to do it. The problem in AI at the moment is mainly developing talent and expertise, not capital funding. I'm all for advanced research into technologies. However we need to advance that research on as many different fronts as possible in a flexible way simply because we don't know how we are going to solve the problem of general AI. Guessing on an approach and spending billions on it would be a complete waste. We're simply not at a point where that would do any good.
  - sgt101 9y ago
    Agree, they should have focused on much more tractable domains before Medicine. I think that there's an business case issue here; MBA's see this as the mega win, the research staff can't say "it won't work" (because you are immediately flagged as an obstacle to progress and the execs either cease to listen because they don't want to hear, or you get bullied with an avalanche of attacks on your team or sacked).

ChuckMcM 9y ago

When I worked at IBM I expressed concern that the television commercials depicting a HAL9000 level interactive dialog system were dangerously overselling what Watson could do. The challenge, as I saw it, was that no matter how good the tools and products that were used to help companies with data analysis to improve their operations were, when they realize they can't talk to a cube and joke with it about misusing colloquial phrases their disappointment overshadows all the 'good' stuff it was doing for them. No relationship works well if it starts with a lie and as this article shows, people do take those ads at face value and assume there really is a talking AI inside of IBM. Then they are hugely disappointed when they find out it doesn't exist.

ethbro 9y ago
Over-promising has killed AI progress before (expert systems). So history's just repeating itself. People want to believe. They'll pay for their belief. Then when projects start to fail (~5 years after commercialization) they form the opinion that it was never going to work and they'll never do it again. ... then we wait another decade, repeat.
- olewhalehunter 9y ago
  There's this trend of Expert Systems being commented on as a failure and starting some sort of AI winter, yet to my knowledge they were among the first commercially viable digital animation tools, the first high speed trading platforms, and are still being used successfully in military planning. Where's the evidence that they were a failure or that they were oversold?
  - sgt101 9y ago
    Well, there was this place called "The Turing Institute" which was in Glasgow. It was run by a very nice man called Don Michie. I met Don a handful of times, and he was extremely kind and generous to me. He worked at Bletchley Park and died aged 91 having hit a bridge at 120mph going the wrong way up the M11 at 2am sitting next to his wife who had defected to Russia thirty years before. Don was driving, at the time he had been blind for six months. Anyway, The Turing Institute existed because of Don's drive to commercialise Expert Systems. Don was a driver in the Alvey project which was a grand project to solve issues around Expert Systems in response to the Japanese Fifth Generation project. A man called Lighthill then rumbled the Expert System movement. If you want to watch a demolition of it see the following video : https://www.youtube.com/watch?v=yReDbeY7ZMU https://www.youtube.com/watch?v=yReDbeY7ZMU and the other 5 of them (you will find them in the sidebar!) Fun fact. In video 6 you can watch Christopher Strachey have a pop at AI. Christopher didn't work at Bletchley, he was too young, but he did work at Manchester with Alan Turing and they created computer music and love poetry in collaboration before Turing killed himself. Strachey went on to be the first Professor of Computer Science at Oxford, he was a scion of a great family of artists and poets. But enough of this. The Turing Institute was an attempt to say "we are right, look", but in fact it failed. Don was clear why, when you hit 40k rules knowledge bases became completely intractable. It was impossible to make improvements or changes. This was christened the Knowledge Acquisition bottleneck, and Don and others turned to Machine Learning (he liked to call it Behavioral Cloning) to solve it. So : 1. It was done to death by peer review. 2. It died commercially. 3. It's main protagonists decided to do something diffferent.

deleted 9y ago

[deleted]

d--b 9y ago

watson's mistake is to have gone the chat bot route. promising a natural language input for all underlying problems simply discredits everything else...

scottlocklin 9y ago

Yeah, well, "duh." What boggles my mind is people will read this, nod sadly, and continue not to notice that a whole bunch of what they think they know about machine learning, autonomous vehicles and so on is also marketing department hype.

mattnewton 9y ago
I don't know if that's fair. To take self driving cars as an example, I've seen cars driving themselves around mountain View and Waymo isn't promising much more than that. Watson had ads with an intelligent voice but they were selling a bunch of programmers to make expert systems like it was the 80's but in java this time. I think by comparison those other fields are promising less and delivering more. Even if there is still a gap between what people think will happen in the next 10 years and what we can likely deliver in the next 10, it's nowhere like what is described in the article.
- scottlocklin 9y ago
  C'mon man! Google allows one to check on their promises and their progress. For example, from 2012: http://www.computerworld.com/article/2491635/vertical-it/self-driving-cars-a-reality-for--ordinary-people--within-5-years--says-google-s-sergey-b.html http://www.computerworld.com/article/2491635/vertical-it/sel... "Such "autonomous vehicles" will be a reality for "ordinary people" in less than five years, Google co-founder Sergey Brin said Tuesday." How's that working out for you Sergey? Self driving tech hasn't moved appreciably since the 90s. Computers are better. More people are trying it. Reality is, it's just collision avoidance with GPS. They rely on maps; what happens when the maps change? What happens when a kid runs out in front of one of these things?
  - mattnewton 9y ago
    Sergey isn't taking people's money based on that promise. As an engineer I don't beat people up about realizing how hard problems are and not releasing crappy products. Missing estimates happens. It's another thing entirely to try and pass off a product as something it isn't to a buyer.

Probooks 9y ago

Problem is deeper (and simpler). IBM does not look for clients, but rather victims. We clients end up being caught in an internal upsales fight. Nobody cares which is the best solution IBM as a whole can offer to you (their own people do not even know all their available tools!), but rather how much suboptimal stuff each salesman can load onto you. I'm on my way out of IBM...

dislikes_IBM 9y ago

IBM has a toxic culture. They are the vendor lock-in Gods. Every company I've ever worked for has cringed at the mention of IBM, never suggested them as a new solution, and always regretted whatever if anything they locked themselves into. They are the only company that charges you to sample their API's. They are the absolute worst, an infection that needs to be cured.

johngarrison 9y ago
Sometimes I wish I was wealthy enough to buy out horrible companies, just so I could have the power to dismantle them and end the suffering.
- Method-X 9y ago
  You wouldn't be wealthy for very long ;)
smugworth 9y ago
I'm working on a team for a platform built on open-source software which actively replaced a previous IBM solution, and there are more services from other vendors being phased out in favor of internally developed tooling. Seems like a good long-term investment, in an era where people are constantly criticizing CEOs for short-term cash-grabs. Why shackle yourself to a vendor if you plan to be around for awhile.

ghostly_s 9y ago

I overheard a good-'ol-boy businessman at a hotel bar a few months back. He bore an eerie likeness to Bosworth from Halt and Catch Fire, and was telling a younger gentleman about a project he worked on. "...so Watson comes in and they Algorithm the whole thing..." I'm pretty sure he thought Watson was a person.

ExactoKnight 9y ago

Watson's Natural Language Classifier, in particular its categorization API, is actually pretty impressive...

jjm 9y ago

They had so much time to contribute but instead chose marketing and pushed into areas where they didn't really have a handle on yet. As in management didn't understand. I mean all the datasets, dozens of libraries, stunning NN demos and training sets, TPUs (multiple versions at that!) all could've come out of the company. Think if keras and tensor flow were from IBM. Or all those cars now running Nvidia Jetson, or mega datacenters running NV100s or Google TPUs. Shoot they even had a chance to enhance PowerPC ICs for NNs. Alas but nope.

filereaper 9y ago

I'm quite late to this thread, but I worked on Watson very briefly (not on the core development, but overall system performance improvements). I think there's a major misunderstanding of Watson which isn't helped by IBM's Marketing efforts. IBM Marketing has been slapping the "Cognitive" label on everything and is creating unrealistic expectations. The Jeopardy playing Watson (DeepQA pipeline) was a landmark success at Information Retrieval, its architecture is built largely on Apache UIMA and Lucene with proprietary code for scaling out (performance) and filtering & ranking. I'm not an expert on IR so I won't comment further. This is very different from Neural Nets that are all the rage in ML today. I'd like to point the following links from David Ferrucci [1] the original architect of Watson and this technical publication at aaai.org [2]. The DeepQA pipeline wasn't fluff, the intention was to take this question-answer pipeline and apply it to other verticals such as Law and Medicine, essentially replace the Jeopardy playing Watson's corpus of Wikipedia, Britannica etc... with Legal and Medical equivalents. Given its runaway PR success, the Watson brand was applied to many other areas which haven't been successful but I'd like to point out what the original product was here. [1] https://www.singularityweblog.com/david-ferrucci-on-singularity-1-on-1-pursue-the-big-challenges/ https://www.singularityweblog.com/david-ferrucci-on-singular... [2] https://www.aaai.org/Magazine/Watson/watson.php https://www.aaai.org/Magazine/Watson/watson.php

nl 9y ago
This is true. I've read all the Watson Special Edition papers numerous times[1]. The original Watson work was very good, and adding neural-network based models as alternative question parsers and answer rankers is a clear way forward. It's a pity IBM decided to use the Watson name for everthing they do in ML now. [1] http://brenocon.com/watson_special_issue/ http://brenocon.com/watson_special_issue/
denzil_correa 9y ago
> The Jeopardy playing Watson (DeepQA pipeline) was a landmark success at Information Retrieval, its architecture is built largely on Apache UIMA and Lucene with proprietary code for scaling out (performance) and filtering & ranking. I'm not an expert on IR so I won't comment further. This is very different from Neural Nets that are all the rage in ML today. The question I would like to ask is that if the architecture is built on UIMA and Lucence, what exactly did "Watson" do?

laichzeit0 9y ago

I had problems with Watson to the effect that not even the documentation matches reality. There are some fairly basic things missing from their NERC offering. I can tell you that the functionality that is missing is so basic (e.g. negation) that without a doubt, no one in IBM has ever used this offering in practise beyond a toy example. The idea that IBM Watson is some uniform AI in a box with a bunch of REST API's to "expose" its intelligence seems to be the sales pitch. It's not. It's just a bunch of acquired products (you can see this when e.g. Watson Knowledge Studio breaks and you see the Python scripts that glues everything together in the backend) that are poorly integrated, probably because the left hand has no idea what the right hand is doing. Caveat emptor! https://stackoverflow.com/questions/44796501/ibm-watson-knowledge-studio-entities-with-role-attribute-and-extracting-it-fro https://stackoverflow.com/questions/44796501/ibm-watson-know... https://stackoverflow.com/questions/44800879/ibm-watson-knowledge-studio-annotating-negative-negated-mentions https://stackoverflow.com/questions/44800879/ibm-watson-know...

chisleu 9y ago

I'm late but have something to add. Until last week I was on a 6 month contract as a senior DevOps engineer for IBM/Watson. I was responsible for one of the huge real-time data ingestion pipelines that Watson receives. I left to work elsewhere in spite of being offered an excellent position. (If you guys are reading this, hi.) I went to IBM not expecting much more than working as a cog in a lumbering giant. Watson is the fastest growing part of IBM. If IBM has all of those eggs in one basket, it is the Watson basket. There were lots of jokes about cognitive in the office pool. That said, it was by far one of the best managed companies I've seen. They have some fantastic data engineers and scientists. They are backing most of the open source projects related to AI and next generation tech. Spark, VoltDB... The ads might seem sensational, but the concept of a black box that orders preemptive maintenance for an elevator isn't far fetched... More over, Watson had so many current customers because it is valuable. The technical advisors that but products don't put faith in ads any more than we do.

megamindbrian 9y ago

Everything to do with IBM is too expensive for the average user.

JimsWonderWorld 9y ago

The IBM Watson win on jeopardy was completely made up. IBM does not have an AI that can answer natural language questions now and they also did not have that 7 years ago. IBM'ers are delusional.