9 ms·

"Offering managers didn’t have technical backgrounds and sometimes came up with ideas for new products that were simply impossible." Sounds like they drank the

by glup 8y ago

"Offering managers didn’t have technical backgrounds and sometimes came up with ideas for new products that were simply impossible." Sounds like they drank their own kool-aid, e.g., "Products That Enhance and Amplify Human Expertise," rather than understand the actual limitations and possibilities of ML. And it seems to me that they're still doing it with this nonsense about a human-level "AI" debating stack. The oversell seems a real shame in light of how much good can be done with EMR and machine learning / NLP.

pm90 8y ago
That is my biggest gripe with this as well. We certainly don't want an another AI winter, especially not in health, where I'm hoping (perhaps too optimistically) that it will allow better and cheaper care.
- ska 8y ago
  that it will allow better and cheaper care. That will happen, but you have to think 20 year plans, not 5 year plans.
krona 8y ago
That's surely part of the problem, but the catalyst is the marketing strategy that is used to brainwash the employees. In essence; sell the experience, not the product. This works well for IBM generally (the products are shit) but especially well for Watson because it's extremely easy to sell AI without getting bogged down in details. You want to identify brain tumors? We'll just teach Watson to do it. Whilst IBM research might be able to pull it off, it'll never get to market because there is nobody capable of making good products at IBM anymore.
- appstateguy 8y ago
  > Whilst IBM research might be able to pull it off, it'll never get to market because there is nobody capable of making good products at IBM anymore. As an ex-IBMer this is so true and so frustrating at the same time. Engineers are thrashed about on a nearly sprintly basis by PM's with short attention spans and no understanding of how disruptive their continuously changing requirements are. It doesn't help that IBM consistently puts the cart before the horse is even born and pivots multiple teams all at the same time such that nothing you build upon is stable or consistent. Working there was maddening.
  - haskellandchill 8y ago
    I think this is just what large multinationals do these days.
  - workinthehead 8y ago
    Sounds like the way non-tech companies operate. Pretty damning that IBM can't understand why leadership of a tech company should have a tech background.
- cirgue 8y ago
  > This works well for IBM generally (the products are shit) but especially well for Watson because it's extremely easy to sell AI without getting bogged down in details. The cynic in me says that every use of the term AI in any capacity is to sell experience and not functionality. When was the last time you used a product billed as 'AI' and thought 'wow, this is a huge game changer'? Siri is cool, but it's ultimately not super useful. Google translate is incredible, but it can only do what it can do because of the absolutely mind-boggling amount of training data that google can access. Most disciplines have the problem of not enough data, despite what 'big-data' folks say. In contrast, humans can extrapolate and make reliable predictions about the future based on really small sample sizes. We can pick up a new skill or recognize a new pattern with a high degree of accuracy really effing fast compared to a computer. This gives humans an enormous advantage. If IBM and anyone else in this space were really focused on delivering excellent real-world results, step 0 is building out world-class data integration and search tools (which we still actually suck at, weirdly.)
  - NicoJuicy 8y ago
    Many people mention even a worser search experience with Google and AI
    - davidf18 8y ago
      Could you please elaborate? Does that include DeepMind?
  - mlthoughts2018 8y ago
    In fact, I would say apart from maybe self-driving cars, almost all of the biggest gains from machine learning are in unsexy, hidden backend problems, like automatically rectifying disparate data, optimizing resource utilization, flagging difficult-to-articulate events or triggers in a stream of data too large for human evaluation, machine translation, and other “unsexy” things. Product interfaces usually offer simple features to users and the value proposition is easy to see. Effective use of machine learning is well hidden upstream in a bunch of unsexy preprocessing or heavy lifting to get to the interface. Not something you’d ever need to emphasize in marketing, except maybe at tech meetups or in recruiting materials, but not to the end consumer. It just makes pop references to AI-powered products more egregious.
- ThomPete 8y ago
  Agree. Consultancies rarely sell products or solutions. Instead they sell project management by making it sound like you are a more safe bet than a smaller product studio who actually do make products work. Its a real shame but i mostly blame the zero mistake KPI culture primarily fueled by how managers on the client side are promoted.
- mathattack 8y ago
  Over the past 10 years I’ve been surprised when anyone smart would join IBM. I understand why people hung on, but why join that dying ship? Now I can’t think of any still there.
  - occamrazor 8y ago
    Who says IBM is dying? They aren’t anymore (since a long time) at the bleeding edge of research, but they still have a solid consulting and integration business. They also offer decent salaries and good opportunities for sales-oriented technical people. It wouldn’t be my first choice of employer, however I can see how some people would enjoy working there.
    - mathattack 8y ago
      Dying may be extreme. Perhaps it’s safer to say that they are in the mode of returning capital to investors rather than growing. I’m at a large customer of theirs and they are bleeding the customer for every dollar as they get phased out. Very low caliber of services professionals too.
stefan_ 8y ago
It's double whammy for the Watson stuff, because not only did IBM lose track with AI, they also had that whole cloud thing whoosh by them. So not only do you have marketing telling outrageous lies about the abilities of their AI systems, they're also exaggerating the cloud angle where Watson is a cornerstone.
- bunderbunder 8y ago
  Possibly the most outrageous lie was calling it AI in the first place. Watson is a $@#% amazing information retrieval system. Information retrieval is only a small part of what people think of when they think, "AI".
  - goatlover 8y ago
    > amazing information retrieval system Which I imagine still has lots of value on it's own for large, complicated data sets.
    - bunderbunder 8y ago
      Yep. But if your marketing and product management are all focused on selling AI instead of IR, then they're not really working toward finding a way to deliver the IR value they have to the people who need it. I'd actually like to give Watson a spin for an IR problem I'm looking at, but, thanks to their hype machine being set to overdrive, they've got the thing priced in the "The Bold Leaders of the Future Creating a Bright New Tomorrow Full of People in Glasses Staring Wistfully Toward the Right Edge of the Photograph, While Blue Curvy Streaks Wave Through the Background and Random Zeroes and Ones Float Around Their Heads" tier. Sadly, I've only got a "businesses solving business problems" sized budget.
      - solarkraft 8y ago
        What is your use case and what are the alternatives you're considering? I'm trying to understand what to imagine here.
      - 8y ago
alienreborn 8y ago
Whats EMR? (definitely not Elastic Map Reduce)
- a5huynh 8y ago
  Electronic Medical Record(s)
- deleted 8y ago
  [deleted]
andyidsinga 8y ago
There's a pretty good podcast interview with Eugene Dubossarsky that has relevant discussion about issues with management and data science in general. https://www.datafuturology.com/podcast/1 https://www.datafuturology.com/podcast/1 Here are a few of my notes (my words not the interviewee's): - in order to use data science, you have to have creative people thinking about data on the front end - they don't have to be data scientists, but they need to be creative and want data to support decisions and iteration via feedback loops - that creativity and desire will lead to "doing good data science" - management on the receiving end of data science output must be intelligent in terms of synthesizing many inputs and have a strong desire to puzzle through the implications. If management is asking the data science to actually make the decisions - the situation is broken - data science must be done with provisions for decision support and feedback loops; this is the output that is helping drive the business. - Lack of desire for decision support and feedback loops leads to "fancy pets" and management using data science as a means to brag about what they are doing; but the data science might not being doing anything to drive the business meaningfully. - data science that attempts to actually make decisions vs providing decision support is likely in the category of "commodity data science". Corollary : non-commodity data science is the kind that supports decisions in executing higher-level business strategy. Strategy at that level has rather unique attributes and is embedded in unique circumstances for a particular business. This requires a good data scientist to help tackle. (hope this is useful) (edit typos,grammar)
- andyidsinga 8y ago
  BTW - in listening to that podcast I found a lot of parallels with database design. whenever I'm asked to design a database for an early-stage system (I work in early stage tech ventures), I ask the following: - what are the questions that this database should answer for you? How are those questions supporting your business goals 3,6,12 months out? (I'm trying to get to the business requirements here) - who will be asking those questions (I'm trying to put together some user personas in my head) - how frequently will they be asking these questions? corollary: how often will historical data be needed? (I'm thinking hot vs cold and complexity of retrieval, minimally required performance) - how much data to we anticipate is needed to answer the questions (this is really tricky in new ventures - often the answer is more data than what will actually occur in practice in the first year)? - finally, what systems & tools are people using to ask the questions and be notified of events? (I'm thinking about interfacing, apis) its all an attempt to stay very focused on the questions and business drivers and the people who use the answers.
wnevets 8y ago
>"Offering managers didn’t have technical backgrounds and sometimes came up with ideas for new products that were simply impossible." how many of those managers will now be able to get even a higher paying job because they have manager of Watson AI project on their resume?
lifeisstillgood 8y ago
It has been ever thus with IBM adverts - this was me moaning online from early 2000's http://www.mikadosoftware.com/articles/ibmadverts http://www.mikadosoftware.com/articles/ibmadverts
- emmelaich 8y ago
  That's great! Thanks.
LrnByTeach 8y ago
following is a text snippet from the Article . I do not know which part of the following is AI , it is pure "data analysis" writing few SQL queries . This is the Biggest problem of IBM, they bill these kind of things as AI . > A clinic could use the system to search its patient records and find, for example, all the men over age 45 who were overdue for a colonoscopy, and then use an autocall to remind them to schedule the dreaded appointment.
- fjsolwmv 8y ago
  The point is it's NOT SQL. It's NLP and whatnot to extract value from relatively natural non-fully schematized data.
  - boxy310 8y ago
    DOB and date of last checkup are highly schematizable. This specific example is very well-suited to a SQL solution dumping into an autocaller queue. Maybe this was a terrible example, and the author didn't grasp a good example of legitimately non-schematic data points?
- cabaalis 8y ago
  Being directly involved in the execution of the example you gave, I believe ML would be unnecessary and error-prone. Even if successful, a system which could "interpret" a health record (such as a freetext note) using anything other than properly codified data would set the health industry back a decade. Moving doctors away from freetexting their notes is the only way to advance the industry.
  - skolman 8y ago
    This is something I've always wondered about- how would you do it? How do you break the cycle of freetext EHR notes? Doing so could have incredible utility for sharing data across various clinics/hospitals/pharmacies/etc.
    - sbarre 8y ago
duxup 8y ago
Other reports seemed to indicate that Watson (and maybe all AI) requires a lot of very careful, slow, and somewhat arduous data entry and testing to get good results. I can imagine someone who doesn't know at IBM selling a product: "Hey we will solve all these problems like magic!" Then IBM comes back: "Hey do you have all this data in a specific format and a ton of time to enter and test it and maybe we'll get back to you???" That's a big loss of trust there with the customer if you come back with that. It seems like these are products where a lot of caveats needs to be made clear to customers and a real careful technical partnership formed with them to succeed long term. You have to bring the customer along for the ride and exploration and keep them excited for a long time it sounds to make it work.
WhitneyLand 8y ago
Are the articles at ieee.org usually technically competent? To see one of their articles with the common press confusion of mixing different definitions and interpretations of AI (correct or incorrect) doesn’t help build confidence. For example, what was used to play Jeopardy vs. approaches being taken to improve cancer treatment, are just so different, it seems almost disingenuous to throw it all haphazardly into one conceptual bucket. The article does IBM a disservice in some ways. They come off looking bad overall but some of the failed projects mentioned like MD Anderson, failed for reasons beyond any control they had, other than recognizing some obvious red flags earlier and detaching their name and participation from it. On the other hand I believe the article lets them offf the hook to easily when they bring out the old trope they’ve been using for years, which is encapsulated here: “IBM Watson has great AI” [one engineer said] “It’s like having great shoes, but not knowing how to walk—they have to figure out how to use it.” It doesn’t make sense to say, xyz is great we just have to figure out how to use it, as a stand alone argument. It’s nonsensical unless you mention something about the seeming implied untapped potential, specific innovations, novel approach, or whatever makes it great. I’m not familiar with all their IP so maybe there are some great things, you just don’t get to claim that and get off the hook so many times in the press without providing at least some detail or reference point.
anonymous5133 8y ago
IMO, this is a bigger issue in the industry. People are hyping up ML/AI to the point where the actual application is either impossible or extremely difficult. Just look at how many people are so fearful of AI/ML taking away jobs and replacing humans. Anyone in the field knows that AI/ML can take away jobs but it is more of the low-end jobs and everything happens gradually rather than immediately. AI/ML is being more as a tool to enhance human productivity rather than as a direct replacement for entire occupations unless the job is very basic to begin with.
- xyhopguy 8y ago
  IMO AI/ML scientists are drastically underestimating the complexity of biology. Once scientists in biology and healthcare get on board like they are in linguistics and computer vision I'd expect things to pick up.
  - anonymous5133 8y ago
    agree
tlobes 8y ago
The article only mentions the engineers being laid off, not the managers who botched it, nor the executives who hired with incompetence. The reward structure of such a corporate environment would seem counter-intuitive to success.

by glup 8y ago

pm90 8y ago

That is my biggest gripe with this as well. We certainly don't want an another AI winter, especially not in health, where I'm hoping (perhaps too optimistically) that it will allow better and cheaper care.

ska 8y ago
that it will allow better and cheaper care. That will happen, but you have to think 20 year plans, not 5 year plans.

krona 8y ago

That's surely part of the problem, but the catalyst is the marketing strategy that is used to brainwash the employees. In essence; sell the experience, not the product. This works well for IBM generally (the products are shit) but especially well for Watson because it's extremely easy to sell AI without getting bogged down in details. You want to identify brain tumors? We'll just teach Watson to do it. Whilst IBM research might be able to pull it off, it'll never get to market because there is nobody capable of making good products at IBM anymore.

appstateguy 8y ago
> Whilst IBM research might be able to pull it off, it'll never get to market because there is nobody capable of making good products at IBM anymore. As an ex-IBMer this is so true and so frustrating at the same time. Engineers are thrashed about on a nearly sprintly basis by PM's with short attention spans and no understanding of how disruptive their continuously changing requirements are. It doesn't help that IBM consistently puts the cart before the horse is even born and pivots multiple teams all at the same time such that nothing you build upon is stable or consistent. Working there was maddening.
- haskellandchill 8y ago
  I think this is just what large multinationals do these days.
- workinthehead 8y ago
  Sounds like the way non-tech companies operate. Pretty damning that IBM can't understand why leadership of a tech company should have a tech background.
cirgue 8y ago
> This works well for IBM generally (the products are shit) but especially well for Watson because it's extremely easy to sell AI without getting bogged down in details. The cynic in me says that every use of the term AI in any capacity is to sell experience and not functionality. When was the last time you used a product billed as 'AI' and thought 'wow, this is a huge game changer'? Siri is cool, but it's ultimately not super useful. Google translate is incredible, but it can only do what it can do because of the absolutely mind-boggling amount of training data that google can access. Most disciplines have the problem of not enough data, despite what 'big-data' folks say. In contrast, humans can extrapolate and make reliable predictions about the future based on really small sample sizes. We can pick up a new skill or recognize a new pattern with a high degree of accuracy really effing fast compared to a computer. This gives humans an enormous advantage. If IBM and anyone else in this space were really focused on delivering excellent real-world results, step 0 is building out world-class data integration and search tools (which we still actually suck at, weirdly.)
- NicoJuicy 8y ago
  Many people mention even a worser search experience with Google and AI
  - davidf18 8y ago
    Could you please elaborate? Does that include DeepMind?
- mlthoughts2018 8y ago
  In fact, I would say apart from maybe self-driving cars, almost all of the biggest gains from machine learning are in unsexy, hidden backend problems, like automatically rectifying disparate data, optimizing resource utilization, flagging difficult-to-articulate events or triggers in a stream of data too large for human evaluation, machine translation, and other “unsexy” things. Product interfaces usually offer simple features to users and the value proposition is easy to see. Effective use of machine learning is well hidden upstream in a bunch of unsexy preprocessing or heavy lifting to get to the interface. Not something you’d ever need to emphasize in marketing, except maybe at tech meetups or in recruiting materials, but not to the end consumer. It just makes pop references to AI-powered products more egregious.
ThomPete 8y ago
Agree. Consultancies rarely sell products or solutions. Instead they sell project management by making it sound like you are a more safe bet than a smaller product studio who actually do make products work. Its a real shame but i mostly blame the zero mistake KPI culture primarily fueled by how managers on the client side are promoted.
mathattack 8y ago
Over the past 10 years I’ve been surprised when anyone smart would join IBM. I understand why people hung on, but why join that dying ship? Now I can’t think of any still there.
- occamrazor 8y ago
  Who says IBM is dying? They aren’t anymore (since a long time) at the bleeding edge of research, but they still have a solid consulting and integration business. They also offer decent salaries and good opportunities for sales-oriented technical people. It wouldn’t be my first choice of employer, however I can see how some people would enjoy working there.
  - mathattack 8y ago
    Dying may be extreme. Perhaps it’s safer to say that they are in the mode of returning capital to investors rather than growing. I’m at a large customer of theirs and they are bleeding the customer for every dollar as they get phased out. Very low caliber of services professionals too.

stefan_ 8y ago

It's double whammy for the Watson stuff, because not only did IBM lose track with AI, they also had that whole cloud thing whoosh by them. So not only do you have marketing telling outrageous lies about the abilities of their AI systems, they're also exaggerating the cloud angle where Watson is a cornerstone.

bunderbunder 8y ago
Possibly the most outrageous lie was calling it AI in the first place. Watson is a $@#% amazing information retrieval system. Information retrieval is only a small part of what people think of when they think, "AI".
- goatlover 8y ago
  > amazing information retrieval system Which I imagine still has lots of value on it's own for large, complicated data sets.
  - bunderbunder 8y ago
    Yep. But if your marketing and product management are all focused on selling AI instead of IR, then they're not really working toward finding a way to deliver the IR value they have to the people who need it. I'd actually like to give Watson a spin for an IR problem I'm looking at, but, thanks to their hype machine being set to overdrive, they've got the thing priced in the "The Bold Leaders of the Future Creating a Bright New Tomorrow Full of People in Glasses Staring Wistfully Toward the Right Edge of the Photograph, While Blue Curvy Streaks Wave Through the Background and Random Zeroes and Ones Float Around Their Heads" tier. Sadly, I've only got a "businesses solving business problems" sized budget.
    - solarkraft 8y ago
      What is your use case and what are the alternatives you're considering? I'm trying to understand what to imagine here.
    - 8y ago

alienreborn 8y ago

Whats EMR? (definitely not Elastic Map Reduce)

a5huynh 8y ago
Electronic Medical Record(s)
deleted 8y ago
[deleted]

andyidsinga 8y ago

There's a pretty good podcast interview with Eugene Dubossarsky that has relevant discussion about issues with management and data science in general. https://www.datafuturology.com/podcast/1 https://www.datafuturology.com/podcast/1 Here are a few of my notes (my words not the interviewee's): - in order to use data science, you have to have creative people thinking about data on the front end - they don't have to be data scientists, but they need to be creative and want data to support decisions and iteration via feedback loops - that creativity and desire will lead to "doing good data science" - management on the receiving end of data science output must be intelligent in terms of synthesizing many inputs and have a strong desire to puzzle through the implications. If management is asking the data science to actually make the decisions - the situation is broken - data science must be done with provisions for decision support and feedback loops; this is the output that is helping drive the business. - Lack of desire for decision support and feedback loops leads to "fancy pets" and management using data science as a means to brag about what they are doing; but the data science might not being doing anything to drive the business meaningfully. - data science that attempts to actually make decisions vs providing decision support is likely in the category of "commodity data science". Corollary : non-commodity data science is the kind that supports decisions in executing higher-level business strategy. Strategy at that level has rather unique attributes and is embedded in unique circumstances for a particular business. This requires a good data scientist to help tackle. (hope this is useful) (edit typos,grammar)

andyidsinga 8y ago
BTW - in listening to that podcast I found a lot of parallels with database design. whenever I'm asked to design a database for an early-stage system (I work in early stage tech ventures), I ask the following: - what are the questions that this database should answer for you? How are those questions supporting your business goals 3,6,12 months out? (I'm trying to get to the business requirements here) - who will be asking those questions (I'm trying to put together some user personas in my head) - how frequently will they be asking these questions? corollary: how often will historical data be needed? (I'm thinking hot vs cold and complexity of retrieval, minimally required performance) - how much data to we anticipate is needed to answer the questions (this is really tricky in new ventures - often the answer is more data than what will actually occur in practice in the first year)? - finally, what systems & tools are people using to ask the questions and be notified of events? (I'm thinking about interfacing, apis) its all an attempt to stay very focused on the questions and business drivers and the people who use the answers.

wnevets 8y ago

>"Offering managers didn’t have technical backgrounds and sometimes came up with ideas for new products that were simply impossible." how many of those managers will now be able to get even a higher paying job because they have manager of Watson AI project on their resume?

lifeisstillgood 8y ago

It has been ever thus with IBM adverts - this was me moaning online from early 2000's http://www.mikadosoftware.com/articles/ibmadverts http://www.mikadosoftware.com/articles/ibmadverts

emmelaich 8y ago
That's great! Thanks.

LrnByTeach 8y ago

following is a text snippet from the Article . I do not know which part of the following is AI , it is pure "data analysis" writing few SQL queries . This is the Biggest problem of IBM, they bill these kind of things as AI . > A clinic could use the system to search its patient records and find, for example, all the men over age 45 who were overdue for a colonoscopy, and then use an autocall to remind them to schedule the dreaded appointment.

fjsolwmv 8y ago
The point is it's NOT SQL. It's NLP and whatnot to extract value from relatively natural non-fully schematized data.
- boxy310 8y ago
  DOB and date of last checkup are highly schematizable. This specific example is very well-suited to a SQL solution dumping into an autocaller queue. Maybe this was a terrible example, and the author didn't grasp a good example of legitimately non-schematic data points?
cabaalis 8y ago
Being directly involved in the execution of the example you gave, I believe ML would be unnecessary and error-prone. Even if successful, a system which could "interpret" a health record (such as a freetext note) using anything other than properly codified data would set the health industry back a decade. Moving doctors away from freetexting their notes is the only way to advance the industry.
- skolman 8y ago
  This is something I've always wondered about- how would you do it? How do you break the cycle of freetext EHR notes? Doing so could have incredible utility for sharing data across various clinics/hospitals/pharmacies/etc.
  - sbarre 8y ago

duxup 8y ago

Other reports seemed to indicate that Watson (and maybe all AI) requires a lot of very careful, slow, and somewhat arduous data entry and testing to get good results. I can imagine someone who doesn't know at IBM selling a product: "Hey we will solve all these problems like magic!" Then IBM comes back: "Hey do you have all this data in a specific format and a ton of time to enter and test it and maybe we'll get back to you???" That's a big loss of trust there with the customer if you come back with that. It seems like these are products where a lot of caveats needs to be made clear to customers and a real careful technical partnership formed with them to succeed long term. You have to bring the customer along for the ride and exploration and keep them excited for a long time it sounds to make it work.

WhitneyLand 8y ago

Are the articles at ieee.org usually technically competent? To see one of their articles with the common press confusion of mixing different definitions and interpretations of AI (correct or incorrect) doesn’t help build confidence. For example, what was used to play Jeopardy vs. approaches being taken to improve cancer treatment, are just so different, it seems almost disingenuous to throw it all haphazardly into one conceptual bucket. The article does IBM a disservice in some ways. They come off looking bad overall but some of the failed projects mentioned like MD Anderson, failed for reasons beyond any control they had, other than recognizing some obvious red flags earlier and detaching their name and participation from it. On the other hand I believe the article lets them offf the hook to easily when they bring out the old trope they’ve been using for years, which is encapsulated here: “IBM Watson has great AI” [one engineer said] “It’s like having great shoes, but not knowing how to walk—they have to figure out how to use it.” It doesn’t make sense to say, xyz is great we just have to figure out how to use it, as a stand alone argument. It’s nonsensical unless you mention something about the seeming implied untapped potential, specific innovations, novel approach, or whatever makes it great. I’m not familiar with all their IP so maybe there are some great things, you just don’t get to claim that and get off the hook so many times in the press without providing at least some detail or reference point.

anonymous5133 8y ago

IMO, this is a bigger issue in the industry. People are hyping up ML/AI to the point where the actual application is either impossible or extremely difficult. Just look at how many people are so fearful of AI/ML taking away jobs and replacing humans. Anyone in the field knows that AI/ML can take away jobs but it is more of the low-end jobs and everything happens gradually rather than immediately. AI/ML is being more as a tool to enhance human productivity rather than as a direct replacement for entire occupations unless the job is very basic to begin with.

xyhopguy 8y ago
IMO AI/ML scientists are drastically underestimating the complexity of biology. Once scientists in biology and healthcare get on board like they are in linguistics and computer vision I'd expect things to pick up.
- anonymous5133 8y ago
  agree

tlobes 8y ago

The article only mentions the engineers being laid off, not the managers who botched it, nor the executives who hired with incompetence. The reward structure of such a corporate environment would seem counter-intuitive to success.