4 ms·

>If you think about it, that's the natural outcome. Why? Because people in corporations don't have the incentive to benefit the business but to progress their c

by apohn 6y ago

>If you think about it, that's the natural outcome. Why? Because people in corporations don't have the incentive to benefit the business but to progress their careers and that's done through meeting the goals for their position and make their upper ups progress with their careers too. This is one of the reasons I roll my eyes whenever I read something like "McKinsey says 75% of Big Data/AI/Buzzword projects do not deliver any value." What's the baseline for failing and/or delivering zero value because those projects were destined to fail?

bonoboTP 6y ago
> because of silly management decisions? The whole point is, from their point of view those decisions are rational. It's much more lucrative from their (managers') personal point of view to develop a smokes-and-mirrors looks-good-on-ppt AI project. To be safe from risk, don't give the AI people too much responsibility, let them "do stuff", who cares, the point is we can now say we are an AI-driven company on the brochures, and we have something to report up to upper management. When they ask "are we also doing this deep learning thing? It's important nowadays!" we say "Of course, we have a team working on it, here's a PPT!". An actual AI project would have much bigger risks and uncertainty. I as a manager may be blamed for messing up real company processes if we actually rely on the AI. If it's just there but doesn't actually do anything, it's a net win for me. Note how this is not how things run when there are real goals that can be immediately improved through ML/AI and it shows up immediately on the bottom line, like ad and recommendation optimizations in Youtube or Netflix or core product value like at Tesla etc. The bullshit powerpoint AI with frustrated and confused engineers happens in companies where the connection is less direct and everyone only has a nebulous idea of what they would even want out of the AI system (extract valuable business knowledge!).
- huffmsa 6y ago
  I think the problem a lot of places has been wanting "appealing" ML/AI solutions. The kind you write papers about and put on Powerpoints. The useful AI/ML isn't glamorous, it's quite boring and ugly. Things like spam detection, image labeling, event parsing, text classification. It's hard to get a big, shiny model into direct user facing systems.
  - bonoboTP 6y ago
    What would you categorize as shiny in this case? "spam detection, image labeling, event parsing, text classification" can be implemented in lots of ways, simple and shiny as well. Either way I don't think it matters too much because people can't really tell simple from shiny as long as the buzzword bullet points are there. The point is rather that the job of the data science team is to deliver prestige to the manager, not to deliver data science solutions to actual practical problems. It's enough if they work on toy data and show "promising results" and can have percentages, impressive serious charts and numbers on the powerpoint slides. I've heard from many data scientists in such situations that they don't get any input on what they should actually do, so they make up their own questions and own tasks to model, which often has nothing to do with actual business value, but they toy around with their models, produce accuracy percentages and that's enough.
beambot 6y ago
Followed immediately by the solution: Hire McKinsey analysts to help you deliver insights -- which may or may not get implemented or deliver the results, but it won't matter because everyone has moved on to the next "project".
- deleted 6y ago
  [deleted]
mgleason_3 6y ago
OK, so, we're scientists...and we're in the middle of a pandemic...amplifying/arguing over a graph showing a steep decline in job listing...that doesn't control for the pandemic...or even include a line for "overall job loss"... https://www.burning-glass.com/u-s-job-postings-increase-fourth-week-row-may/ https://www.burning-glass.com/u-s-job-postings-increase-four... Looks like all job postings "collapsed during the pandemic"
- stjohnswarts 6y ago
  yeah looks like at the least you might have lines for "overall CS based jobs" and "overall tech industry" and see the same sort of fall off appears. While not all that scientific either logically if you see similaries you can cast some more doubt/support on the hypothesis that ML is special and failing. How is it doing relative to other "hyped" or even just plain technical hiring/firing trends.
jkinudsjknds 6y ago
McKinsey DS here. I don't think I've ever heard such a claim about data science whatever, although I would probably believe it. I do hear such claims a lot in the context of big transformations. These claims are usually high level and based on surveys or whatever. Failing usually means leadership gave up. As far as high level awareness of project success rates, it's probably accurate enough to justify the point: companies are generally bad at doing X. This tends to be true for many different kinds of X, because business is hard. I generally don't agree that people make up destined to fail projects for selfish gains. I'm sure it happens, but that seems bottom of the barrel in terms of problems to fix. With DS specifically, leaders just don't know what to do. So they hire data scientists, and the data scientists don't know anything about the business, so they make some dashboards or whatever and nobody uses them. It's really not easy. Business is hard.
xmprt 6y ago
Why do you roll your eyes? Isn't it a useful metric to know that most of the projects that are hiring these buzzword technologies are destined to fail (whether that's because the problem space wasn't fit for ML or whether management went on a hiring spree to pump their resume)?
- apohn 6y ago
  I dislike these types headlines because I've interacted with a lot of people who see this as evidence that ML/AI is BS and destined to fail. The reality is that a project with unrealistic expectations is going to fail regardless of it being an AI project or somebody baking a loaf of bread at home. It's important to understand why stuff fails. That's the only way to stop things from failing in the future and make sure people are on the right path to not failing. If a large number project failures are management failures, it's useful to know that. Otherwise you try to fix everything except the management structure.
  - eaton 6y ago
    Statistically, though, that's what the people in this thread are saying — that the majority of the projects in ML/AI are destined to fail because they're BS with unrealistic goals. "Personalization" is in a similar place for digital publishing; everyone wants it, products and services carry big price tags, and few organizations want to invest in foundational work or simple, iterative improvements. So they swing for the stars with unrealistic goals like "micro-targeted messaging perfectly tailored to every visitor, no matter where they are in the customer journey" and the results are predictable… I take the increasingly grim accounts of project failure rates from analyst firms as a good sign — they can be used to sober up executives with unrealistic dreams.
tuyiown 6y ago
Funny thing is, McKinsey consultants exactly knows why it fails, but won't say it because the responsibility falls on the execs that choose them for their reports or consulting. A paying customer should not have any reason to fear to regret choosing you as a consultant.

by apohn 6y ago

bonoboTP 6y ago

> because of silly management decisions? The whole point is, from their point of view those decisions are rational. It's much more lucrative from their (managers') personal point of view to develop a smokes-and-mirrors looks-good-on-ppt AI project. To be safe from risk, don't give the AI people too much responsibility, let them "do stuff", who cares, the point is we can now say we are an AI-driven company on the brochures, and we have something to report up to upper management. When they ask "are we also doing this deep learning thing? It's important nowadays!" we say "Of course, we have a team working on it, here's a PPT!". An actual AI project would have much bigger risks and uncertainty. I as a manager may be blamed for messing up real company processes if we actually rely on the AI. If it's just there but doesn't actually do anything, it's a net win for me. Note how this is not how things run when there are real goals that can be immediately improved through ML/AI and it shows up immediately on the bottom line, like ad and recommendation optimizations in Youtube or Netflix or core product value like at Tesla etc. The bullshit powerpoint AI with frustrated and confused engineers happens in companies where the connection is less direct and everyone only has a nebulous idea of what they would even want out of the AI system (extract valuable business knowledge!).

huffmsa 6y ago
I think the problem a lot of places has been wanting "appealing" ML/AI solutions. The kind you write papers about and put on Powerpoints. The useful AI/ML isn't glamorous, it's quite boring and ugly. Things like spam detection, image labeling, event parsing, text classification. It's hard to get a big, shiny model into direct user facing systems.
- bonoboTP 6y ago
  What would you categorize as shiny in this case? "spam detection, image labeling, event parsing, text classification" can be implemented in lots of ways, simple and shiny as well. Either way I don't think it matters too much because people can't really tell simple from shiny as long as the buzzword bullet points are there. The point is rather that the job of the data science team is to deliver prestige to the manager, not to deliver data science solutions to actual practical problems. It's enough if they work on toy data and show "promising results" and can have percentages, impressive serious charts and numbers on the powerpoint slides. I've heard from many data scientists in such situations that they don't get any input on what they should actually do, so they make up their own questions and own tasks to model, which often has nothing to do with actual business value, but they toy around with their models, produce accuracy percentages and that's enough.

beambot 6y ago

Followed immediately by the solution: Hire McKinsey analysts to help you deliver insights -- which may or may not get implemented or deliver the results, but it won't matter because everyone has moved on to the next "project".

deleted 6y ago
[deleted]

mgleason_3 6y ago

OK, so, we're scientists...and we're in the middle of a pandemic...amplifying/arguing over a graph showing a steep decline in job listing...that doesn't control for the pandemic...or even include a line for "overall job loss"... https://www.burning-glass.com/u-s-job-postings-increase-fourth-week-row-may/ https://www.burning-glass.com/u-s-job-postings-increase-four... Looks like all job postings "collapsed during the pandemic"

stjohnswarts 6y ago
yeah looks like at the least you might have lines for "overall CS based jobs" and "overall tech industry" and see the same sort of fall off appears. While not all that scientific either logically if you see similaries you can cast some more doubt/support on the hypothesis that ML is special and failing. How is it doing relative to other "hyped" or even just plain technical hiring/firing trends.

jkinudsjknds 6y ago

McKinsey DS here. I don't think I've ever heard such a claim about data science whatever, although I would probably believe it. I do hear such claims a lot in the context of big transformations. These claims are usually high level and based on surveys or whatever. Failing usually means leadership gave up. As far as high level awareness of project success rates, it's probably accurate enough to justify the point: companies are generally bad at doing X. This tends to be true for many different kinds of X, because business is hard. I generally don't agree that people make up destined to fail projects for selfish gains. I'm sure it happens, but that seems bottom of the barrel in terms of problems to fix. With DS specifically, leaders just don't know what to do. So they hire data scientists, and the data scientists don't know anything about the business, so they make some dashboards or whatever and nobody uses them. It's really not easy. Business is hard.

xmprt 6y ago

Why do you roll your eyes? Isn't it a useful metric to know that most of the projects that are hiring these buzzword technologies are destined to fail (whether that's because the problem space wasn't fit for ML or whether management went on a hiring spree to pump their resume)?

apohn 6y ago
I dislike these types headlines because I've interacted with a lot of people who see this as evidence that ML/AI is BS and destined to fail. The reality is that a project with unrealistic expectations is going to fail regardless of it being an AI project or somebody baking a loaf of bread at home. It's important to understand why stuff fails. That's the only way to stop things from failing in the future and make sure people are on the right path to not failing. If a large number project failures are management failures, it's useful to know that. Otherwise you try to fix everything except the management structure.
- eaton 6y ago
  Statistically, though, that's what the people in this thread are saying — that the majority of the projects in ML/AI are destined to fail because they're BS with unrealistic goals. "Personalization" is in a similar place for digital publishing; everyone wants it, products and services carry big price tags, and few organizations want to invest in foundational work or simple, iterative improvements. So they swing for the stars with unrealistic goals like "micro-targeted messaging perfectly tailored to every visitor, no matter where they are in the customer journey" and the results are predictable… I take the increasingly grim accounts of project failure rates from analyst firms as a good sign — they can be used to sober up executives with unrealistic dreams.

tuyiown 6y ago

Funny thing is, McKinsey consultants exactly knows why it fails, but won't say it because the responsibility falls on the execs that choose them for their reports or consulting. A paying customer should not have any reason to fear to regret choosing you as a consultant.