15 ms·

AI skills are worth less than you think (2019)

89 points by tate 6y ago

goatinaboat 6y ago
This mirrors what I see - domain experts using models to validate hypotheses trump p-value fishers without that domain knowledge every time. The only risk the former take is getting trapped in a local minima but that trap can largely be avoided simply by knowing its there.
- beforeolives 6y ago
  > domain experts using models to validate hypotheses trump p-value fishers without that domain knowledge every time This basically says that people who have one subset of the required skills are better than people who have none of them. That's obviously true but I'm not sure why it's relevant to use the worst part of a group ("p-value fishers") in that comparison. The more interesting question is whether skills in statistics, applied math and computer science carry over across different domains - and I think the answer is that they clearly do.
yowlingcat 6y ago
I agree with most of the premises in this article. It reminds me of one of my favorite seminal a16z articles: https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-its-different-from-traditional-software/ https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-i... Basically, a16z argue that AI businesses by default are closer to consultingware than software companies because their data hungry nature works to impede rather than enable unit economics. I find this to be broadly accurate. Many of the existential challenges facing a would be AI implementation include data unit economics and domain validation -- the 0-1 product problems, not the 1-1000 problems.
- dpflan 6y ago
  While looking up more about the author of the post, you'll find that he is the "CTO at MicroscopeIT" (https://twitter.com/szopa?lang=en https://twitter.com/szopa?lang=en), which upon further link-clicking appears to be what you say: "consultingware" (https://www.microscopeit.com/ https://www.microscopeit.com/) [But I don't see him listed there...]
dusted 6y ago
They may be worth little, but probably not less than _I_ think ;-)
dpflan 6y ago
Seems like this focus of this article is the field moves quickly. That's it. The AI and software engineering skills required to make something that was painfully tedious X years ago and make it easy today is clearly valuable. It takes such "AI skills" to work on these projects. The AI/ML component of a company is a smaller part of the entire infrastructure. You still need to be build the infra. Which means that in a fast paced industry, with proper infra, you could replace/substitute your AI/ML component.
- chromanoid 6y ago
  I agree. When you truly understand what you are doing even in a field that moves quickly, your knowledge stays at least to some extent valuable. I would claim that this is especially true for the field of AI. While libraries and frameworks will move as quick as the JavaScript ecosystem, the essence will stay the same. The joke here is that "skills" in AI seem to be more like being able to intuitively turn the right knobs to make the AI algorithm do what you want. This "skill" is of course highly dependent on what knobs there are and what algorithm is used. It's like being able to develop AngularJS only. Nobody should expect their skills being valuable after some time regarding frameworks and libraries in a buzzing field like AI. Just look at how much web development changed in the last decade.
selimthegrim 6y ago
Antonio Criminisi is at Amazon now IIRC
Hotple 6y ago
People who just use library functions without having any understanding of how they work are not going to do as well as the people who actually understand the math. The people who have real understanding will always be called in to figure out the hard stuff.
- dpflan 6y ago
  Completely agree, sure you can make superficial progress, but then you will need to further improve, and without deep knowledge of the underlying mechanics you will be lost.
  - datameta 6y ago
    Also rather important is to know what paths not to pursue in the search for a solution. Basically a solid understand of bias/variance will carry you far and help you avoid doing a week of work for an optimization that doesn't actually pan out.
  - omgwtfbbq 6y ago
    Are you talking specifically about research? Because in my experience the time and $ delta needed to bring a classification model from say 93-96% accuracy is not worth it to most businesses. So all your special "deep knowledge" is irrelevant in most use cases
    - dpflan 6y ago
      If you're in research I would assume a deeper background or at least an environment where you are encouraged to develop your background. In industry, yes, time to value is important, but it seems clear that in some part of the market competition will require squeezing out those remaining % points of performance -- a company scaling up with poorer accuracy would have more problems...right? Do you have experience in industry? I would like to hear more from your perspective?
      - omgwtfbbq 6y ago
johbjo 6y ago
His examples are CNN image applications, and it is true that these were spectacular new results 10 years ago. Since then, the tedious engineering has improved and recreating the results now are indeed just a few lines of python. The reason they were so impressive then is because it seemed they were "more intelligent" than they actually are, and people extrapolated the same level of performance on any task. Achieving it seemed more impressive then, than now. But "AI skills" should be broader than image classification.
- willeh 6y ago
  I think the many point of the article isn't necessarily that it is easy to construct models in terms of lines of code but that any tweaking of the model is dominated by increases in data-size. Why pay for a PHD when you can invest in more data instead.
  - AstralStorm 6y ago
    Except when you cannot: with spaces too big, such as strategies and games, or too rarified to be sampled. Or ones where generating completely new output is required, such as music and writing.
evancox100 6y ago
I'm sorry, I just can't get over why they called Roberta "Bob"
- bradjohnson 6y ago
  Alice and Bob are common placeholder names in crypto examples
  - baobabKoodaa 6y ago
    When both Alice and Bob represent women, you have achieved 100% diversity. Thus, Bob shall be henceforth shorthand for Roberta.
jplr8922 6y ago
What about AI soft skills? The ability to explain your model to non-technical people? To picth your solution to executives? To understand business concepts critical to increase the value of the product? Dealing with co-researchers from a different field who might have different criterias about what is a valid factor? Dealing with a IT director who thinks he gets AI because he did stats101 over 10 years ago? The value of AI hard skills do not exist in a vacuum. It might be true that doing AI is easier when all the infrastructure and management buy-in is there. Unfortunatly, these are hard to come by, and easy to loose...
- tootie 6y ago
  Yes. This is a huge gap right now. AI/ML have advanced to the point that complex tasks can be done fairly cheaply. Implementations are easier to find than valid use cases. I've seen more than a few enterprises build giant data lakes and feed them into sophisticated analysis tools, yet the data never seems to inform any decisions. I've seen computer vision used for some cute gimmicks and it's very accurate, but no one could figure out what it was actually good for in a practical application. Same for AR, blockchain and some others. The tech is amazing, but no one knows what to do with it. We need some user experience visionaries more than we need more engineers right now.
darepublic 6y ago
giant google boss laughs diabolically. you ignorant fools... do you think you can compete with me? with all my data (points to mountains of high quality crawled sentence fragments and images). Author of article siddles up. I told you so, he says, shaking his head.
- bckr 6y ago
  YC startup founder floating above silicon valley in a black jumpsuit, greasy hair floating upwards, charged with the power of pre seed "I'll make you eat those words!"
  - bpiche 6y ago
    > YC startup founder shoots a powerful wave of investment capital energy at a research problem Google is working on > Google throws it back at the YC startup founder, gravely wounding them > YC startup founder draws on the energy of crowdsourced data from social media users all over planet Earth who think they are just answering reCAPTCHA questions or something > bootstraps a predictive spirit bomb with a GAN and throws it back at Google, destroying the Earth
magwa101 6y ago
The picture in the article illustrates the over dramatization. Humans always seek the edge of knowledge. We expand all the time. It's not like we don't have a billion problems to solve.
amelius 6y ago
The deep learning expert is the new web developer.
deleted 6y ago
[deleted]
Tycho 6y ago
Two scenarios to consider: 1. Firm creates an AI group to provide innovative solutions to its various business problems. A promising, seemingly well-suited problem is chosen, but after a few months of requirements gathering and prototyping, it turns out that the AI methods are not very effective. At this point the AI group and the business partners either double down, preferring to save face and live with a sub-optimal solution, or else the AI group abandon the project and the business partners are back to square one, having wasted months, and will now bear all the repeated overhead costs of restarting the project with another set of engineers. 2. Firm has an engineering group who are familiar with AI techniques but are also prepared to rely on 'old fashioned' approaches to get the job done. A problem is identified that initially seems like a good opportunity to use AI methods, but after a few months of requirement gathering and prototyping it turns out they are not very effective. The engineer working on the project then takes off their 'data scientist' hat and puts on their 'software developer' hat and solves the problem with other methods. Since they have already been working on the problem for a few months, no additional problem scoping is required and the business partners get their problem solved with minimal delay.

89 points by tate 6y ago

This mirrors what I see - domain experts using models to validate hypotheses trump p-value fishers without that domain knowledge every time. The only risk the former take is getting trapped in a local minima but that trap can largely be avoided simply by knowing its there.

beforeolives 6y ago
> domain experts using models to validate hypotheses trump p-value fishers without that domain knowledge every time This basically says that people who have one subset of the required skills are better than people who have none of them. That's obviously true but I'm not sure why it's relevant to use the worst part of a group ("p-value fishers") in that comparison. The more interesting question is whether skills in statistics, applied math and computer science carry over across different domains - and I think the answer is that they clearly do.

yowlingcat 6y ago

I agree with most of the premises in this article. It reminds me of one of my favorite seminal a16z articles: https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-its-different-from-traditional-software/ https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-i... Basically, a16z argue that AI businesses by default are closer to consultingware than software companies because their data hungry nature works to impede rather than enable unit economics. I find this to be broadly accurate. Many of the existential challenges facing a would be AI implementation include data unit economics and domain validation -- the 0-1 product problems, not the 1-1000 problems.

dpflan 6y ago
While looking up more about the author of the post, you'll find that he is the "CTO at MicroscopeIT" (https://twitter.com/szopa?lang=en https://twitter.com/szopa?lang=en), which upon further link-clicking appears to be what you say: "consultingware" (https://www.microscopeit.com/ https://www.microscopeit.com/) [But I don't see him listed there...]

dusted 6y ago

They may be worth little, but probably not less than _I_ think ;-)

dpflan 6y ago

Seems like this focus of this article is the field moves quickly. That's it. The AI and software engineering skills required to make something that was painfully tedious X years ago and make it easy today is clearly valuable. It takes such "AI skills" to work on these projects. The AI/ML component of a company is a smaller part of the entire infrastructure. You still need to be build the infra. Which means that in a fast paced industry, with proper infra, you could replace/substitute your AI/ML component.

chromanoid 6y ago
I agree. When you truly understand what you are doing even in a field that moves quickly, your knowledge stays at least to some extent valuable. I would claim that this is especially true for the field of AI. While libraries and frameworks will move as quick as the JavaScript ecosystem, the essence will stay the same. The joke here is that "skills" in AI seem to be more like being able to intuitively turn the right knobs to make the AI algorithm do what you want. This "skill" is of course highly dependent on what knobs there are and what algorithm is used. It's like being able to develop AngularJS only. Nobody should expect their skills being valuable after some time regarding frameworks and libraries in a buzzing field like AI. Just look at how much web development changed in the last decade.

selimthegrim 6y ago

Antonio Criminisi is at Amazon now IIRC

Hotple 6y ago

People who just use library functions without having any understanding of how they work are not going to do as well as the people who actually understand the math. The people who have real understanding will always be called in to figure out the hard stuff.

dpflan 6y ago
Completely agree, sure you can make superficial progress, but then you will need to further improve, and without deep knowledge of the underlying mechanics you will be lost.
- datameta 6y ago
  Also rather important is to know what paths not to pursue in the search for a solution. Basically a solid understand of bias/variance will carry you far and help you avoid doing a week of work for an optimization that doesn't actually pan out.
- omgwtfbbq 6y ago
  Are you talking specifically about research? Because in my experience the time and $ delta needed to bring a classification model from say 93-96% accuracy is not worth it to most businesses. So all your special "deep knowledge" is irrelevant in most use cases
  - dpflan 6y ago
    If you're in research I would assume a deeper background or at least an environment where you are encouraged to develop your background. In industry, yes, time to value is important, but it seems clear that in some part of the market competition will require squeezing out those remaining % points of performance -- a company scaling up with poorer accuracy would have more problems...right? Do you have experience in industry? I would like to hear more from your perspective?
    - omgwtfbbq 6y ago

johbjo 6y ago

His examples are CNN image applications, and it is true that these were spectacular new results 10 years ago. Since then, the tedious engineering has improved and recreating the results now are indeed just a few lines of python. The reason they were so impressive then is because it seemed they were "more intelligent" than they actually are, and people extrapolated the same level of performance on any task. Achieving it seemed more impressive then, than now. But "AI skills" should be broader than image classification.

willeh 6y ago
I think the many point of the article isn't necessarily that it is easy to construct models in terms of lines of code but that any tweaking of the model is dominated by increases in data-size. Why pay for a PHD when you can invest in more data instead.
- AstralStorm 6y ago
  Except when you cannot: with spaces too big, such as strategies and games, or too rarified to be sampled. Or ones where generating completely new output is required, such as music and writing.

evancox100 6y ago

I'm sorry, I just can't get over why they called Roberta "Bob"

bradjohnson 6y ago
Alice and Bob are common placeholder names in crypto examples
- baobabKoodaa 6y ago
  When both Alice and Bob represent women, you have achieved 100% diversity. Thus, Bob shall be henceforth shorthand for Roberta.

jplr8922 6y ago

What about AI soft skills? The ability to explain your model to non-technical people? To picth your solution to executives? To understand business concepts critical to increase the value of the product? Dealing with co-researchers from a different field who might have different criterias about what is a valid factor? Dealing with a IT director who thinks he gets AI because he did stats101 over 10 years ago? The value of AI hard skills do not exist in a vacuum. It might be true that doing AI is easier when all the infrastructure and management buy-in is there. Unfortunatly, these are hard to come by, and easy to loose...

tootie 6y ago
Yes. This is a huge gap right now. AI/ML have advanced to the point that complex tasks can be done fairly cheaply. Implementations are easier to find than valid use cases. I've seen more than a few enterprises build giant data lakes and feed them into sophisticated analysis tools, yet the data never seems to inform any decisions. I've seen computer vision used for some cute gimmicks and it's very accurate, but no one could figure out what it was actually good for in a practical application. Same for AR, blockchain and some others. The tech is amazing, but no one knows what to do with it. We need some user experience visionaries more than we need more engineers right now.

darepublic 6y ago

giant google boss laughs diabolically. you ignorant fools... do you think you can compete with me? with all my data (points to mountains of high quality crawled sentence fragments and images). Author of article siddles up. I told you so, he says, shaking his head.

bckr 6y ago
YC startup founder floating above silicon valley in a black jumpsuit, greasy hair floating upwards, charged with the power of pre seed "I'll make you eat those words!"
- bpiche 6y ago
  > YC startup founder shoots a powerful wave of investment capital energy at a research problem Google is working on > Google throws it back at the YC startup founder, gravely wounding them > YC startup founder draws on the energy of crowdsourced data from social media users all over planet Earth who think they are just answering reCAPTCHA questions or something > bootstraps a predictive spirit bomb with a GAN and throws it back at Google, destroying the Earth

magwa101 6y ago

The picture in the article illustrates the over dramatization. Humans always seek the edge of knowledge. We expand all the time. It's not like we don't have a billion problems to solve.

amelius 6y ago

The deep learning expert is the new web developer.

deleted 6y ago

[deleted]

Tycho 6y ago

Two scenarios to consider: 1. Firm creates an AI group to provide innovative solutions to its various business problems. A promising, seemingly well-suited problem is chosen, but after a few months of requirements gathering and prototyping, it turns out that the AI methods are not very effective. At this point the AI group and the business partners either double down, preferring to save face and live with a sub-optimal solution, or else the AI group abandon the project and the business partners are back to square one, having wasted months, and will now bear all the repeated overhead costs of restarting the project with another set of engineers. 2. Firm has an engineering group who are familiar with AI techniques but are also prepared to rely on 'old fashioned' approaches to get the job done. A problem is identified that initially seems like a good opportunity to use AI methods, but after a few months of requirement gathering and prototyping it turns out they are not very effective. The engineer working on the project then takes off their 'data scientist' hat and puts on their 'software developer' hat and solves the problem with other methods. Since they have already been working on the problem for a few months, no additional problem scoping is required and the business partners get their problem solved with minimal delay.