7 ms·

>A large portion of AI investment is in self-driving cars, and as fully autonomous vehicles are still far from being a commodity, some hypothesize that investor

>A large portion of AI investment is in self-driving cars, and as fully autonomous vehicles are still far from being a commodity, some hypothesize that investors will lose hope in AI altogether. Google has freezed hiring for ML researchers. Uber laid off the research half of their AI team. There’s rumor that due to a large number of people taking ML courses, there will be far more people with ML skills than ML jobs. Ouch >.< As someone who does ML for a living this hits. When switching companies in 2019 for the first time I had a hard time despite having 10+ years experience. Linkedin maxed out saying over 200 people had applied to every job I was looking at. Thankfully I have a rare skill set. It's not found in classes, books, or boot camps. On the downside there are not many jobs to choose from as companies don't know what I do can be done. (Time series classification and prediction, usually sensor data.) Hopefully the industry matures. I've bumped into data scientists who don't know how to do feature engineering as well as ones who can't grok ML. It makes the whole industry look bad.

huac 6y ago
# of people who want to do "machine learning" > # of ML jobs > # of people good at ML
- proverbialbunny 6y ago
  There should be a law that require all ML and DS classes, books, and bootcamps start by manually labeling data.
- kqr 6y ago
  Yes, I was going to say. Taking courses in "ML" is a great way to get to write a hyped-up term on your resume. Actually understanding and being able to write code for statistical learning is something else entirely. Edit: On the other hand, I think Carmack observed that as long as you get the sign right, you'll get something out of it, even if it won't learn very quickly. So maybe speed of implementation and learning is how skilled people will differentiate themselves?
  - proverbialbunny 6y ago
    Research tasks take a long time. If you're going off of speed, it's no longer data science it's software engineering. (Though, to be fair, with all the new software engineers becoming data scientists, the data science title is becoming more like software engineering.) imho metaphysics and metalearning might be a better way to differentiate.
    - kqr 6y ago
      Right. There's room for research, and then there's room for commoditising existing research. I think the latter is where we currently have the most to gain from expanding our efforts in, and this was the context in which I made my comment. I see now that I made that very unclear. Sorry!
      - deleted 6y ago
        [deleted]
      - proverbialbunny 6y ago
        No need to be sorry. I'm equally to blame for my own lack of communication skills. ^_^
flak48 6y ago
As for Uber, the AI researchers that were laid off were not from the self driving division. Google's freeze as far as I know is for all engineers ( only backfilling attrition is permitted), not limited to ML. I personally feel this slump in the ML job market is only temporary and related to a temporary loss of risk taking appetite. As for the legit vs non legit ML practitioners, I don't see how the situation is going to improve any time soon. Because it's not just random engineers fooling themselves by saying they are great at ML, there are entire companies new and old where there has been an increasing focus/pressure to sell AI powered $XYZ snake oil. They don't care who they hire, and most likely they don't even know who/how to hire when it comes to ML/AI, because the whole process right from the top is let's sell it first and wing it later
- joshvm 6y ago
  Also these hiring freezes may be due to covid, not loss of hope in ML. Google's AI residency program seems to be delayed but still going ahead, almost all the others were cancelled due to covid logistics (to my knowledge, at least FAIR, MS and Uber). That doesn't detract from the article's point though - lots more interest in those programs than there are places.
  - calebkaiser 6y ago
    I'd also add that the cuts to ML teams seem to reflect more of a freeze on R&D in general--totally normal in a downturn--and that ML just happens to be a feature in a lot of R&D at this point. Most of these companies have substantial investments in ML that aren't research-focused, they're powering their flagship products. At this point, ETA prediction isn't a speculative experiment for Uber or Google Maps, just like RankBrain, Smart Compose, spam filtering, etc.
curiousgal 6y ago
Exactly why as a grad student I have decided to go into Quantative Finance instead of Data Science. Literally anyone can become a data scientist, which is not a bad thing by all means. But for me, it makes it hard to justify me going to an elite French school and dealing with theoretical stuff (that will not be useful on a job, think VC dimension and all) for over a year.
- proverbialbunny 6y ago
  Good luck! I started with quant research and was successful at it before moving over to data science. It's a lot of fun! The technical challenges are more difficult too. Part of it is the area. I'm in the SF/Bay Area. The other part is schooling as the industry is a bit more ivory. I got my first tech job when I was 17.
moandcompany 6y ago
As with data science, the number of people that say they do "data science" and "machine learning" are in far greater supply than the number of opportunities. Likewise, the number of people that are able to apply their domain knowledge to actually help their company's businesses/product or problems will always have work. I've often used an analogy comparing this to cooking. Just because people say they know how to "cook" a dish, doesn't mean they'll be able to function or be effective in a restaurant or other type of business context kitchen. Yes, there's an element of this domain knowledge required, which are table stakes, but often the factors that distinguish success have more to do with understanding the context of the work and the surrounding business. To update this for recent fads, learning to make sourdough bread does not mean one has learned how to run a bakery. If you're able to be a pure researcher, or pure specialist, you're very lucky and working somewhere whey they can afford to have deep specializations. The plethora of emerging tools and practices related to AI/ML "workflow" are related to the collective realization that training ML models is just one piece of the puzzle: https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf https://papers.nips.cc/paper/5656-hidden-technical-debt-in-m... (See: Figure 1)
KKKKkkkk1 6y ago
So my perspective is that 2020:AI is as 2000:Internet. In 2000, the idea that all commerce is going to be on the Internet was obviously true, just as obviously true as the fact that all of the companies that IPOed at the time had no way of getting there (except for Amazon). AI one day is going to be the foundation of successful money-making products and services, but AlexNet and GPT-3 are not even stepping stones in that direction. Self-driving cars are a distraction like virtual worlds.
- proverbialbunny 6y ago
  It's entirely possible, but also AI is older than the internet, and Google was the first big ML company, gaining their success as early as '99. It seems like ML has a longer adoption curve to it, moving slower. Also, AI is anything cutting edge. The fill tool in Paint was once considered AI, so AI will continue on as long as we have new tech.
  - deleted 6y ago
    [deleted]
  - MauranKilom 6y ago
    I'm having a hard time seeing PageRank '99 being ML just because it uses a lot of data and is linear algebra at the core. It's one large eigenvalue problem.
    - hypmfd 6y ago
      It's just because you're having a hard time seeing that most ML is linear algebra at the core.
      - nl 6y ago
        No. I mean it's true that ML uses linear algebra. But the difference between ML and PageRank is that ML is Machine Learning. The learning part (involving storing and updating based on new data) is different to statically calculating Pagerank using Map/Reduce and applying it as a ranking function.
raverbashing 6y ago
> Hopefully the industry matures. I've bumped into data scientists who don't know how to do feature engineering as well as ones who can't grok ML. It makes the whole industry look bad. Agreed I suspect that for 9 in 10 ML "experts" in the field there are corresponding 9 in 10 ML "jobs" in the field that are just hiring for the buzzword and seeing what happens
- TomMarius 6y ago
  Have you ever interviewed applicants for software jobs? I think at least 9/10 applicants often have no idea how to code. They can write down some JavaScript, but don't know why.
scared2 6y ago
I'm interested to know more about your skill set. If you don't mind.
- scared2 6y ago
  Ah sorry. I think you already mentioned timeseries classification.
  - proverbialbunny 6y ago
    np. I've done quite a few out there things on previous projects. Some of the more extreme are: - I invented a new kind of ML for quant work (day trading bot) that ended up being quite successful. It's not published anywhere for hopefully obvious reasons. - I've written models that I've productionized to embedded. I also wrote a script that converts between languages, automating the process. - I reverse engineered google's page rank tech (my first DS job actually) and ended up getting higher accuracy on website classification than a team of experts manually classifying web pages could do.
    - tzekid 6y ago
      That sounds impressive! I'm a student who just started a real life project (basic Visual Computing). Any practical tips of what I should look out for / focus on in the future?
discobot 6y ago
Google will happily hire and pay batshit crazy salary to any capable engineer with ML background
abhgh 6y ago
I empathize. Been doing ML for a similar amount of time, and the amount of people who claim to know ML, but lack basics, abound. I don't think they are to blame (after all learning isn't bad, and everyone wants to capitalize on a market); we need to adapt to the fact that the definition of ML practitioner has considerably broadened: from someone who knows only to use vgg16 from PyTorch, without understanding the optimization, loss, alternative algorithms etc to someone with a much broader and deeper view of the area. Problems arise when people expect candidates of the latter type but instead end up running into a steady stream of candidates of the former type. Hiring Managers probably need to get used to the fact the term "ML practitioner" alone doesn't mean much anymore, and they need to set expectations accordingly.
bryanrasmussen 6y ago
>As someone who does ML for a living this hits. When switching companies in 2019 for the first time I had a hard time despite having 10+ years experience. Linkedin maxed out saying over 200 people had applied to every job I was looking at. sure you're not just getting old? My experience is that once I got over 10+ years in experience in development people generally didn't want to hire me full-time, only consulting. >Hopefully the industry matures. I've bumped into data scientists who don't know how to do feature engineering as well as ones who can't grok ML. It makes the whole industry look bad This is what happens when a tech industry matures! Before it's mature and lucrative only the dedicated are into it, once it has matured there are layers of skillfulness and somewhere at the bottom the totally unskilled trying to get something. I suppose you will be receiving an ML take home project for your next job shift.
- proverbialbunny 6y ago
  >sure you're not just getting old? My experience is that once I got over 10+ years in experience in development people generally didn't want to hire me full-time, only consulting. I don't know. I don't know how to break into consulting but I want to do it. I've been through three IPOs in the last 11 years, and we didn't really hire consultants (except people out in the field to collect data sometimes, but that was contract employees), so I'm a bit out of the loop. I did everything wearing multiple hats, even going so far as to productionizing models I've made onto embedded, so there has been little reason to hire outside help.
  - proverbialbunny 6y ago
    I must have been half way awake. Not three IPOs. I went through one of those. I went through three acquisitions.
The_rationalist 6y ago
Interesting: For time series forecasting do you use the state of the art AKA GRU-ODE-bayes ? https://paperswithcode.com/sota/multivariate-time-series-forecasting-on-mimic https://paperswithcode.com/sota/multivariate-time-series-for... It seems to be an accuracy breakthrough in the field
screye 6y ago
Honestly, I got my foot in the door and now get to work on pretty cool ML stuff as a DS. I'm gonna stick it out for a few years (I like my job) and pivot to ML-PM right as there is a glut of technical people in the field. In my experience, product/program managers that understand ML are a very rare breed, and often end up being the bottleneck when it comes to taking an ML model from research to users. At the same time, becoming a PM too soon will be me being all talk no substance. Don't want that.
- proverbialbunny 6y ago
  Good luck! ^_^
softwaredoug 6y ago
There’s a real lack of good engineering in machine learning and too much emphasis on using bleeding edge research. There’s a lot of boring models and practices that get the job done for the 80%. That don’t require as much talent to do well. That scale, quick to develop, and can be implemented on prod with ease. But there’s a problem in tech culture of reaching first for the most sophisticated ML, not the most practical.
TrackerFF 6y ago
But keep in mind, the VAST majority of those people have only toyed around with ML libraries or frameworks on the most trivial data. There's A LOT of script kiddies right now, that just copy/paste the same tutorials, maybe even publish yet another "How to classify MNIST dataset with NN / SVM / Naive Bayes / etc." on medium or whatever, just to come off as authority. So while there's a lot more competition, the majority of these don't even meet the minimum standards to get called into interviews. If you have solid work experience to show for, then that alone should differentiate you from the the masses. Again, a lot of these people have NEVER worked with anything relevant, and can only show you a bunch of tutorials they've followed. IF lucky, they've actually taken on more exciting datasets from sources like Kaggle, and tried to come up with something. edit: Not to mention the data - I get extremely impressed when someone actually shows a project which involves them collecting data from the wild, wrangling it, and then doing meaningful work. Too many beginners only work on neat / ready datasets. You'd be surprised how many candidates get completely blindsided when asked to do some trivial tasks on pre-processing data.
- cirgue 6y ago
  I have been doing ML engineering professionally for the 6 years and have been involved in about one hiring round per year for the last 4. If you filter resumes down to “has at least 1 year of professional experience on a deployed ML product”, then you’re lucky to have one resume out of 100, and that’s after HR has had their crack at the applicant pool. We still have to hire people who don’t really have much actual professional ML experience (which imo is fine). We invite literally anyone who has shipped ML code professionally for interviews, and we get ghosted a quarter of the time. It’s not like we pay below market either, it’s just that the demographic of people who already know what they’re doing can still afford to be really picky.
- bulldoa 6y ago
  > minimum standards to get called into interviews. I am a bit curious on what is the minimum standard? It does feel to me (an outsider that took 1 class of ML in college) that you need at least Master in ML to get intuition on the probabilistic and linear algebra theory behind ML concepts.
indymike 6y ago
There's always a land rush of pretenders in any new sector of tech. Not sure where I heard it, but there's a definition of expert that I think a lot of people believe: "an expert is a person who knows one more thing than everyone else." It takes a while for experience to smoke the paper-thin experts out.

huac 6y ago

# of people who want to do "machine learning" > # of ML jobs > # of people good at ML

proverbialbunny 6y ago
There should be a law that require all ML and DS classes, books, and bootcamps start by manually labeling data.
kqr 6y ago
Yes, I was going to say. Taking courses in "ML" is a great way to get to write a hyped-up term on your resume. Actually understanding and being able to write code for statistical learning is something else entirely. Edit: On the other hand, I think Carmack observed that as long as you get the sign right, you'll get something out of it, even if it won't learn very quickly. So maybe speed of implementation and learning is how skilled people will differentiate themselves?
- proverbialbunny 6y ago
  Research tasks take a long time. If you're going off of speed, it's no longer data science it's software engineering. (Though, to be fair, with all the new software engineers becoming data scientists, the data science title is becoming more like software engineering.) imho metaphysics and metalearning might be a better way to differentiate.
  - kqr 6y ago
    Right. There's room for research, and then there's room for commoditising existing research. I think the latter is where we currently have the most to gain from expanding our efforts in, and this was the context in which I made my comment. I see now that I made that very unclear. Sorry!
    - deleted 6y ago
      [deleted]
    - proverbialbunny 6y ago
      No need to be sorry. I'm equally to blame for my own lack of communication skills. ^_^

flak48 6y ago

As for Uber, the AI researchers that were laid off were not from the self driving division. Google's freeze as far as I know is for all engineers ( only backfilling attrition is permitted), not limited to ML. I personally feel this slump in the ML job market is only temporary and related to a temporary loss of risk taking appetite. As for the legit vs non legit ML practitioners, I don't see how the situation is going to improve any time soon. Because it's not just random engineers fooling themselves by saying they are great at ML, there are entire companies new and old where there has been an increasing focus/pressure to sell AI powered $XYZ snake oil. They don't care who they hire, and most likely they don't even know who/how to hire when it comes to ML/AI, because the whole process right from the top is let's sell it first and wing it later

joshvm 6y ago
Also these hiring freezes may be due to covid, not loss of hope in ML. Google's AI residency program seems to be delayed but still going ahead, almost all the others were cancelled due to covid logistics (to my knowledge, at least FAIR, MS and Uber). That doesn't detract from the article's point though - lots more interest in those programs than there are places.
- calebkaiser 6y ago
  I'd also add that the cuts to ML teams seem to reflect more of a freeze on R&D in general--totally normal in a downturn--and that ML just happens to be a feature in a lot of R&D at this point. Most of these companies have substantial investments in ML that aren't research-focused, they're powering their flagship products. At this point, ETA prediction isn't a speculative experiment for Uber or Google Maps, just like RankBrain, Smart Compose, spam filtering, etc.

curiousgal 6y ago

Exactly why as a grad student I have decided to go into Quantative Finance instead of Data Science. Literally anyone can become a data scientist, which is not a bad thing by all means. But for me, it makes it hard to justify me going to an elite French school and dealing with theoretical stuff (that will not be useful on a job, think VC dimension and all) for over a year.

proverbialbunny 6y ago
Good luck! I started with quant research and was successful at it before moving over to data science. It's a lot of fun! The technical challenges are more difficult too. Part of it is the area. I'm in the SF/Bay Area. The other part is schooling as the industry is a bit more ivory. I got my first tech job when I was 17.

moandcompany 6y ago

As with data science, the number of people that say they do "data science" and "machine learning" are in far greater supply than the number of opportunities. Likewise, the number of people that are able to apply their domain knowledge to actually help their company's businesses/product or problems will always have work. I've often used an analogy comparing this to cooking. Just because people say they know how to "cook" a dish, doesn't mean they'll be able to function or be effective in a restaurant or other type of business context kitchen. Yes, there's an element of this domain knowledge required, which are table stakes, but often the factors that distinguish success have more to do with understanding the context of the work and the surrounding business. To update this for recent fads, learning to make sourdough bread does not mean one has learned how to run a bakery. If you're able to be a pure researcher, or pure specialist, you're very lucky and working somewhere whey they can afford to have deep specializations. The plethora of emerging tools and practices related to AI/ML "workflow" are related to the collective realization that training ML models is just one piece of the puzzle: https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf https://papers.nips.cc/paper/5656-hidden-technical-debt-in-m... (See: Figure 1)

KKKKkkkk1 6y ago

So my perspective is that 2020:AI is as 2000:Internet. In 2000, the idea that all commerce is going to be on the Internet was obviously true, just as obviously true as the fact that all of the companies that IPOed at the time had no way of getting there (except for Amazon). AI one day is going to be the foundation of successful money-making products and services, but AlexNet and GPT-3 are not even stepping stones in that direction. Self-driving cars are a distraction like virtual worlds.

proverbialbunny 6y ago
It's entirely possible, but also AI is older than the internet, and Google was the first big ML company, gaining their success as early as '99. It seems like ML has a longer adoption curve to it, moving slower. Also, AI is anything cutting edge. The fill tool in Paint was once considered AI, so AI will continue on as long as we have new tech.
- deleted 6y ago
  [deleted]
- MauranKilom 6y ago
  I'm having a hard time seeing PageRank '99 being ML just because it uses a lot of data and is linear algebra at the core. It's one large eigenvalue problem.
  - hypmfd 6y ago
    It's just because you're having a hard time seeing that most ML is linear algebra at the core.
    - nl 6y ago
      No. I mean it's true that ML uses linear algebra. But the difference between ML and PageRank is that ML is Machine Learning. The learning part (involving storing and updating based on new data) is different to statically calculating Pagerank using Map/Reduce and applying it as a ranking function.

raverbashing 6y ago

> Hopefully the industry matures. I've bumped into data scientists who don't know how to do feature engineering as well as ones who can't grok ML. It makes the whole industry look bad. Agreed I suspect that for 9 in 10 ML "experts" in the field there are corresponding 9 in 10 ML "jobs" in the field that are just hiring for the buzzword and seeing what happens

TomMarius 6y ago
Have you ever interviewed applicants for software jobs? I think at least 9/10 applicants often have no idea how to code. They can write down some JavaScript, but don't know why.

scared2 6y ago

I'm interested to know more about your skill set. If you don't mind.

scared2 6y ago
Ah sorry. I think you already mentioned timeseries classification.
- proverbialbunny 6y ago
  np. I've done quite a few out there things on previous projects. Some of the more extreme are: - I invented a new kind of ML for quant work (day trading bot) that ended up being quite successful. It's not published anywhere for hopefully obvious reasons. - I've written models that I've productionized to embedded. I also wrote a script that converts between languages, automating the process. - I reverse engineered google's page rank tech (my first DS job actually) and ended up getting higher accuracy on website classification than a team of experts manually classifying web pages could do.
  - tzekid 6y ago
    That sounds impressive! I'm a student who just started a real life project (basic Visual Computing). Any practical tips of what I should look out for / focus on in the future?

discobot 6y ago

Google will happily hire and pay batshit crazy salary to any capable engineer with ML background

abhgh 6y ago

I empathize. Been doing ML for a similar amount of time, and the amount of people who claim to know ML, but lack basics, abound. I don't think they are to blame (after all learning isn't bad, and everyone wants to capitalize on a market); we need to adapt to the fact that the definition of ML practitioner has considerably broadened: from someone who knows only to use vgg16 from PyTorch, without understanding the optimization, loss, alternative algorithms etc to someone with a much broader and deeper view of the area. Problems arise when people expect candidates of the latter type but instead end up running into a steady stream of candidates of the former type. Hiring Managers probably need to get used to the fact the term "ML practitioner" alone doesn't mean much anymore, and they need to set expectations accordingly.

bryanrasmussen 6y ago

>As someone who does ML for a living this hits. When switching companies in 2019 for the first time I had a hard time despite having 10+ years experience. Linkedin maxed out saying over 200 people had applied to every job I was looking at. sure you're not just getting old? My experience is that once I got over 10+ years in experience in development people generally didn't want to hire me full-time, only consulting. >Hopefully the industry matures. I've bumped into data scientists who don't know how to do feature engineering as well as ones who can't grok ML. It makes the whole industry look bad This is what happens when a tech industry matures! Before it's mature and lucrative only the dedicated are into it, once it has matured there are layers of skillfulness and somewhere at the bottom the totally unskilled trying to get something. I suppose you will be receiving an ML take home project for your next job shift.

proverbialbunny 6y ago
>sure you're not just getting old? My experience is that once I got over 10+ years in experience in development people generally didn't want to hire me full-time, only consulting. I don't know. I don't know how to break into consulting but I want to do it. I've been through three IPOs in the last 11 years, and we didn't really hire consultants (except people out in the field to collect data sometimes, but that was contract employees), so I'm a bit out of the loop. I did everything wearing multiple hats, even going so far as to productionizing models I've made onto embedded, so there has been little reason to hire outside help.
- proverbialbunny 6y ago
  I must have been half way awake. Not three IPOs. I went through one of those. I went through three acquisitions.

The_rationalist 6y ago

Interesting: For time series forecasting do you use the state of the art AKA GRU-ODE-bayes ? https://paperswithcode.com/sota/multivariate-time-series-forecasting-on-mimic https://paperswithcode.com/sota/multivariate-time-series-for... It seems to be an accuracy breakthrough in the field

screye 6y ago

Honestly, I got my foot in the door and now get to work on pretty cool ML stuff as a DS. I'm gonna stick it out for a few years (I like my job) and pivot to ML-PM right as there is a glut of technical people in the field. In my experience, product/program managers that understand ML are a very rare breed, and often end up being the bottleneck when it comes to taking an ML model from research to users. At the same time, becoming a PM too soon will be me being all talk no substance. Don't want that.

proverbialbunny 6y ago
Good luck! ^_^

softwaredoug 6y ago

There’s a real lack of good engineering in machine learning and too much emphasis on using bleeding edge research. There’s a lot of boring models and practices that get the job done for the 80%. That don’t require as much talent to do well. That scale, quick to develop, and can be implemented on prod with ease. But there’s a problem in tech culture of reaching first for the most sophisticated ML, not the most practical.

TrackerFF 6y ago

But keep in mind, the VAST majority of those people have only toyed around with ML libraries or frameworks on the most trivial data. There's A LOT of script kiddies right now, that just copy/paste the same tutorials, maybe even publish yet another "How to classify MNIST dataset with NN / SVM / Naive Bayes / etc." on medium or whatever, just to come off as authority. So while there's a lot more competition, the majority of these don't even meet the minimum standards to get called into interviews. If you have solid work experience to show for, then that alone should differentiate you from the the masses. Again, a lot of these people have NEVER worked with anything relevant, and can only show you a bunch of tutorials they've followed. IF lucky, they've actually taken on more exciting datasets from sources like Kaggle, and tried to come up with something. edit: Not to mention the data - I get extremely impressed when someone actually shows a project which involves them collecting data from the wild, wrangling it, and then doing meaningful work. Too many beginners only work on neat / ready datasets. You'd be surprised how many candidates get completely blindsided when asked to do some trivial tasks on pre-processing data.

cirgue 6y ago
I have been doing ML engineering professionally for the 6 years and have been involved in about one hiring round per year for the last 4. If you filter resumes down to “has at least 1 year of professional experience on a deployed ML product”, then you’re lucky to have one resume out of 100, and that’s after HR has had their crack at the applicant pool. We still have to hire people who don’t really have much actual professional ML experience (which imo is fine). We invite literally anyone who has shipped ML code professionally for interviews, and we get ghosted a quarter of the time. It’s not like we pay below market either, it’s just that the demographic of people who already know what they’re doing can still afford to be really picky.
bulldoa 6y ago
> minimum standards to get called into interviews. I am a bit curious on what is the minimum standard? It does feel to me (an outsider that took 1 class of ML in college) that you need at least Master in ML to get intuition on the probabilistic and linear algebra theory behind ML concepts.

indymike 6y ago

There's always a land rush of pretenders in any new sector of tech. Not sure where I heard it, but there's a definition of expert that I think a lot of people believe: "an expert is a person who knows one more thing than everyone else." It takes a while for experience to smoke the paper-thin experts out.