5 ms·
From the other side of the table, the machine learning candidate pool is also a clown show right now. I did some hiring for a very real machine learning (AI if
by Aurornis 3y ago
From the other side of the table, the machine learning candidate pool is also a clown show right now.
I did some hiring for a very real machine learning (AI if you want to call it that) initiative that started even before the LLM explosion. The number of candidates applying with claimed ML/AI experience who haven’t done anything more than follow online tutorials is wild. This was at a company that had a good reputation among tech people and paid above average, so we got a lot of candidates hoping to talk their way into ML jobs after completing some basic courses online.
The weirdest trend was all of the people who had done large AI projects on things that didn’t need AI at all. We had people bragging about spending a year or more trying to get an AI model to do simple tasks that were easily solved deterministically with simple math, for example. There was a lot of AI-ification for the sake of using AI.
It feels similar to when everyone with a Raspberry Pi started claiming embedded expertise or when people who worked with analytics started branding themselves as Big Data experts.
- jvalencia 3y ago100% this. ML has been very hyped for a while now and having it was seen as a badge for the company. To be fair, ML is not also something that was historically central to a degree, so many people wanting to get into AI, even good engineers, did not have the background in it. This too is changing though, but the hype and the lack of an experienced pool doesn't help.
- mikrl 3y agoI don’t think the background is really that important tbh. From physics I have a good theoretical grounding in how ML works (optimizing a cost function over a high dimensional manifold to reconstruct a probability distribution, then using the distribution for some task) but I personally find actually ‘doing ML’ to be rather dull.
- hashtag-til 3y ago> The weirdest trend was all of the people who had done large AI projects on things that didn’t need AI at all. I can relate to this a lot. In my company many things you can sell as "AI" can really be solved with traditional data processing.
- MaxBarraclough 3y ago> We had people bragging about spending a year or more trying to get an AI model to do simple tasks that were easily solved deterministically with simple math, for example. Fad-chasing often leads to silly technical decisions. Same thing happened with blockchains when they were at the peak of the famous hype cycle. [0] [0] https://en.wikipedia.org/wiki/Gartner_hype_cycle https://en.wikipedia.org/wiki/Gartner_hype_cycle
- sillysaurusx 3y ago> The weirdest trend was all of the people who had done large AI projects on things that didn’t need AI at all. This is how people get experience with ML though. I don’t think that’s a bad thing. It sounds like you’re looking for a candidate with current ML experience. But I’ve seen so many people go from zero knowledge to capable devs that this seems like a mistake. You’ll end up overpaying. Just try to find someone with a burning ambition to learn. That seems like the key to get someone capable in the long run. If they point out something beyond Kaggle that makes you think, pay attention to that feeling — it means they’re in it for more than the money.
- discmonkey 3y agoI have to agree. Especially given the very real possibility that your ML project won't be cutting edge research grade. At that point someone who doesn't have bias and is willing to search for a reasonable looking approximation to the problem and try a canned solution may actually be an optimal candidate.
- 0cf8612b2e1e 3y agoConsidering the number of problems that could be plugged into a random forest with good results, data proficiency seems more important than strong ML experience.
- uoaei 3y agoDepends heavily on the application once you get to more specialized domains. I wish there was an easier way to label roles differently based on when you just need to throw X or Y model at some chunk of data and when more specialized modeling is required. Previously it was roughly delineated by "data science" vs "ML" roles but the recent AI thing has really messed with this.
- deleted 3y ago[deleted]
- htrp 3y ago>Just try to find someone with a burning ambition to learn. That seems like the key to get someone capable in the long run. If they point out something beyond Kaggle that makes you think, pay attention to that feeling — it means they’re in it for more than the money. If you're teaching them, you shouldn't be paying them at the AI expert rate.
- softwaredoug 3y agoReminds me of the software hiring market during the dotcom boom. I think the hype on the field and the shitty candidate pool go hand in hand. The shitty candidate pool will groupthink / cargo cult the space without much critical thinking. The groupthink / hype will cause people to jump into the field who don't have any business being in the field.
- TrackerFF 3y ago"The weirdest trend was all of the people who had done large AI projects on things that didn’t need AI at all. We had people bragging about spending a year or more trying to get an AI model to do simple tasks that were easily solved deterministically with simple math, for example. There was a lot of AI-ification for the sake of using AI." I've seen two variants of this 1) People that have worked for traditional (as in non-tech) companies, where there's been a huge push for digitalization and "AI". These things come from the very top, and you don't really have much say. I've been there myself. The upper echelon wants "AI" so that they can tick off boxes to the board of directors. With these folks, its all about managing expectations - but frankly, they don't care if you implement a simple regression model, or spend a fortune on overkill models. The most important part is that you've brought "AI" to the company. 2) The people that want to pad their resumes. There's no need, no push, but no-one is stopping you. You can add "designed and implemented AI products to the business operation blablabla" to your CV. These days, I've seen and experienced 1) an awful lot. It's all about keeping up with the joneses.
- haltist 3y agoWhat's hard about AI that requires special expertise? In many ways it is much simpler than regular software engineering because the conceptual landscape in AI is much simpler. Every AI framework offers the same conceptual primitives and even deployment targets whereas most web frameworks have entirely different conceptions of something as simple as MVC so knowing one framework isn't very useful for learning and understanding another one but if you know how to use PyTorch then you can very easily transfer that knowledge to another framework like Tensorflow or Jax. It should be possible for a competent software engineer to get up to speed in AI in less than 6 months and much of that time can be on the job itself.
- Jensson 3y agoAI is much harder if you need competitive results, and if you don't need competitive results you don't need to hire a dedicated AI person. Just feed data into some library which is typical software engineering and doesn't have anything to do with AI.
- haltist 3y agoThe only metric that matters for a business is whatever helps their bottom line. No one really cares about competitive results if they can just fine tune some open source model on their own data set and get good business outcomes. So if there is good data and access to compute infrastructure to train and fine tune some open source model then the only obstruction to figuring out if AI works for the business or not is just a matter of setting up the training and deployment pipeline. That requires some expertise but that can be learned on the job as well or from any number of freely available tutorials. I don't think AI is hard to learn. The fundamentals are extremely simple and a competent software engineer can learn all the required concepts in a few months. It's easier if you already have a background in mathematics but not required. If you can write software then you can learn how to write differentiable tensor programs with any of the AI frameworks.
- Jensson 3y agoYes, and those businesses don't need to hire an AI person. This topic is AI research jobs, not for people who sometimes has to call an ML library once in a while in their normal software job. Edit: You asked what it is about these jobs that requires expertise. I answered: it requires expertise to create competitive models. So companies that need competitive models requires expertise.
- MattGaiser 3y agoHasn't that just been the tech market ever since software dev appeared on the lists of best paid low stress jobs?
- notsurenymore 3y agoI think it’s not even about low stress, but low barrier to entry. There are plenty of things I’d rather be doing than software development (in fact I never planned on going into this field professionally), but I just can’t. I’m also not surprised by the “The number of candidates applying with claimed ML/AI experience who haven’t done anything more than follow online tutorials is wild”. Just go look at any Ask HN thread about “how do I get into ML/AI”. This is pretty typical advice. Hell it’s pretty typical advice given to people asking how to get into any domain. Now sure we’ll how it works outside of bog standard web development though.
- UncleOxidant 3y ago> The number of candidates applying with claimed ML/AI experience who haven’t done anything more than follow online tutorials is wild. Sure, I get this, but I suspect that the number of people who have actual ML/AI experience is pretty small given that the field is nascent. If you really want to hire people to do this kind of work you're going to need to go with people who have done the online tutorials, read the papers, have an interest, etc. Yes, once in a while you're going to find someone who has actual solid ML experience, but you're also going to have to pay them a lot. That's just how things work in a field like this that's growing rapidly.
- archero 3y agoI'm genuinely curious, what is your expectation of candidates looking to get into ML at the entry level? You seem to look down on those who have 1) learned from online courses or 2) used AI on tasks that don't require it Isn't this a bit contradictory? Or you expect candidates to have found a completely novel usecase for AI on their own? I understand that most ML roles prefer a master's degree or PhD, but from my experience most of the master's degrees in ML being offered right now were spawned from all the "AI hype". That is to say, they may not include a lot of core ML courses and probably are not a significantly better signal of a candidate's qualifications than some of the good online courses out there. So what does that leave, only those with a PhD? I think it's unreasonable that someone should need that many years of formal education to get an entry level position. Maybe I'm missing something, but I'm really wondering, what do you expect from candidates? I think a few years of professional software engineering experience with some demonstrated interest in AI via online courses and personal projects should be enough.
- michaelt 3y agoIt sounds like Aurornis was not, in fact, trying to hire people at the entry level. Most companies doing regular, non-ML development hire a mix of junior and experienced engineers, with the latter providing code reviews, mentorship and architectural advice alongside normal programming duties. It's understandable that someone kicking off a new ML project would hope to get the experienced hires on board first. But there are a lot more junior people on the market than senior people right now - as is the nature of a fast growing market.
- archero 3y agoOk, that makes sense. I agree, it's problematic that there are so many more juniors than seniors in the industry right now. I feel like many juniors are being left without mentorship, and then it becomes much harder for them to grow and eventually become qualified for senior roles. So that could help explain why many candidates seem so weak, alongside with all the recent hype. I guess eventually the market will cool off and the hype will die down since this stuff seems to be cyclical, and the junior engineers who are determined enough to stick it out and seek out mentorship will be able to grow and become seniors. But it definitely seems like the number of seniors is a bottleneck for talent across the industry.
- lawlessone 3y ago>We had people bragging about spending a year or more trying to get an AI model to do simple tasks that were easily solved deterministically with simple math, for example. TBF theres whole companies doing this. It's a good way to learn too, as you have existing solutions to compare yourself too.
- BeetleB 3y agoA former colleague of mine (SW guy) took Andrew's Coursera course, downloaded some Kaggle sets, fiddled with them, and put his Jupyter notebooks online. He learned the lingo of deep learning (no experience in them, though). Then he hit the interview circuit. Got a senior ML position in a well known Fortune 500 company. Senior enough that he sets his goals - no one gives him work to do. He just goes around asking for data and does analyses. When he left our team he told me "Now that I have this opportunity, I can actually really learn ML instead of faking it." If you think that's bad, you should hear the stories he tells at that company. Since senior leadership knows nothing about ML practices, practices are sloppy to get impressive numbers. Things like reporting quality based on performance on training data. And when going from a 3% to a 6% prediction success rate, they boast about "doubling the performance". He eventually left for another company because it was harder to compete against bigger charlatans than he was.
- rg111 3y ago> "took Andrew's Coursera course" If he really did take those and did all the assignments himself and understood all the concepts, that still puts him at least in the 95th percentile among ML job seekers.
- BeetleB 3y agoI don't disagree. Still, would you give such a person a senior ML role and let him do whatever work he wants?
- rg111 3y agoNo. Absolutely not. But it shows intent for junior roles. People have no idea how many people just put "AI enthusiast" on LinkedIn profile and start to seek ML roles. I have had people only with Excel skills apply to ML roles. And, true thing is every company’s ML tooling/stack/procedure is wildly different. One has to learn on the job.
- CamperBob2 3y agoIf you think that's bad (Shrug) I don't. Hustle gets rewarded, as usual. Sounds like he contributed at least as much value as he captured.
- i1856511 3y agoI have a background in computational linguistics from a good university, and then I got sidetracked by life for the last decade. What real experience did you look for that was a good signal?
- klyrs 3y ago> The weirdest trend was all of the people who had done large AI projects on things that didn’t need AI at all. Yeah this is a major phenomenon. Everybody's putting "ai" stickers on everything. So the job market screams "we need ai experts!" in numbers far exceeding the supply of ai experts, because it was a tiny niche until a couple years ago. Industry asks for garbage, industry gets garbage.
- paxys 3y agoThis isn't unique to AI. Post any programming job and something like 50-80% of applicants with seemingly perfect resumes won't be able to pass a FizzBuzz test.
- screye 3y agoIt is getting doubly weird with the LLM/Diffusion explosion over the last year. The applied research ML role has evolved from being a computational math role to a Pytorch role to a 'informed throw things at the wall' role. I went from reading textbooks (Murphy, Ian Goodfellow, Bishop) to watching curated NIPS talks to reading Axriv papers to literally trawling random discord channels and subreddits to get a few month leg up on anyone in research. Recently, a paper formally cited /r/localllama for their core idea. > follow online tutorials The Open Source movement moves so quickly, that running someone's collab notebook is the way to be at the cutting edge of research. The entire agents, task planning and meta-prompting field was invented in random forums. ________________ This is mostly relevant to the NLP/Vision world......but take a break for 1-2 years, and your entire skill set is obsolete.
- breakds 3y agoI had two successful hires who just graduate from college, with no machine learning experience (major in accounting and civil engineering). With 3 months training by working on real world projects, they become quite fine machine learning engineers. You probably do not need AI experts if you just need good machine learning engineer to build models to solve problems.
- akomtu 3y agoWhat motivates you and other ML researchers to do this work? What's the end goal and why do you want it?
- _ea1k 3y agoSo having no experience is bad, but also going out of their way to get experience is also bad? Isn't this presenting a bit of a no-win scenarios? Then again, that seems to be common with the job market.
- __loam 3y agoCorporations want trained people but don't want to invest in training them.
- benreesman 3y agoI’ve done heavy infra, serious model and feature engineering, or both on all of FB Ads and IG organic before 2019, did a startup on attention/transformer models at the beginning of 2019, and worked in extreme multi-class settings in bioinformatics this year. And out of all the nightmare “we have so many qualified candidates we can’t even do price discovery” conversations in 2023, the ML ones have been the worst. If you’re running a serious shop that isn’t screwing around and you’re having trouble finding tenured pros who aren’t screwing around, email me! :)
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- lakevieew 3y agoThis comment describes a real problem for senior engineers who want to explore a new domain. It is impractical for someone with 10+ years of experience to work as an entry level engineer. What other options exist besides completing online course to get experience in that domain? This is not specific to ML/AI roles. The same problem applies to anyone who wants to explore any of these domains - SRE, DataEng, Backend, Frontend. Personally, I am a backend engineer who wants to get into ML Infra roles. My current plan is to do these online courses, and hopefully transfer internally to a team working in this area. Only after having real industry experience in this area, look for more opportunities elsewhere. I am genuinely curious if anyone has better ideas for people in my situation.