5 ms·

I'm a DS with ~10 yoe mostly in classical ML and I've been wondering how much/how fast I need to pick up "high level" LLM skills. I've seen the way the winds ar

by extr 3y ago

I'm a DS with ~10 yoe mostly in classical ML and I've been wondering how much/how fast I need to pick up "high level" LLM skills. I've seen the way the winds are blowing wrt to the DS job title and so have been moving towards more of engineering roles. But it seems like every job posting right now for ML engineering positions is asking for "pytorch, DL, LLM" experience, even for companies where getting into the nuts and bolts of inference would not really be a core value prop. Like I really find it hard to believe that financial/insurance companies (my primary domain) are building and fine tuning their own models for chat/RAG applications or whatever (or if they are, should they be?). I haven't really seen huge value add from even the most sophisticated RAG setups yet. I use LLMs every day for code assistance so I'm not totally naive to the whole thing, but I seriously wonder what direction to take my career, exactly. Should I be going all-in on learning CUDA? Is knowing how to wire up a few API calls enough for "soft" applications? Should I in fact be keeping up with the latest RAG techniques (which seem to change every time a new foundational model gets released)? Should I just stay out of the whole thing and double down on classical ML applications in my domain?

boredemployee 3y ago
>> Should I in fact be keeping up with the latest RAG techniques (which seem to change every time a new foundational model gets released)? >> Should I just stay out of the whole thing and double down on classical ML applications in my domain? Thank you. Both questions are what I'm wondering right now.
- extr 3y ago
  If you want my personal opinion, I think the hype around RAG techniques is mostly a fad. I don't see how it becomes a super broad high value technique that your average DS or ML Engineer is getting paid big bucks to know. It's also in danger of becoming totally obsolete with further LLM advances (Claude 3 already really good at needle in the haystack tasks). Not something worth spending a ton of time on unless you're basically already knee deep in that kind of problem space. I think the only reason it's trendy right now is because LLMs are hammers looking for nails and semantic Q&A is one of the few identified nails. I'm pursuing of a strategy of: * Doubling down on domain knowledge. Even if coding becomes completely automated, there will always be value in knowing some particular corner of an industry very well. For me, that's insurance. This includes knowing how classical ML techniques can be applied in my industry (LLMs are not replacing xgboost any time soon). * Keeping tabs on bleeding edge LLM capabilities, just to have a sense of what's possible. However, staying out of highly technical LLM implementation details like CUDA, inference, batching, etc. Perhaps fine tuning will be valuable but probably I would focus on understanding "fine tuning as a service" offerings rather than rolling my own. * Investing in knowledge of infrastructure/orchestration of LLM services in a way that doesn't tie me to any particular problem space. It's hard to predict right now what kinds of new products LLMs might enable. But I think there will be value in being able to translate business problems into a high level architecture for an LLM-enabled app, even if that just means wiring together existing service offerings.
  - boredemployee 3y ago
    Yes. I agree with everything you said, and "infrastructure/orchestration" in the LLM space, seems to be what some ppl are already calling as "LLMops". Thanks for your input
  - philomath_mn 3y ago
    > It's also in danger of becoming totally obsolete with further LLM advances (Claude 3 already really good at needle in the haystack tasks) Maybe, but for every haystack that an LLM can search, RAG could search an even larger one. I don't think RAG is a killer app but I think tools that help LLMs find the right context will continue to be useful. > Doubling down on domain knowledge I'm a dev, but this is my main strategy right now. I am digging deep into my corner of fintech and hoping that will stave off some of the disruption I see coming for tech. Personally, I don't see why an AI couldn't also become a domain expert in my niche, but it should buy me some time and potentially align me with some powerful stakeholders (portfolio managers just need to convince the big money that a human touch is good and I will still be business).
  - DecentRecruiter 3y ago
    Excellent advice. Question: in what ways does insurance domain knowledge set you apart? I understand certain industries benefit from Engineers with domain experience (e.g. Healthtech companies needing HIPAA knowledge). I'm curious as to what particular skills insurance companies need from Engineers?
    - extr 3y ago
      For pure SWE I don't think I have a great answer for you, but on the ML/DE side there are a lot of industry-specific problems, especially on the P&C side, it helps to have context and experience with: * Rating (a deep, classical, tabular ML problem) is still catching up to SOTA techniques in many insurance companies + uneven regulatory environment makes it tricky to navigate. * Marketing is another classical ML problem, the goal is to optimize ad/lead broker spend/conversion funnel. * CS of course has huge opportunity with LLMs. * Automated claim fraud detection is a very difficult problem amenable to LLMs/MM DL models. Lots of "pivot to AI" going on here. And there are others of course. The main thing is that insurance companies have limits on product innovation - policy terms are in many cases mandated by law, so the main differentiator is operational efficiency and exellency. Solutions that move the needle on big revenue/cost levers are very valuable and can be applied widely.
avereveard 3y ago
> I seriously wonder what direction to take my career, exactly. same. future looks very uncertain. on one hand, the new figure of "ai-assisted x" where x can range from developer to copywriter to a whealth of other primarily creative job will replace the non ai assisted variants due of the tangible increase in productivity on the other hand the reduction in these cost possibly means 5x less capital requirement to build a startup, whose primary cost today is personell, at least in their early stages. wether will that increase in company creation be enough to raise the demand for personell to a level where the society is stable I don't really know. the primary risk I see is not from career prospet but from the massive capital investment that is required to be in the club of model providers, these will be the new elite, with unparallel power over economies and entire states. if you think the robber baron period was harsh, wait until three companies are ingrained in every aspect of your life from bureoucracy to healthcare and having to have to argue with a self moderated model instead of a bureaucrat, and if that's not enough imagine that maybe you have a uncommon or suggestive name and the model won't believe you no matter what proof you might give it. imagine these model being part one way or another in the production of entartainment, generating a monoculture that is extremely prude and milquetoast. or in general the model being wrong about anything really, you cannot reason with them. my concern is that there's a lot that should be happening right now at a legislative level, not in term of stifling innovation, but let's say putting certain role outside of the sole control of AI, which it isn't, and if we wait to be reactive about it, it's gonna end up like it did for self drivin: blood on the road and I cannot have my car acting as a taxi and earning for me during off hours because for safety reason only the tech owner can profit from it, yeah, right.
__loam 3y ago
I feel like there's a fuckload of hype and froth in whatever the LLM market is. Lots of money getting dumped into it and not a lot to show for it.
- emmender2 3y ago
  lot of investment being pumped into the shovel makers (nvidia), and the shovel sellers (csps). those panning for gold (app devs, startups etc) may or may not find it. remains to be seen and i remain skeptical.
  - iknownthing 3y ago
    Yeah seems like more of a shovel-rush than a gold-rush
    - extr 3y ago
      That's a good way of putting it. In my experience so far everyone is very excited to build LLM-enabled apps but outside of some customer service deployments (mostly at the megascale enterprise level), I haven't seen many actually successful efforts reported. Even the customer service bots always end up roundly mocked on social media once someone figures out they're talking to an LLM and tells it to pretend it's a fairy princess.
shankr 3y ago
Yeah I was in the same boat coming from classical ML. I am just doing MLOps/DE these days. Although I have been down levelled to almost junior level after having 10+ years of experience. Struggling a bit with being treated like a junior again! Much less autonomy to pick my own task.
- extr 3y ago
  I'm going through a similar transition right now. I wouldn't say I've been downleveled but I left FAANG to go to a startup (pay cut) where I've been able to transition pretty fluidly from "Data Science" to "MLOps + some backend SWE + some infra + some actual ML Engineering (non-DL)". Aside from the pay cut (which was not the end of the world, think high 200s to low 200s), it's worked out very well, and I'm getting the experience that I think will allow me to rebrand successfully. As to whether that rebrand itself is successful, I don't know. It feels like the DS role is undergoing a maturation where different skillsets are being cleaved into different roles. I was always a more natural fit for the Ops side, I don't have the academic creds for a research-oriented position. But I'm not sure how much of ops means "LLM ops".
next_xibalba 3y ago
> I've seen the way the winds are blowing wrt to the DS job title Can you elaborate on this?
- extr 3y ago
  I mention this in another reply but tl;dr the DS role is maturing and getting cleaved into different specialist skillsets. Research, ML Engineering, MLOps, Data Engineering, Analytics are what I see as the main ones. In 2016 you could get hired as a Data Scientist knowing some SQL, XGBoost, and how to use git (I know I did). That's not really the case in 2024, you are expected to be an expert in at least one of those areas and job titles are starting to reflect the specialization. Companies that are still hiring for generic "Data Scientist" positions with scikit-learn in the job description are not places you want to work.
  - BOOSTERHIDROGEN 3y ago
    The questions is how deep should you go of each subset ?
Bnjoroge 3y ago
definitely shifting to more engineering-focused type of roles so at the very I'd work on swe/infra skills

by extr 3y ago

boredemployee 3y ago

>> Should I in fact be keeping up with the latest RAG techniques (which seem to change every time a new foundational model gets released)? >> Should I just stay out of the whole thing and double down on classical ML applications in my domain? Thank you. Both questions are what I'm wondering right now.

extr 3y ago
If you want my personal opinion, I think the hype around RAG techniques is mostly a fad. I don't see how it becomes a super broad high value technique that your average DS or ML Engineer is getting paid big bucks to know. It's also in danger of becoming totally obsolete with further LLM advances (Claude 3 already really good at needle in the haystack tasks). Not something worth spending a ton of time on unless you're basically already knee deep in that kind of problem space. I think the only reason it's trendy right now is because LLMs are hammers looking for nails and semantic Q&A is one of the few identified nails. I'm pursuing of a strategy of: * Doubling down on domain knowledge. Even if coding becomes completely automated, there will always be value in knowing some particular corner of an industry very well. For me, that's insurance. This includes knowing how classical ML techniques can be applied in my industry (LLMs are not replacing xgboost any time soon). * Keeping tabs on bleeding edge LLM capabilities, just to have a sense of what's possible. However, staying out of highly technical LLM implementation details like CUDA, inference, batching, etc. Perhaps fine tuning will be valuable but probably I would focus on understanding "fine tuning as a service" offerings rather than rolling my own. * Investing in knowledge of infrastructure/orchestration of LLM services in a way that doesn't tie me to any particular problem space. It's hard to predict right now what kinds of new products LLMs might enable. But I think there will be value in being able to translate business problems into a high level architecture for an LLM-enabled app, even if that just means wiring together existing service offerings.
- boredemployee 3y ago
  Yes. I agree with everything you said, and "infrastructure/orchestration" in the LLM space, seems to be what some ppl are already calling as "LLMops". Thanks for your input
- philomath_mn 3y ago
  > It's also in danger of becoming totally obsolete with further LLM advances (Claude 3 already really good at needle in the haystack tasks) Maybe, but for every haystack that an LLM can search, RAG could search an even larger one. I don't think RAG is a killer app but I think tools that help LLMs find the right context will continue to be useful. > Doubling down on domain knowledge I'm a dev, but this is my main strategy right now. I am digging deep into my corner of fintech and hoping that will stave off some of the disruption I see coming for tech. Personally, I don't see why an AI couldn't also become a domain expert in my niche, but it should buy me some time and potentially align me with some powerful stakeholders (portfolio managers just need to convince the big money that a human touch is good and I will still be business).
- DecentRecruiter 3y ago
  Excellent advice. Question: in what ways does insurance domain knowledge set you apart? I understand certain industries benefit from Engineers with domain experience (e.g. Healthtech companies needing HIPAA knowledge). I'm curious as to what particular skills insurance companies need from Engineers?
  - extr 3y ago
    For pure SWE I don't think I have a great answer for you, but on the ML/DE side there are a lot of industry-specific problems, especially on the P&C side, it helps to have context and experience with: * Rating (a deep, classical, tabular ML problem) is still catching up to SOTA techniques in many insurance companies + uneven regulatory environment makes it tricky to navigate. * Marketing is another classical ML problem, the goal is to optimize ad/lead broker spend/conversion funnel. * CS of course has huge opportunity with LLMs. * Automated claim fraud detection is a very difficult problem amenable to LLMs/MM DL models. Lots of "pivot to AI" going on here. And there are others of course. The main thing is that insurance companies have limits on product innovation - policy terms are in many cases mandated by law, so the main differentiator is operational efficiency and exellency. Solutions that move the needle on big revenue/cost levers are very valuable and can be applied widely.

avereveard 3y ago

> I seriously wonder what direction to take my career, exactly. same. future looks very uncertain. on one hand, the new figure of "ai-assisted x" where x can range from developer to copywriter to a whealth of other primarily creative job will replace the non ai assisted variants due of the tangible increase in productivity on the other hand the reduction in these cost possibly means 5x less capital requirement to build a startup, whose primary cost today is personell, at least in their early stages. wether will that increase in company creation be enough to raise the demand for personell to a level where the society is stable I don't really know. the primary risk I see is not from career prospet but from the massive capital investment that is required to be in the club of model providers, these will be the new elite, with unparallel power over economies and entire states. if you think the robber baron period was harsh, wait until three companies are ingrained in every aspect of your life from bureoucracy to healthcare and having to have to argue with a self moderated model instead of a bureaucrat, and if that's not enough imagine that maybe you have a uncommon or suggestive name and the model won't believe you no matter what proof you might give it. imagine these model being part one way or another in the production of entartainment, generating a monoculture that is extremely prude and milquetoast. or in general the model being wrong about anything really, you cannot reason with them. my concern is that there's a lot that should be happening right now at a legislative level, not in term of stifling innovation, but let's say putting certain role outside of the sole control of AI, which it isn't, and if we wait to be reactive about it, it's gonna end up like it did for self drivin: blood on the road and I cannot have my car acting as a taxi and earning for me during off hours because for safety reason only the tech owner can profit from it, yeah, right.

__loam 3y ago

I feel like there's a fuckload of hype and froth in whatever the LLM market is. Lots of money getting dumped into it and not a lot to show for it.

emmender2 3y ago
lot of investment being pumped into the shovel makers (nvidia), and the shovel sellers (csps). those panning for gold (app devs, startups etc) may or may not find it. remains to be seen and i remain skeptical.
- iknownthing 3y ago
  Yeah seems like more of a shovel-rush than a gold-rush
  - extr 3y ago
    That's a good way of putting it. In my experience so far everyone is very excited to build LLM-enabled apps but outside of some customer service deployments (mostly at the megascale enterprise level), I haven't seen many actually successful efforts reported. Even the customer service bots always end up roundly mocked on social media once someone figures out they're talking to an LLM and tells it to pretend it's a fairy princess.

shankr 3y ago

Yeah I was in the same boat coming from classical ML. I am just doing MLOps/DE these days. Although I have been down levelled to almost junior level after having 10+ years of experience. Struggling a bit with being treated like a junior again! Much less autonomy to pick my own task.

extr 3y ago
I'm going through a similar transition right now. I wouldn't say I've been downleveled but I left FAANG to go to a startup (pay cut) where I've been able to transition pretty fluidly from "Data Science" to "MLOps + some backend SWE + some infra + some actual ML Engineering (non-DL)". Aside from the pay cut (which was not the end of the world, think high 200s to low 200s), it's worked out very well, and I'm getting the experience that I think will allow me to rebrand successfully. As to whether that rebrand itself is successful, I don't know. It feels like the DS role is undergoing a maturation where different skillsets are being cleaved into different roles. I was always a more natural fit for the Ops side, I don't have the academic creds for a research-oriented position. But I'm not sure how much of ops means "LLM ops".

next_xibalba 3y ago

> I've seen the way the winds are blowing wrt to the DS job title Can you elaborate on this?

extr 3y ago
I mention this in another reply but tl;dr the DS role is maturing and getting cleaved into different specialist skillsets. Research, ML Engineering, MLOps, Data Engineering, Analytics are what I see as the main ones. In 2016 you could get hired as a Data Scientist knowing some SQL, XGBoost, and how to use git (I know I did). That's not really the case in 2024, you are expected to be an expert in at least one of those areas and job titles are starting to reflect the specialization. Companies that are still hiring for generic "Data Scientist" positions with scikit-learn in the job description are not places you want to work.
- BOOSTERHIDROGEN 3y ago
  The questions is how deep should you go of each subset ?

Bnjoroge 3y ago

definitely shifting to more engineering-focused type of roles so at the very I'd work on swe/infra skills