4 ms·

My 2 cents is that ML oriented job titles are too fine grained with everyone in theory serving a mythic modeler who just needs to be fead data. Everyone finds t

by lumost 6y ago

My 2 cents is that ML oriented job titles are too fine grained with everyone in theory serving a mythic modeler who just needs to be fead data. Everyone finds the 1% of the time they spend building models the rewarding part - however many of the job titles like Data engineer, or ML engineer do not include modeling in the job description. There are companies which differentiate scientists into multiple categoies based on whether they do ML, can code, or just statistics. Now the problem is that any of these folks can get into trouble for doing the others job, and the pay scales are often radically different. Relegating core responsibilities such as data cleaning/collection to support roles. I'd bet that flattening the title structure in these groups would yield better results, and allow team members to more seamlessly move across Backend, Data, ML, and science oriented tasks.

ethbr0 6y ago
I think it's a problem of visibility up the management ladder. (A) If management understands ML enough to launch a feasible project, (B) if management understands that models are a component of ML, (C) if management understands models are created from data, (D) if management understands that production models must continually be fed data. Typically, understanding seems to stop somewhere between A & B.
lostdog 6y ago
Some companies don't separate ML roles by title, but that still leads to problems. Without management-driven separation, too many people try to work on just modeling. You wind up with nobody labeling and cleaning the data. Everyone complains that it's hard to get their models to work on the limited data available, but no one wants to be relegated to the data cleaning role. I have yet to see a good company layout that makes everyone happy. Too many people want to work on modeling, and the worst part is that individuals are right to fight for it. The more practice you get, the more you improve at the work. Plus, companies tend to value good modelers, so your salary goes up too. It's a very frustrating feedback loop. Separating modeling work by title or by team is one of the few ways companies can keep from having a run on the modeling projects. There are a few other ways, but they are all pretty bad too. You can have a "keep what you kill" structure, where everyone gets first dibs on doing modeling on the data they've collected and cleaned. I've also seen a company that makes the modeling job as miserable as possible with slow tooling and high expectations, so lots of people wash out and only a few modelers are left. I wish I knew a good solution to this. My only idea is to play-up the importance (and salary!) of the support roles, but I'm not satisfied yet.
- disgruntledphd2 6y ago
  Like, this blows my mind. I can't even imagine a world where one would model without data cleaning. In the olden times, your model would normally error out if you hadn't done any data cleaning (length 1 factors, constant columns etc) but i suppose with NN's they just ignore stuff like that because of the regularisation, I assume. Every good DS I know regards the data as much more important than the models, normally from bitter, bitter experience. Models are "sexy" though, so people love them. Additionally you can do the XKCD compiling thing x 10 with models, as they take forever to fit on any non-trivial datasets. I actually think that the cause of this is poor management. ML/DS//statistics are tools, they do not supply any business value by themselves. If people focus too much on the tools, then their manager should refocus them on the actual problem. Mind you, if you don't have the background in this area as a manager, you're less likely to be able to spot these pathologies. Personally, as an experienced DS, I know that my time is best spent understanding the data and the business problem, as that will drive better results. It's sad that people don't realise this (including some of my current management chain, unfortunately).
- dagw 6y ago
  As the more senior data analyst/ML person I've found myself taking on that role more and more. I've been 'relegating' the hands on modeling to the more junior members of the team, taking only a more supervisory/mentoring role when it comes to modelling. Quite frankly the modeling job is a lot easier and is more suitable for people fresh out of school, as it aligns more with what they've been studying. Data collection/cleaning/labeling etc. and generating good data sets that solve the problem we're facing requires a lot more domain knowledge and experience, and quite frankly I'm a lot better and faster at it compared to a freshly minted Masters/PhD student. I've seen far more ML projects fail because they couldn't get the right data set, and very few fail because they couldn't find the right model. So I'm getting more and more convinced that getting the data set right should be the primary responsibility of the most senior member(s) of the project team.
- bluGill 6y ago
  There is the bank solution. Working collections is the poster for job that will make you hate your life, but it must be done. So the rule is all jobs must be posted to internal transfers for two weeks, and if someone qualifies applies you can't look for someone outside the company. As a result it is nearly impossible to get a job without first working a collections job for 9 months. Collections hires basically anyone and helps them move on to a better position after you have done your time. Everyone else knows you don't hire anyone who hasn't done their time in collections as it is the only way that needed job can be done. Data is not nearly as evil - hate your whole life - as collections, so I hope that we don't have to stoop to this level. It is an option though.
beforeolives 6y ago
This is interesting because one of the criticisms towards data science have been that it's a term so broad that two people can have the title data scientist and do completely different jobs. Hence the differentiation over time into research scientists, ML engineers etc. Personally I might prefer the generalist hat but I see things moving in the opposite direction. There is also the problem of hiring - if 1-10% of the job requires a lot of statistical theory/research and the rest doesn't, do you hire PhDs and ask them to mostly do the engineering tasks that they aren't necessarily prepared to do? What about the people who are perfectly capable of doing 99% of the job and picking up the extra 1% of theory - do you exclude them because they don't have the academic credentials?
- lumost 6y ago
  I would take the approach that there is a spectrum of knowledge and hire the quote "T" shaped individual. Strong in at least one area, but broad enough to contribute anywhere. People change faster than orgs/titles, it would be better to keep this part flexible to maximize their growth. This can be further extended to the whole engineering culture. It's not uncommon that a frontend engineer shares a title with a backend software engineer - these skills are different enough that the individuals are no longer fungible for the most part. Why not give DS's engineering titles and treat it as a management objective to ensure you have the right people on any given project? If scientist is the preferred title, why not give anyone the science title?

by lumost 6y ago

ethbr0 6y ago

I think it's a problem of visibility up the management ladder. (A) If management understands ML enough to launch a feasible project, (B) if management understands that models are a component of ML, (C) if management understands models are created from data, (D) if management understands that production models must continually be fed data. Typically, understanding seems to stop somewhere between A & B.

lostdog 6y ago

Some companies don't separate ML roles by title, but that still leads to problems. Without management-driven separation, too many people try to work on just modeling. You wind up with nobody labeling and cleaning the data. Everyone complains that it's hard to get their models to work on the limited data available, but no one wants to be relegated to the data cleaning role. I have yet to see a good company layout that makes everyone happy. Too many people want to work on modeling, and the worst part is that individuals are right to fight for it. The more practice you get, the more you improve at the work. Plus, companies tend to value good modelers, so your salary goes up too. It's a very frustrating feedback loop. Separating modeling work by title or by team is one of the few ways companies can keep from having a run on the modeling projects. There are a few other ways, but they are all pretty bad too. You can have a "keep what you kill" structure, where everyone gets first dibs on doing modeling on the data they've collected and cleaned. I've also seen a company that makes the modeling job as miserable as possible with slow tooling and high expectations, so lots of people wash out and only a few modelers are left. I wish I knew a good solution to this. My only idea is to play-up the importance (and salary!) of the support roles, but I'm not satisfied yet.

disgruntledphd2 6y ago
Like, this blows my mind. I can't even imagine a world where one would model without data cleaning. In the olden times, your model would normally error out if you hadn't done any data cleaning (length 1 factors, constant columns etc) but i suppose with NN's they just ignore stuff like that because of the regularisation, I assume. Every good DS I know regards the data as much more important than the models, normally from bitter, bitter experience. Models are "sexy" though, so people love them. Additionally you can do the XKCD compiling thing x 10 with models, as they take forever to fit on any non-trivial datasets. I actually think that the cause of this is poor management. ML/DS//statistics are tools, they do not supply any business value by themselves. If people focus too much on the tools, then their manager should refocus them on the actual problem. Mind you, if you don't have the background in this area as a manager, you're less likely to be able to spot these pathologies. Personally, as an experienced DS, I know that my time is best spent understanding the data and the business problem, as that will drive better results. It's sad that people don't realise this (including some of my current management chain, unfortunately).
dagw 6y ago
As the more senior data analyst/ML person I've found myself taking on that role more and more. I've been 'relegating' the hands on modeling to the more junior members of the team, taking only a more supervisory/mentoring role when it comes to modelling. Quite frankly the modeling job is a lot easier and is more suitable for people fresh out of school, as it aligns more with what they've been studying. Data collection/cleaning/labeling etc. and generating good data sets that solve the problem we're facing requires a lot more domain knowledge and experience, and quite frankly I'm a lot better and faster at it compared to a freshly minted Masters/PhD student. I've seen far more ML projects fail because they couldn't get the right data set, and very few fail because they couldn't find the right model. So I'm getting more and more convinced that getting the data set right should be the primary responsibility of the most senior member(s) of the project team.
bluGill 6y ago
There is the bank solution. Working collections is the poster for job that will make you hate your life, but it must be done. So the rule is all jobs must be posted to internal transfers for two weeks, and if someone qualifies applies you can't look for someone outside the company. As a result it is nearly impossible to get a job without first working a collections job for 9 months. Collections hires basically anyone and helps them move on to a better position after you have done your time. Everyone else knows you don't hire anyone who hasn't done their time in collections as it is the only way that needed job can be done. Data is not nearly as evil - hate your whole life - as collections, so I hope that we don't have to stoop to this level. It is an option though.

beforeolives 6y ago

This is interesting because one of the criticisms towards data science have been that it's a term so broad that two people can have the title data scientist and do completely different jobs. Hence the differentiation over time into research scientists, ML engineers etc. Personally I might prefer the generalist hat but I see things moving in the opposite direction. There is also the problem of hiring - if 1-10% of the job requires a lot of statistical theory/research and the rest doesn't, do you hire PhDs and ask them to mostly do the engineering tasks that they aren't necessarily prepared to do? What about the people who are perfectly capable of doing 99% of the job and picking up the extra 1% of theory - do you exclude them because they don't have the academic credentials?

lumost 6y ago
I would take the approach that there is a spectrum of knowledge and hire the quote "T" shaped individual. Strong in at least one area, but broad enough to contribute anywhere. People change faster than orgs/titles, it would be better to keep this part flexible to maximize their growth. This can be further extended to the whole engineering culture. It's not uncommon that a frontend engineer shares a title with a backend software engineer - these skills are different enough that the individuals are no longer fungible for the most part. Why not give DS's engineering titles and treat it as a management objective to ensure you have the right people on any given project? If scientist is the preferred title, why not give anyone the science title?