3 ms·

Some companies don't separate ML roles by title, but that still leads to problems. Without management-driven separation, too many people try to work on just mo

Some companies don't separate ML roles by title, but that still leads to problems. Without management-driven separation, too many people try to work on just modeling. You wind up with nobody labeling and cleaning the data. Everyone complains that it's hard to get their models to work on the limited data available, but no one wants to be relegated to the data cleaning role. I have yet to see a good company layout that makes everyone happy. Too many people want to work on modeling, and the worst part is that individuals are right to fight for it. The more practice you get, the more you improve at the work. Plus, companies tend to value good modelers, so your salary goes up too. It's a very frustrating feedback loop. Separating modeling work by title or by team is one of the few ways companies can keep from having a run on the modeling projects. There are a few other ways, but they are all pretty bad too. You can have a "keep what you kill" structure, where everyone gets first dibs on doing modeling on the data they've collected and cleaned. I've also seen a company that makes the modeling job as miserable as possible with slow tooling and high expectations, so lots of people wash out and only a few modelers are left. I wish I knew a good solution to this. My only idea is to play-up the importance (and salary!) of the support roles, but I'm not satisfied yet.

disgruntledphd2 6y ago
Like, this blows my mind. I can't even imagine a world where one would model without data cleaning. In the olden times, your model would normally error out if you hadn't done any data cleaning (length 1 factors, constant columns etc) but i suppose with NN's they just ignore stuff like that because of the regularisation, I assume. Every good DS I know regards the data as much more important than the models, normally from bitter, bitter experience. Models are "sexy" though, so people love them. Additionally you can do the XKCD compiling thing x 10 with models, as they take forever to fit on any non-trivial datasets. I actually think that the cause of this is poor management. ML/DS//statistics are tools, they do not supply any business value by themselves. If people focus too much on the tools, then their manager should refocus them on the actual problem. Mind you, if you don't have the background in this area as a manager, you're less likely to be able to spot these pathologies. Personally, as an experienced DS, I know that my time is best spent understanding the data and the business problem, as that will drive better results. It's sad that people don't realise this (including some of my current management chain, unfortunately).
dagw 6y ago
As the more senior data analyst/ML person I've found myself taking on that role more and more. I've been 'relegating' the hands on modeling to the more junior members of the team, taking only a more supervisory/mentoring role when it comes to modelling. Quite frankly the modeling job is a lot easier and is more suitable for people fresh out of school, as it aligns more with what they've been studying. Data collection/cleaning/labeling etc. and generating good data sets that solve the problem we're facing requires a lot more domain knowledge and experience, and quite frankly I'm a lot better and faster at it compared to a freshly minted Masters/PhD student. I've seen far more ML projects fail because they couldn't get the right data set, and very few fail because they couldn't find the right model. So I'm getting more and more convinced that getting the data set right should be the primary responsibility of the most senior member(s) of the project team.
bluGill 6y ago
There is the bank solution. Working collections is the poster for job that will make you hate your life, but it must be done. So the rule is all jobs must be posted to internal transfers for two weeks, and if someone qualifies applies you can't look for someone outside the company. As a result it is nearly impossible to get a job without first working a collections job for 9 months. Collections hires basically anyone and helps them move on to a better position after you have done your time. Everyone else knows you don't hire anyone who hasn't done their time in collections as it is the only way that needed job can be done. Data is not nearly as evil - hate your whole life - as collections, so I hope that we don't have to stoop to this level. It is an option though.
- lumost 6y ago
  what is collections in this case?
  - bluGill 6y ago
    Calling people who aren't paying their mortgage and threatening to fore close if they don't make their payments. Or similar to those who don't pay their credit cards. Generally people who are in this situation are having a lot or trouble in life.
lumost 6y ago
This is a common management dilemma when there are specific tasks that are highly valuable in future engineering careers but have limited application within the current organization. Or when key "promotion worthy" projects come around and individuals are strongly incentivised to have the "core" contribution. The best approach I've found is to reward based on impact to multiple individuals, and to some extent modulate it by rewarding individuals who "run towards fires". A straight scientist who makes the best datasets in a scalable fashion will thus have high impact on a painful part of the business that no one else will work on.

disgruntledphd2 6y ago

Like, this blows my mind. I can't even imagine a world where one would model without data cleaning. In the olden times, your model would normally error out if you hadn't done any data cleaning (length 1 factors, constant columns etc) but i suppose with NN's they just ignore stuff like that because of the regularisation, I assume. Every good DS I know regards the data as much more important than the models, normally from bitter, bitter experience. Models are "sexy" though, so people love them. Additionally you can do the XKCD compiling thing x 10 with models, as they take forever to fit on any non-trivial datasets. I actually think that the cause of this is poor management. ML/DS//statistics are tools, they do not supply any business value by themselves. If people focus too much on the tools, then their manager should refocus them on the actual problem. Mind you, if you don't have the background in this area as a manager, you're less likely to be able to spot these pathologies. Personally, as an experienced DS, I know that my time is best spent understanding the data and the business problem, as that will drive better results. It's sad that people don't realise this (including some of my current management chain, unfortunately).

dagw 6y ago

As the more senior data analyst/ML person I've found myself taking on that role more and more. I've been 'relegating' the hands on modeling to the more junior members of the team, taking only a more supervisory/mentoring role when it comes to modelling. Quite frankly the modeling job is a lot easier and is more suitable for people fresh out of school, as it aligns more with what they've been studying. Data collection/cleaning/labeling etc. and generating good data sets that solve the problem we're facing requires a lot more domain knowledge and experience, and quite frankly I'm a lot better and faster at it compared to a freshly minted Masters/PhD student. I've seen far more ML projects fail because they couldn't get the right data set, and very few fail because they couldn't find the right model. So I'm getting more and more convinced that getting the data set right should be the primary responsibility of the most senior member(s) of the project team.

bluGill 6y ago

There is the bank solution. Working collections is the poster for job that will make you hate your life, but it must be done. So the rule is all jobs must be posted to internal transfers for two weeks, and if someone qualifies applies you can't look for someone outside the company. As a result it is nearly impossible to get a job without first working a collections job for 9 months. Collections hires basically anyone and helps them move on to a better position after you have done your time. Everyone else knows you don't hire anyone who hasn't done their time in collections as it is the only way that needed job can be done. Data is not nearly as evil - hate your whole life - as collections, so I hope that we don't have to stoop to this level. It is an option though.

lumost 6y ago
what is collections in this case?
- bluGill 6y ago
  Calling people who aren't paying their mortgage and threatening to fore close if they don't make their payments. Or similar to those who don't pay their credit cards. Generally people who are in this situation are having a lot or trouble in life.

lumost 6y ago

This is a common management dilemma when there are specific tasks that are highly valuable in future engineering careers but have limited application within the current organization. Or when key "promotion worthy" projects come around and individuals are strongly incentivised to have the "core" contribution. The best approach I've found is to reward based on impact to multiple individuals, and to some extent modulate it by rewarding individuals who "run towards fires". A straight scientist who makes the best datasets in a scalable fashion will thus have high impact on a painful part of the business that no one else will work on.