3 ms·
The lack of a roadmap is disastrous in terms of professional quality IMO. A lot of orgs have data scientists who are good at coding and maths who can't do their
by baazaa 3y ago
The lack of a roadmap is disastrous in terms of professional quality IMO. A lot of orgs have data scientists who are good at coding and maths who can't do their work because the data engineers, who got their jobs because they kinda knew SQL a decade ago, are incompetent.
I'm sure this is less of a problem in tech companies which are skimming the cream of the crop, but in other organisations it's apparent that a third-rate data scientist or software engineer is probably much better at their job than a second-rate data engineer. And indeed if I were running an IT dept I'd probably just fill the data engineering roles with decent coders and tell them to reskill a bit.
- samplatt 3y ago>the data engineers, who got their jobs because they kinda knew SQL a decade ago, are incompetent. I don't think I've ever been called out more in my life.
- xmcqdpt2 3y agoI've been in such an org. The problem is that few decent coders want to do data engineering work, because they'd rather build software and because data engineering pays less and is kind of a dead end in a lot of org. You end up with a lot of DBAs that self-style as Data Engineers.
- baazaa 3y agoI'm in Australia and data engineers here are overpaid, afaict slightly better paid on average than software engineers who are better paid than data people. So you end up with bizarro stuff where a $150k AUD data engineer who basically can't code or even set-up FTP telling some maths PhD data scientist on $100k what he can or can't do. Managers, unable to gauge technical ability, assume the data engineer must be highly skilled because they're getting paid so much. The wages and lack of skill I think are symptomatic of there being no real entry-point in data eng. There's a very slow osmosis of people from genuinely skilled areas into data eng, the main drawback being that the entire culture of data eng is defective with so many unskilled hacks.
- hulahoof 3y agoI work in the aus data space, I’ve met a lot of intelligent PHD data scientists, but very rare is it to meet one who can scale their work efficiently across the org. The amount of time wasted on models that never make it to production is ridiculous.
- deleted 3y ago[deleted]