3 ms·

I would give the exact opposite advice - data engineering is one of the least rewarding or respected career paths.

mistrial9 2y ago
a large organization that can pay stable wages over time is going to look for formal credentials when making a hiring decision. Further from that kind of (big,bureaucratic) group, are quickly assembling and then dissolving entities .. in other words "nice work if you can get it" .. with far less stability. Another term for this is "the Wild West" environment. There are some applicable analogies to the way Hollywood makes movies also perhaps, but with far less grounding. In the Hollywood example from what I know of.. small "tiger teams" assemble with fundamentals, then quickly farm out the sexy work to disposable contracting firms, who then hire even more disposable people with various skill levels. In other words, lots of fun and excitement but also lots of work place abuses and low stability. Over time, Hollywood formed unions for a surprisingly large number of roles (like writers) because the real truth of business is not pretty. Needless to say, Silicon Valley has moved very quickly, and the Hollywood stories are not exactly applicable.
sevensor 2y ago
If you find that this is the case in your organization, watch out. It's a sign that you're trying to build performative models that aren't grounded in reality. Depending on your organization's goals, this may take a long time to catch up with you.
jakeoverflow 2y ago
Data engineering feels like the DevOps of AI/ML. Important, but you have to have the right interests for it. This comes from someone doing web dev after studying data science, so I don't know how well it reflects reality as I've never worked as a Data Eng. myself
- sk11001 2y ago
  > Data engineering feels like the DevOps of AI/ML. I wouldn't label it that way, AI/ML has its own ops and devops which is very distinct from data engineering.

a large organization that can pay stable wages over time is going to look for formal credentials when making a hiring decision. Further from that kind of (big,bureaucratic) group, are quickly assembling and then dissolving entities .. in other words "nice work if you can get it" .. with far less stability. Another term for this is "the Wild West" environment. There are some applicable analogies to the way Hollywood makes movies also perhaps, but with far less grounding. In the Hollywood example from what I know of.. small "tiger teams" assemble with fundamentals, then quickly farm out the sexy work to disposable contracting firms, who then hire even more disposable people with various skill levels. In other words, lots of fun and excitement but also lots of work place abuses and low stability. Over time, Hollywood formed unions for a surprisingly large number of roles (like writers) because the real truth of business is not pretty. Needless to say, Silicon Valley has moved very quickly, and the Hollywood stories are not exactly applicable.

sevensor 2y ago

If you find that this is the case in your organization, watch out. It's a sign that you're trying to build performative models that aren't grounded in reality. Depending on your organization's goals, this may take a long time to catch up with you.

jakeoverflow 2y ago

Data engineering feels like the DevOps of AI/ML. Important, but you have to have the right interests for it. This comes from someone doing web dev after studying data science, so I don't know how well it reflects reality as I've never worked as a Data Eng. myself

sk11001 2y ago
> Data engineering feels like the DevOps of AI/ML. I wouldn't label it that way, AI/ML has its own ops and devops which is very distinct from data engineering.