4 ms·

> Is there a conclusive list of job titles and specifications one could rely on? No, there isn't. What you're describing could be called a lead data engineer

by jstx1 4y ago

> Is there a conclusive list of job titles and specifications one could rely on? No, there isn't. What you're describing could be called a lead data engineer but you can also call them a lead software engineer if you don't want to distinguish between data and non-data engineers with the title. But the "what to measure" and "where to measure" parts don't really fit under it - that's more like a data analyst or someone in a business role (depending on what you're measuring and how important/advanced it is), or a data scientist... but then not every part of the job needs to be reflected in the title. Basically there are many different ways to set up the titles, and most of them are fine.

jghn 4y ago
Data Engineer vs Software Engineer is always hard for me. It's always a choice between the lesser of two evils: 1) Advertise a data engineer role and get applications from rebranded DBAs who can't code their way out of a paper bag or 2) Advertise a software engineer role and get applications from software engineers who believe joining a couple of entities in their ORM counts as deep database experience
- nkmnz 4y ago
  This is what I struggle with as well. We have the coding capabilities measured by fingers that can type quality code, but we don't have the time nor the nerves to fail our way to solutions that work.
- jstx1 4y ago
  Yeah, it's messy. From the applicant side I prefer the title software engineer over data engineer. A more general title is better for career flexibility, and for some reason I've run into quite a few teams where the DEs have lower perceived status (perceived to not be able to code compared to software engineers, perceived to not be that smart/educated compared to a data science team etc).
  - jghn 4y ago
    The DE being lower status thing is precisely due to the rebranded DBA comment I made. It's no different than how the term "DevOps" was coopted by sys admins rebranding themselves with a hipper title. And we're starting to see the same thing with SRE being used by QA folks seeking a similar rebranding. As an applicant it's hard to figure out which form of Data Eng, DevOps, SRE the company practices and is seeking. As a company, it's hard to communicate your interpretation of the terms. IMO at least with DevOps one tell is if they consider it to be a role and not a philosophy, they probably are using it to mean sysadmins.
    - jstx1 4y ago
      There's also quite a few new-to-the-industry data engineers who are just starting out (many more than junior DevOps or SRE). It's a decently sized track for people who are graduating with CS degrees and for those who are switching from other fields. And I know quite a few people who started out trying to become data scientists, along the way figured out that they like DE more and stayed there.
- 3np 4y ago
  "Big Data Software Engineer"? (You can drop the "Big" if you have less big and want to appear less cringe) You could also attach a specification to the title like "Senior Software Engineer: Data [Services/Infrastructure/Platform]"
  - jghn 4y ago
    Ah but "Big Data" attracts the people who took a course on Spark and now see it as the tool to apply everywhere. It's turtles all the way down ;) When I'm hiring I've settled on sticking with "Software Engineer" and gasp being as specific as possible on what I'm looking for. But nothing will ever be perfect, or even great.
  - mellavora 4y ago
    Right, my "eats shoots and leaves" panda just put the comma in the wrong place; also remembering that one of the best engineers I ever had the please of working with was indeed "big" titles are almost as hard as naming things
- dominotw 4y ago
  I rebranded myself as dataengineer because i was sick of doing crud apps. Now i write etl all day.
- Test0129 4y ago
  It is strange to me you get DBAs applying for data engineering roles. Especially given that DBA is a very specific title and track. The companies I have worked at typically had data engineers separated because it was a highly specialized form of software engineering.
  - jghn 4y ago
    What I mean is, I see a ton of resumes of people who have a background of "DB Guy". Perhaps they've taken a class or two for coding, or writing airflow pipelines. Or maybe they've done a project or two in something buzz worthy. But they still pretty much just know how to operate, manage, and interact with DBs in the same manner as someone from 2005. That's not a bad skillset, those people are valuable. They're just not what I need.

by jstx1 4y ago

jghn 4y ago

Data Engineer vs Software Engineer is always hard for me. It's always a choice between the lesser of two evils: 1) Advertise a data engineer role and get applications from rebranded DBAs who can't code their way out of a paper bag or 2) Advertise a software engineer role and get applications from software engineers who believe joining a couple of entities in their ORM counts as deep database experience

nkmnz 4y ago
This is what I struggle with as well. We have the coding capabilities measured by fingers that can type quality code, but we don't have the time nor the nerves to fail our way to solutions that work.
jstx1 4y ago
Yeah, it's messy. From the applicant side I prefer the title software engineer over data engineer. A more general title is better for career flexibility, and for some reason I've run into quite a few teams where the DEs have lower perceived status (perceived to not be able to code compared to software engineers, perceived to not be that smart/educated compared to a data science team etc).
- jghn 4y ago
  The DE being lower status thing is precisely due to the rebranded DBA comment I made. It's no different than how the term "DevOps" was coopted by sys admins rebranding themselves with a hipper title. And we're starting to see the same thing with SRE being used by QA folks seeking a similar rebranding. As an applicant it's hard to figure out which form of Data Eng, DevOps, SRE the company practices and is seeking. As a company, it's hard to communicate your interpretation of the terms. IMO at least with DevOps one tell is if they consider it to be a role and not a philosophy, they probably are using it to mean sysadmins.
  - jstx1 4y ago
    There's also quite a few new-to-the-industry data engineers who are just starting out (many more than junior DevOps or SRE). It's a decently sized track for people who are graduating with CS degrees and for those who are switching from other fields. And I know quite a few people who started out trying to become data scientists, along the way figured out that they like DE more and stayed there.
3np 4y ago
"Big Data Software Engineer"? (You can drop the "Big" if you have less big and want to appear less cringe) You could also attach a specification to the title like "Senior Software Engineer: Data [Services/Infrastructure/Platform]"
- jghn 4y ago
  Ah but "Big Data" attracts the people who took a course on Spark and now see it as the tool to apply everywhere. It's turtles all the way down ;) When I'm hiring I've settled on sticking with "Software Engineer" and gasp being as specific as possible on what I'm looking for. But nothing will ever be perfect, or even great.
- mellavora 4y ago
  Right, my "eats shoots and leaves" panda just put the comma in the wrong place; also remembering that one of the best engineers I ever had the please of working with was indeed "big" titles are almost as hard as naming things
dominotw 4y ago
I rebranded myself as dataengineer because i was sick of doing crud apps. Now i write etl all day.
Test0129 4y ago
It is strange to me you get DBAs applying for data engineering roles. Especially given that DBA is a very specific title and track. The companies I have worked at typically had data engineers separated because it was a highly specialized form of software engineering.
- jghn 4y ago
  What I mean is, I see a ton of resumes of people who have a background of "DB Guy". Perhaps they've taken a class or two for coding, or writing airflow pipelines. Or maybe they've done a project or two in something buzz worthy. But they still pretty much just know how to operate, manage, and interact with DBs in the same manner as someone from 2005. That's not a bad skillset, those people are valuable. They're just not what I need.