4 ms·
This lines up with my experience, and I've found it heavily depends on what industry you are in. > software engineer in the data domain. I was basically this
by snapetom 3y ago
This lines up with my experience, and I've found it heavily depends on what industry you are in.
> software engineer in the data domain.
I was basically this for the past ten years. Maybe it was because I was working only in startups.
Outside of tech/startup orgs, "data engineer" at least I found, were SQL specialists. About six years ago, I went into healthcare, and discovered there were about 30 people across five teams that were data engineers. "Oh cool. My colleagues," I thought. Imagine my surprise when I found they only knew SQL, knew data modeling theory, and had basically no SDLC experience. At my present job, in a traditionally blue-collar industry, I took over a team with the only data engineer in the whole company. He, too, knew only SQL. I've had shove Python at him and get him working in SDLC.
I think these people though, are shrinking. Putting pressure on this from the other side, Python is a common skill with data analyst these days. Software engineers do the heavy lifting and good-enough data modeling, while data analyst do business-specific analysis and good-enough software development like writing DAGs with Dagster. Knowing SQL isn't enough to get by in the job market.
- desert_rue 3y agoI know plenty of people who only use sql. There is now a role called analytics engineer that primarily sql and often with transformation tools like dbt. I personally haven’t met a lot of software devs who would call their data modeling capabilities ‘good enough’. It is a different way of thinking to go from building 3rd form normalized tables to denormalizing everything. Plus if a company is merging data from many systems, it makes it more complicated for the software developer.
- fifilura 3y agoThe moment you have learned SQL, is when you have forgotten everything about C/C++