3 ms·
What type of Data Engineer do you want to be? We're starting to see an unbundling of data engineering into analytics Engineering (BI developer on steroids), ML
by beckingz 4y ago
What type of Data Engineer do you want to be?
We're starting to see an unbundling of data engineering into analytics Engineering (BI developer on steroids), ML engineering (AutoML is good enough that if you can do good feature engineering and pipelines the marginal value of adding a data scientist is not worth the cost, and data platform engineering (K8s. K8s everywhere).
- thewhitetulip 4y agoI am sorry, I did not understand Analytics engineering. do you mean that the BI developer will be expected to do DE stuff? it was how my prev org rolled!!
- edmundsauto 4y agoAnalytics engineers are towards the end of data flow - they built data marts and dashboards, often using code.
- panda888888 4y agoYes, look up the company called dbt to learn more. (I don't work there or anything, it's just a popular tool and their website explains it well.)
- beckingz 4y agoCorrect. The idea is to have a role that sits right between data engineering and data analysis, ideally with a breadth that overlaps both.
- dataminded 4y agoAnalytics Engineering are folks that transform data in an ELT model. The most popular tool is DBT. I think the work will continue as companies embrace the idea of not writing custom ETL code anymore. The tools are super immature though. DBT is a huge improvement over the past but far from a final solution to the problem.