3 ms·
Not a hacker, not even a good programmer, but I can relate from another perspective. I have been working as a BI developer->Data Engineer for a few medium size
by hnthrowaway0315 4y ago
Not a hacker, not even a good programmer, but I can relate from another perspective.
I have been working as a BI developer->Data Engineer for a few medium size modern corporations. Modern in the sense of FAANG-like. Each company gives me the feeling that the data infrastructure is siloed and unhealthy.
When business grows, different business units drill deeper into the data domain. Each business unit quickly develops its own shadow ETL which works to certain extend but without proper structure or any data governance. Once business grows to certain size, it is mandatory for business leaders to realize that such siloed and unprofessional data practices should stop and data requirements should be consolidated to a central data team that consists of data engineers and maybe a couple of BA or a PM who mediates the engineers and the business stakeholders.
Sadly in none of the corporations I worked for I found this transformation happened. Some prefer to keep data siloed and people are getting used to it. Some tried to transform to a central data team but it failed to maintain expectation of all business stakeholders so each business unit keeps its data unit around, for convenience. These situations create more headcounts, more costs, more bad practices and more frustration.