3 ms·
Excellent executive summary for non-executives. Some of these (client) companies are so afraid of code it's bewildering. It's almost like they don't trust thems
by mrslave 5y ago
Excellent executive summary for non-executives. Some of these (client) companies are so afraid of code it's bewildering. It's almost like they don't trust themselves to test & maintain even small code bases.
I've seen them go out of their way to pursue point-and-click solutions to data warehouses/lakes: thousands of ETL jobs manually coded (and manually tested) with very little "code" reuse. The inconsistencies/deviations from conventions were worse than the development waste. Consumers of these data will have to deal with inconsistencies in naming conventions, versioning strategies, and broken SCDs for years to come. Too often the architects/data modelers/ETL developers don't even know what the primary key is! (Don't lose your keys folks.)
- lumost 5y agoDevelopers are often uninterested in data pipelines with the exception of high scale, high throughput, business critical data pipelines e.g. Analytics product offerings, backend work, ML and other activities. Standard BI suffers from incoherent requirements and often lacking an internal customer. It's in some ways a good problem to have 100 different independent reports in case you decide to shut 50-80 of them off one day.