3 ms·
Hundred times this. You see Qlik/Cognos Analytics/PowerBI/Alteryx/whatever sales guys making demos that make executives drool over the seeming easiness and wow-
by gerhardi 10y ago
Hundred times this. You see Qlik/Cognos Analytics/PowerBI/Alteryx/whatever sales guys making demos that make executives drool over the seeming easiness and wow-factor these tools are capable of producing. When the time comes to plug those over your production operative systems, CRM, whatelse, there comes "the now wait a minute" moment especially if your systems and their data models happen to be even slightly on the more complex side.
Edit: The basis of succesful implementation of these tools is to have the data in digestible format and I feel that transforming the data to that business usable format is where the big job is.
In my opinion well done ETL and DW are not going anywhere, even though in some circles they are said to be things of yesterday. Then there's a huge difference between an OK ETL/DW and a Brilliant ETL/DW. Designing a good ETL process is as large parts business and context knowledge as it is a application of data engineering skills. For example, it requires business knowledge AND data engineering knowledge to determine what kind of granular level advanced metrics could or should be calculated during ETL. Service level metrics and service level categorization for different kind of customers/claims/orders/... would be a perfect simple to understand example problem - there could be attributes and value ranges behind multiple relations that probably need to be taken into account and understood.
Edit 2: I've been involved in both sales and execution of so called data discovery sprints, which are a 4-6 week periods where we bring a data engineer, a subject matter expert and client key personnel working together and let them go "fishing". The key thing is that this provides an low cost way for the clients to possibly gain insight on the potential their data could provide. On the other hand, many prospective clients just have so messy data that this data discovery job can't be recommended, which leads to other possible opportunities (MDM, ETL, DW).
- vram22 10y agoMDM is Master Data Management? (from a quick google). Hadn't heard of that specific term before. I'm interested in data projects.
- gerhardi 10y agoYes correct. In a way it's a combination of philosophy, agreed practices and techical solutions. I think this one is a good introduction: The What, Why, and How of Master Data Management https://msdn.microsoft.com/en-us/library/bb190163.aspx https://msdn.microsoft.com/en-us/library/bb190163.aspx
- vram22 10y agoJust checked it out - looks good. Thanks.