4 ms·
One of the problems I’ve seen with folks in data engineering is an underdeveloped sense of data modeling and table design. And this article is an example of tha
by andelink 3y ago
One of the problems I’ve seen with folks in data engineering is an underdeveloped sense of data modeling and table design. And this article is an example of that. It heavily emphasized SQL, but didn’t mention anything about data modeling. If you aren’t designing your tables thoughtfully, no amount of SQL proficiency will save you.
- deleted 3y ago[deleted]
- nevi-me 3y agoThis reminds me of an interesting project when I was still a consultant. Our partner (audit + consultancy firm) sold the idea of a magical "analytical base table" which would consolidate the client's data in a single place for quicker reporting and analysis. Of course all of this was new to me (the ABT thing really obscured the problem). I was going for quality review with another partner at the firm whose bread & butter are data management. Let's say it was a very harsh review. At the end of the review, he recommended that I learn data modelling properly. He gave me a copy of his Kimball book for the December holidays. I went through and completed it, and on the other side of that we were able to build a coherent dimensional data model to address the client's problem efficiently. That knowledge makes me agree with you, merely knowing SQL or your way around a database is often not enough, and knowing some theory about data modelling (regardless of which type you prefer) is a key element in a data engineer's toolkit.