3 ms·

As a somewhat technical leader/manager, I’m pretty comfortable with SQL but that also means I know i could pretty easily goof up these queries. I don’t know mu

As a somewhat technical leader/manager, I’m pretty comfortable with SQL but that also means I know i could pretty easily goof up these queries. I don’t know much about the quality or exceptions that may be present in the underlying data either. I simply wouldn’t trust my own results for fear I was overlooking something. So, I’d rather ask the BI person to make this for me. They should be more intimately familiar with any footguns. For that reason, I see the technical part of this post at odds with the initial premise that a leadership role should need to learn how to query their data. If the resulting information, stats, etc are being used to answer business questions and make business decisions, it would be best that the person that specializes in this produces said information/queries. If there’s some tool GUI interface and the datasets are clean or well documented, then maybe the self service nature is on the table again but anything moderately complex likely will still be run through the BI team. In a sense, it’s just basic QC, it’s not that I’m completely helpless. I might even do a first pass and kick it to BI for them to review, but seldom do I find myself in a real world situation where I’m confident enough about my knowledge of the underlying data so that gives me a huge pause most of the time.

simonw 2y ago
I'm a little skeptical of dedicated BI teams as the sole oracles of correct queries. In my experience it's sadly common for an engineering team to make a schema or product design change that isn't instantly obvious to the BI team, resulting in incorrect queries. I think the best way to address this is to aim for a culture of transparency: any time anyone in a company presents a report or dashboard or similar, the query that was used to create it needs to be easily accessible. That way there's a much higher chance of mistakes being spotted. The BI equivalent of "given enough eyeballs, all bugs are shallow". Another cultural trick that's important is that engineering teams need to consider schema changes as part of their documented and supported API. If a change might affect BI reports the people who write and use those reports need to be told about it.
mble_ 2y ago
Author here. One of the main things here is that you should know your data well enough to articulate the right request from BI. In my experience, BI often end up as pure order takers - if you ask the wrong question, you get a lovingly formatted but wrong answer. The other thing is that this assumes you have a BI team at hand - smaller teams/orgs often don't! Perhaps I should make this a little more explicit. My central thesis, also not made explicit, is that leaders should be appropriately curious _and_ leverage the tools they have to be able to do things like "hey, this looks weird, what's up?" and share the data and their methodology - that way it can be corrected/investigated etc.
- PaulHoule 2y ago
  From my POV I have a choice of a database or a tool like pandas. Anybody who is interested in this sort of work has a choice of doing it with databases or with a specialized data analysis tool. What's your take on that?
  - mble_ 2y ago
    Why not both? There are times when pushing the work down to the database layer is appropriate - databases are quite good at a lot of these operations - but if you need more nuanced approaches (e.g. ANOVA, ARIMA, other kinds of forecasting or analysis), leverage the appropriate tools.
- conductr 2y ago
  Thanks for chiming in, great post, I like the premise - I just think we must have completely different working experiences. I'm typically in a larger org that has multiple systems feeding data into a data lake or something similar that has been normalized but also can still usually has some quirks. Articulating the right request to BI is certainly a skill, but my approach/experience is that I try to paint the picture of the end goal and let them fill in the gaps as needed. Sometimes that's literally drawing out a graph or chart that I want to exist. Even when no BI team is dedicated, there's usually someone that's wearing that hat. Someone setup those schemas and data pipelines, etc or is responsible for maintaining them. That person is probably the one that knows "make sure you exclude the NULL items" or something similar. I do like being in touch with changing data trends from a leadership perspective. It's either real and could be a valuable insight or it's a bug that needs to be addressed before any ill advised decisions are made from the 'info'. I find this can often be setup proactively and put into a dashboard. In that way, identifying it and raising concern can be 'my job' but when investigating it, it could be a team effort.
  - mble_ 2y ago
    > I just think we must have completely different working experiences. Likely! I've generally worked in smaller orgs (including as part of a much larger org, as with my current employer) and there is less access to dedicated resources. > Even when no BI team is dedicated, there's usually someone that's wearing that hat. 100%. Unfortunately, this has commonly be me from my personal experience. > In that way, identifying it and raising concern can be 'my job' but when investigating it, it could be a team effort. Totally agreed. For some additional context, I've spent my working career on data systems so I likely feel a much stronger affinity to this type of self-serve analysis than your average bear.

simonw 2y ago

I'm a little skeptical of dedicated BI teams as the sole oracles of correct queries. In my experience it's sadly common for an engineering team to make a schema or product design change that isn't instantly obvious to the BI team, resulting in incorrect queries. I think the best way to address this is to aim for a culture of transparency: any time anyone in a company presents a report or dashboard or similar, the query that was used to create it needs to be easily accessible. That way there's a much higher chance of mistakes being spotted. The BI equivalent of "given enough eyeballs, all bugs are shallow". Another cultural trick that's important is that engineering teams need to consider schema changes as part of their documented and supported API. If a change might affect BI reports the people who write and use those reports need to be told about it.

mble_ 2y ago

Author here. One of the main things here is that you should know your data well enough to articulate the right request from BI. In my experience, BI often end up as pure order takers - if you ask the wrong question, you get a lovingly formatted but wrong answer. The other thing is that this assumes you have a BI team at hand - smaller teams/orgs often don't! Perhaps I should make this a little more explicit. My central thesis, also not made explicit, is that leaders should be appropriately curious _and_ leverage the tools they have to be able to do things like "hey, this looks weird, what's up?" and share the data and their methodology - that way it can be corrected/investigated etc.

PaulHoule 2y ago
From my POV I have a choice of a database or a tool like pandas. Anybody who is interested in this sort of work has a choice of doing it with databases or with a specialized data analysis tool. What's your take on that?
- mble_ 2y ago
  Why not both? There are times when pushing the work down to the database layer is appropriate - databases are quite good at a lot of these operations - but if you need more nuanced approaches (e.g. ANOVA, ARIMA, other kinds of forecasting or analysis), leverage the appropriate tools.
conductr 2y ago
Thanks for chiming in, great post, I like the premise - I just think we must have completely different working experiences. I'm typically in a larger org that has multiple systems feeding data into a data lake or something similar that has been normalized but also can still usually has some quirks. Articulating the right request to BI is certainly a skill, but my approach/experience is that I try to paint the picture of the end goal and let them fill in the gaps as needed. Sometimes that's literally drawing out a graph or chart that I want to exist. Even when no BI team is dedicated, there's usually someone that's wearing that hat. Someone setup those schemas and data pipelines, etc or is responsible for maintaining them. That person is probably the one that knows "make sure you exclude the NULL items" or something similar. I do like being in touch with changing data trends from a leadership perspective. It's either real and could be a valuable insight or it's a bug that needs to be addressed before any ill advised decisions are made from the 'info'. I find this can often be setup proactively and put into a dashboard. In that way, identifying it and raising concern can be 'my job' but when investigating it, it could be a team effort.
- mble_ 2y ago
  > I just think we must have completely different working experiences. Likely! I've generally worked in smaller orgs (including as part of a much larger org, as with my current employer) and there is less access to dedicated resources. > Even when no BI team is dedicated, there's usually someone that's wearing that hat. 100%. Unfortunately, this has commonly be me from my personal experience. > In that way, identifying it and raising concern can be 'my job' but when investigating it, it could be a team effort. Totally agreed. For some additional context, I've spent my working career on data systems so I likely feel a much stronger affinity to this type of self-serve analysis than your average bear.