3 ms·

So if engineering team uses an ORM, pushes a massive API change that renames columns and removes others, and drops FKs, and leaves it as a change item in a rele

by ttz 5y ago

So if engineering team uses an ORM, pushes a massive API change that renames columns and removes others, and drops FKs, and leaves it as a change item in a release list in an access controlled repo without notifying anyone, it's BI team's fault the DW ETL breaks? IMO it's not fruitful to throw blame on teams or call them unskilled. Usually, everyone is trying their best to do their job, and things that make this not happen should be seen as "how can we improve/fix this" - better communication is almost always the solution.

Tabular-Iceberg 5y ago
It’s the engineering team that creates the actual value for the company. Of course they should have an easy path to clear technical debt. Database level integration is tempting at first since it’s zero effort up front, but it usually ends badly. It’s well worth it to make the BI team go through an API, even spend the effort on a well-designed one to give them a nice experience doing so.
- ttz 5y ago
  > It’s well worth it to make the BI team go through an API, even spend the effort on a well-designed one to give them a nice experience doing so. In an ideal world yes. But sometimes, the engineering team wants nothing to do with the data itself, and so the BI team is often left to do what they can with what they have in order to do their job at all. Being a data engineer, both sides of the coin often fall on me, and so I understand the difficulties on both side. Almost always (anecdotally, so take with grain of salt), the problem lies with upper management because they aren't willing to invest the time to actually build a proper infrastructure in place; the focus is on building features to satisfy clients, without much thought on how this can affect the overall internal system, and so both sides have to rush to do things without thinking how it affects the other side. However, it likely won't change because at the end of the day, the client is happy - whether or not this is a "unfortunate reality" of business/tech, is a discussion for another day.
  - Tabular-Iceberg 5y ago
    Until I read the second I was going to say to your first paragraph "then the company clearly doesn't value BI enough to warrant the existence of a BI team". It's probably more a symptom of upper management valuing neither the core team nor the BI team, thinking software writes itself, and business insights come for free with the MBA. We're really in the same boat here. All the more reason for core team to make nice hooks for the BI team. They won't be appreciated by management, they probably won't be appreciated by the customers, but at least they can be appreciated by each other.
    - ttz 5y ago
      In my experience (which doesn't mean I'm disagreeing with your experience), it's not that upper mgmt doesn't recognize the value of BI - they just don't want to invest in it. They want their cake without having to pay for it. It can be hard to find a place with mgmt that really understands that investing in developer velocity and good infrastructure pays way more dividends in the long term than constant short term "promise X, deliver X*0.8 to make customer think they got what they want, and repeat to get the contract". But yeah, we could complain on and on about all the problems of shortsighted management... it's a tale as old as time.

by ttz 5y ago

Tabular-Iceberg 5y ago

It’s the engineering team that creates the actual value for the company. Of course they should have an easy path to clear technical debt. Database level integration is tempting at first since it’s zero effort up front, but it usually ends badly. It’s well worth it to make the BI team go through an API, even spend the effort on a well-designed one to give them a nice experience doing so.

ttz 5y ago
> It’s well worth it to make the BI team go through an API, even spend the effort on a well-designed one to give them a nice experience doing so. In an ideal world yes. But sometimes, the engineering team wants nothing to do with the data itself, and so the BI team is often left to do what they can with what they have in order to do their job at all. Being a data engineer, both sides of the coin often fall on me, and so I understand the difficulties on both side. Almost always (anecdotally, so take with grain of salt), the problem lies with upper management because they aren't willing to invest the time to actually build a proper infrastructure in place; the focus is on building features to satisfy clients, without much thought on how this can affect the overall internal system, and so both sides have to rush to do things without thinking how it affects the other side. However, it likely won't change because at the end of the day, the client is happy - whether or not this is a "unfortunate reality" of business/tech, is a discussion for another day.
- Tabular-Iceberg 5y ago
  Until I read the second I was going to say to your first paragraph "then the company clearly doesn't value BI enough to warrant the existence of a BI team". It's probably more a symptom of upper management valuing neither the core team nor the BI team, thinking software writes itself, and business insights come for free with the MBA. We're really in the same boat here. All the more reason for core team to make nice hooks for the BI team. They won't be appreciated by management, they probably won't be appreciated by the customers, but at least they can be appreciated by each other.
  - ttz 5y ago
    In my experience (which doesn't mean I'm disagreeing with your experience), it's not that upper mgmt doesn't recognize the value of BI - they just don't want to invest in it. They want their cake without having to pay for it. It can be hard to find a place with mgmt that really understands that investing in developer velocity and good infrastructure pays way more dividends in the long term than constant short term "promise X, deliver X*0.8 to make customer think they got what they want, and repeat to get the contract". But yeah, we could complain on and on about all the problems of shortsighted management... it's a tale as old as time.