3 ms·

I was a SI/BI engineer for a while; let me chime in to defend my people just a little bit :) Certainly there are lots of bullshit metrics, there is often very

I was a SI/BI engineer for a while; let me chime in to defend my people just a little bit :) Certainly there are lots of bullshit metrics, there is often very little desire to audit data or even do a by hand sanity check once and a while. That being said, there were _many_ engineers who actually gave a fuck about making sure we had telemetry in actionable, meaningful, and appropriate places, and unfrotunately the gap in analytics would take place far over their heads, when the upper management would produce documents with all the beautiful charts and graphs, communicating.... absolutely nothing. Graphs with mixed axis scaling (log vs non log) graphs with mixed units, conclusions that are a total stretch from the data that's there and ignoring obvious conclusions that don't line up with what the managers want to say. For there to be USEFUL BI (and such a thing certainly can exist) there needs to be a "this isn't bullshit" mindset up and down the whole stack, not just in engineer land.

endymi0n 11y ago
While what I said was actually an amalgamation of true stories I've seen myself, I was certainly exaggerating a bit :) I think a lot of things come together for this pattern, which definitely happens quite a lot. Best predictor for this scenario is a clear divide between product and BI people, usually exacerbated by the fact that non-technical people get hired for BI. Group bias by backend engineers who see BI as "just some point and click" which isn't really that challenging (but usually much better paid, as they are catering sales/bizdev/C-level which is always closer to the money) also doesn't help. Also, as you say, often times the value of BI doesn't trickle back down the chain and that way, tracking is at most a second thought for application engineers when going prod. Having built two analytics stacks myself (and seen the perspectives from BI, backend, sales and marketing alike) these are exactly the drivers we tackled first at our current company. Our recipes against this common failure are: Marketing directly working with product engineers for their tracking requirements (with just some coaching from tracking pros) - and dual-using our analytics stack (Snowplow -> Redshift) for both operations (user segmenting, push notifications) and business intelligence. This is usually considered a big no-no in BI circles which all tend to duplicate data to be on the safe side, but it helps immensely to make sure product and engineering are just as interested in data quality as the BI guys, as they depend on the very same data. It's certainly not for everyone, but for us, it works really well.

endymi0n 11y ago

While what I said was actually an amalgamation of true stories I've seen myself, I was certainly exaggerating a bit :) I think a lot of things come together for this pattern, which definitely happens quite a lot. Best predictor for this scenario is a clear divide between product and BI people, usually exacerbated by the fact that non-technical people get hired for BI. Group bias by backend engineers who see BI as "just some point and click" which isn't really that challenging (but usually much better paid, as they are catering sales/bizdev/C-level which is always closer to the money) also doesn't help. Also, as you say, often times the value of BI doesn't trickle back down the chain and that way, tracking is at most a second thought for application engineers when going prod. Having built two analytics stacks myself (and seen the perspectives from BI, backend, sales and marketing alike) these are exactly the drivers we tackled first at our current company. Our recipes against this common failure are: Marketing directly working with product engineers for their tracking requirements (with just some coaching from tracking pros) - and dual-using our analytics stack (Snowplow -> Redshift) for both operations (user segmenting, push notifications) and business intelligence. This is usually considered a big no-no in BI circles which all tend to duplicate data to be on the safe side, but it helps immensely to make sure product and engineering are just as interested in data quality as the BI guys, as they depend on the very same data. It's certainly not for everyone, but for us, it works really well.