4 ms·

People get really mad when you call this p-hacking, btw. I used to try and make this point to ”analysts” that I worked with, that science involves coming up wit

People get really mad when you call this p-hacking, btw. I used to try and make this point to ”analysts” that I worked with, that science involves coming up with a hypothesis and then looking at the data. The response I got often enough that I’m sure it’s being taught somewhere was that you “need to let the data tell it’s own story”. Turns out lots of people have decided what the conclusion is and are taking that conclusion and the data, and then blindly munging until they get a path which connects them.

axelf4 4y ago
As my professor said: All good statistics is done before you have looked at the data.
- sideshowb 4y ago
  But where do you get your hypothesis from?
  - beckingz 4y ago
    your hypothetical model of reality. But that's a lot of work so businesses don't want to pay for it.
  - Tams80 4y ago
    Whatever orifice you desire.
- merouan 4y ago
  True, for clean data. You can’t clean data without looking at it (a lot), though.
  - vanviegen 4y ago
    If you have data that's so 'dirty' that you can't decide on the filtering rules in advance (or based on only historic data), then what you have is garbage, not data. Therefore, we could call the art of shaping this into meaningful stories garbage science.
    - WastingMyTime89 4y ago
      Tell me in a comment you have never worked with business data in your life. Business data is full of minor inconsistencies which are not obvious until you sit in front of it. Products are sold by different units. Reporting ranges and aggregates are slightly different. Subsidiaries use categories which are close but not exactly identical. There is generally plenty of massaging to do before you can get the information you need.
SoftTalker 4y ago
> I’m sure it’s being taught somewhere was that you “need to let the data tell it’s own story”. AWS is pushing this message on every NFL broadcast with their Next Gen Stats ads. "The data tells us all"
extr 4y ago
I think a lot of times unless you are operating at hyperscale and bps matter, most business data analysis is more art than science. Ex, if you have < 200 sales in a month and you want to report this, it's totally normal to want to pull it into excel and fiddle with the numbers, make ad-hoc corrections, divide those 200 sales up in very subjective ways, etc. The smaller the company the more the reporting layer is only a weak approximation of reality, there isn't enough there to create a narrative around. "Hypothesis testing" such data is honestly a waste of time. Of course you want to be intellectually honest about it, and I agree that cherry-picking the "right" data can be a big problem in the wrong kind of organization. I remember one time a Jr analyst I managed was asked by the CEO to create a certain chart. I was not looped in. The CEO then used that chart to convince himself and many others on the executive team to make a huge product change that ended up being a complete disaster.
deleted 4y ago
[deleted]

axelf4 4y ago

As my professor said: All good statistics is done before you have looked at the data.

sideshowb 4y ago
But where do you get your hypothesis from?
- beckingz 4y ago
  your hypothetical model of reality. But that's a lot of work so businesses don't want to pay for it.
- Tams80 4y ago
  Whatever orifice you desire.
merouan 4y ago
True, for clean data. You can’t clean data without looking at it (a lot), though.
- vanviegen 4y ago
  If you have data that's so 'dirty' that you can't decide on the filtering rules in advance (or based on only historic data), then what you have is garbage, not data. Therefore, we could call the art of shaping this into meaningful stories garbage science.
  - WastingMyTime89 4y ago
    Tell me in a comment you have never worked with business data in your life. Business data is full of minor inconsistencies which are not obvious until you sit in front of it. Products are sold by different units. Reporting ranges and aggregates are slightly different. Subsidiaries use categories which are close but not exactly identical. There is generally plenty of massaging to do before you can get the information you need.

SoftTalker 4y ago

> I’m sure it’s being taught somewhere was that you “need to let the data tell it’s own story”. AWS is pushing this message on every NFL broadcast with their Next Gen Stats ads. "The data tells us all"

extr 4y ago

I think a lot of times unless you are operating at hyperscale and bps matter, most business data analysis is more art than science. Ex, if you have < 200 sales in a month and you want to report this, it's totally normal to want to pull it into excel and fiddle with the numbers, make ad-hoc corrections, divide those 200 sales up in very subjective ways, etc. The smaller the company the more the reporting layer is only a weak approximation of reality, there isn't enough there to create a narrative around. "Hypothesis testing" such data is honestly a waste of time. Of course you want to be intellectually honest about it, and I agree that cherry-picking the "right" data can be a big problem in the wrong kind of organization. I remember one time a Jr analyst I managed was asked by the CEO to create a certain chart. I was not looped in. The CEO then used that chart to convince himself and many others on the executive team to make a huge product change that ended up being a complete disaster.

deleted 4y ago

[deleted]