4 ms·

This is a pet peeve of mine at work. Any and I mean any statistic someone throws at me I will try and dig in. And if I'm able to, I will usually find that some

This is a pet peeve of mine at work. Any and I mean any statistic someone throws at me I will try and dig in. And if I'm able to, I will usually find that something is very wrong somewhere. As in, the underlying data is usually just wrong, invalidating the whole thing or the data is reasonably sound but the person doing the analysis is making incorrect assumptions about parts of the data and then drawing incorrect conclusions.

aschla 8mo ago
It seems to be an ever-present trait of modern business. There is no rigor, probably partly because most business professionals have never learned how to properly approach and analyze data. Can't tell you how many times I've seen product managers making decisions based on a few hundred analytics events, trying to glean insight where there is none.
- gyomu 8mo ago
  If what you're saying 1) is true and 2) does matter in the success of a business, then wouldn't anyone be able to displace an incumbent trivially by applying a bit of rigor? I think 1) holds (as my experience matches your cynicism :), but I have a feeling that data minded people tend to overestimate the importance of 2)...
  - laserlight 8mo ago
    > does matter in the success of a business In many experience, many of the statistics these people use doesn't matter in the success of a business --- they are vanity metrics. But people use statistics, and especially the wrong statistics, to pass their agenda. Regardless, it's important to fix the statistics.
  - mettamage 8mo ago
    Rigor helps for better insights about data. That can help for entrepreneurship. What also can help for entrepreneurship is having a bias for action. So even if your insights are wrong, if you act and keep acting you will keep acting then you will partially shape reality to your will and bend to its will. So there are certain forces where you can compensate for your lack of rigor. The best companies have both of those things by their side.
- p_v_doom 8mo ago
  Also rigor is slow. Looks like a waste of time. What are you optimizing all that code for, it works doesnt it? Dont let perfect be the enemy of good. If it works 80% thats enough, just push it. What is technical debt?
defrost 8mo ago
I've frequently found, over a few decades, that numerical systems are cyclically 'corrected' until results and performance match prior expectations. There are often more errors. Sometimes the actual results are wildly different in reality to what a model expects .. but the data treatment has been bug hunted until it does what was expected .. and then attention fades away.
- pprotas 8mo ago
  Or the company just changes the definition of success, so that the metrics (that used to be bad last quarter) are suddenly good
skywhopper 8mo ago
This is, unfortunately, a feature of a lot of these systems. The sponsors don’t want truth, they want validation. Generative AI means there don’t even have to be data engineers in the mix to create fake numbers.
riskable 8mo ago
> Any and I mean any statistic someone throws at me I will try and dig in. I bet you do this only 75% of the time.

aschla 8mo ago

It seems to be an ever-present trait of modern business. There is no rigor, probably partly because most business professionals have never learned how to properly approach and analyze data. Can't tell you how many times I've seen product managers making decisions based on a few hundred analytics events, trying to glean insight where there is none.

gyomu 8mo ago
If what you're saying 1) is true and 2) does matter in the success of a business, then wouldn't anyone be able to displace an incumbent trivially by applying a bit of rigor? I think 1) holds (as my experience matches your cynicism :), but I have a feeling that data minded people tend to overestimate the importance of 2)...
- laserlight 8mo ago
  > does matter in the success of a business In many experience, many of the statistics these people use doesn't matter in the success of a business --- they are vanity metrics. But people use statistics, and especially the wrong statistics, to pass their agenda. Regardless, it's important to fix the statistics.
- mettamage 8mo ago
  Rigor helps for better insights about data. That can help for entrepreneurship. What also can help for entrepreneurship is having a bias for action. So even if your insights are wrong, if you act and keep acting you will keep acting then you will partially shape reality to your will and bend to its will. So there are certain forces where you can compensate for your lack of rigor. The best companies have both of those things by their side.
p_v_doom 8mo ago
Also rigor is slow. Looks like a waste of time. What are you optimizing all that code for, it works doesnt it? Dont let perfect be the enemy of good. If it works 80% thats enough, just push it. What is technical debt?

defrost 8mo ago

I've frequently found, over a few decades, that numerical systems are cyclically 'corrected' until results and performance match prior expectations. There are often more errors. Sometimes the actual results are wildly different in reality to what a model expects .. but the data treatment has been bug hunted until it does what was expected .. and then attention fades away.

pprotas 8mo ago
Or the company just changes the definition of success, so that the metrics (that used to be bad last quarter) are suddenly good

skywhopper 8mo ago

This is, unfortunately, a feature of a lot of these systems. The sponsors don’t want truth, they want validation. Generative AI means there don’t even have to be data engineers in the mix to create fake numbers.

riskable 8mo ago

> Any and I mean any statistic someone throws at me I will try and dig in. I bet you do this only 75% of the time.