4 ms·

> A statement I commonly hear in tech-utopian circles is that some seeming inefficiency can’t actually be inefficient because the market is efficient and ineffi

> A statement I commonly hear in tech-utopian circles is that some seeming inefficiency can’t actually be inefficient because the market is efficient and inefficiencies will quickly be eliminated. At the risk of sounding like an old fogey, I believe a similar statement can be made for stats-utopian circles and their belief in ability to quantify every thing. As the post says: > One nice thing about sports is that they often have detailed play-by-play data and well-defined win criteria which lets us tell, on average, what the expected value of a decision is. So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value.

0xcde4c3db 9y ago
> So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Exactly. The biggest problem with quantifying efficiency in particular is that you have to define the inputs and outputs. Your biases are embedded in that definition rather than in the calculation per se. You can make some pretty awful processes "efficient" if you assign zero value to some of the inputs, and conversely some pretty great processes "inefficient" if you assign zero value to some of the outputs.
- Bartweiss 9y ago
  > You can make some pretty awful processes "efficient" if you assign zero value to some of the inputs, and conversely some pretty great processes "inefficient" if you assign zero value to some of the outputs. I'm halfway convinced that time-motion studies are some kind of demonic invention for exactly this reason. We can get 3% more productivity out of an assembly line, just by doubling the accident rate and giving every worker RSI! Goodhart's Law is dangerous even when we're making management and business decisions in good faith, but when ideas like "long term injuries are irrelevant" are implicitly added it becomes monstrous.
pySSK 9y ago
> So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Small aside but some including me would have a problem with even using sports as a proxy. There's a school of thought in soccer that maintains that the best match is a 0-0 draw [1][2][3]. While the data reflects a completely unproductive match, a 0-0 draw means that both sides performed admirably in defending and in countering the efforts of the other side. There is very detailed stats in soccer too now, but, that doesn't weaken the point. [1] http://lastwordonfootball.com/2017/09/09/five-famous-0-0-draws/ http://lastwordonfootball.com/2017/09/09/five-famous-0-0-dra... [2] https://www.theguardian.com/football/2007/aug/14/newsstory.sport14 https://www.theguardian.com/football/2007/aug/14/newsstory.s... [3] http://worldsoccertalk.com/2010/06/22/explaining-a-draw-to-the-non-soccer-crowd/ http://worldsoccertalk.com/2010/06/22/explaining-a-draw-to-t...
andrioni 9y ago
>So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Wouldn't it make his result even stronger? If the thesis fails to hold even in better conditions (like when there is a well-defined expected value), it should also fail to hold in worse conditions.
Spooky23 9y ago
Using sports, particularly baseball is a terrible proxy, as the core assumption is that each game is a finite event. That’s really not true at all — key decisions like whether to relieve a pitcher are not made within the context of a game. At a higher level, decisions to trade players are made that will affect outcome as well. As someone who manages projects, there are parallels from a context point of view to be drawn to baseball. Each project is a measure of success or failure, but my effectiveness isn’t defined by individual projects. I may choose to do a project less efficiently to control cash flow or prioritize something else.
mcguire 9y ago
Absolutely not. In sports, data-based, "rational" decisions are trivial. If poor decisions are made there, there is no chance that a decision in a less well defined field will not be hogwash.

0xcde4c3db 9y ago

> So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Exactly. The biggest problem with quantifying efficiency in particular is that you have to define the inputs and outputs. Your biases are embedded in that definition rather than in the calculation per se. You can make some pretty awful processes "efficient" if you assign zero value to some of the inputs, and conversely some pretty great processes "inefficient" if you assign zero value to some of the outputs.

Bartweiss 9y ago
> You can make some pretty awful processes "efficient" if you assign zero value to some of the inputs, and conversely some pretty great processes "inefficient" if you assign zero value to some of the outputs. I'm halfway convinced that time-motion studies are some kind of demonic invention for exactly this reason. We can get 3% more productivity out of an assembly line, just by doubling the accident rate and giving every worker RSI! Goodhart's Law is dangerous even when we're making management and business decisions in good faith, but when ideas like "long term injuries are irrelevant" are implicitly added it becomes monstrous.

pySSK 9y ago

> So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Small aside but some including me would have a problem with even using sports as a proxy. There's a school of thought in soccer that maintains that the best match is a 0-0 draw [1][2][3]. While the data reflects a completely unproductive match, a 0-0 draw means that both sides performed admirably in defending and in countering the efforts of the other side. There is very detailed stats in soccer too now, but, that doesn't weaken the point. [1] http://lastwordonfootball.com/2017/09/09/five-famous-0-0-draws/ http://lastwordonfootball.com/2017/09/09/five-famous-0-0-dra... [2] https://www.theguardian.com/football/2007/aug/14/newsstory.sport14 https://www.theguardian.com/football/2007/aug/14/newsstory.s... [3] http://worldsoccertalk.com/2010/06/22/explaining-a-draw-to-the-non-soccer-crowd/ http://worldsoccertalk.com/2010/06/22/explaining-a-draw-to-t...

andrioni 9y ago

>So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Wouldn't it make his result even stronger? If the thesis fails to hold even in better conditions (like when there is a well-defined expected value), it should also fail to hold in worse conditions.

Spooky23 9y ago

Using sports, particularly baseball is a terrible proxy, as the core assumption is that each game is a finite event. That’s really not true at all — key decisions like whether to relieve a pitcher are not made within the context of a game. At a higher level, decisions to trade players are made that will affect outcome as well. As someone who manages projects, there are parallels from a context point of view to be drawn to baseball. Each project is a measure of success or failure, but my effectiveness isn’t defined by individual projects. I may choose to do a project less efficiently to control cash flow or prioritize something else.

mcguire 9y ago

Absolutely not. In sports, data-based, "rational" decisions are trivial. If poor decisions are made there, there is no chance that a decision in a less well defined field will not be hogwash.