7 ms·

Is there data on the quality of management decisions?

178 points by luu 9y ago

thisisit 9y ago
> A statement I commonly hear in tech-utopian circles is that some seeming inefficiency can’t actually be inefficient because the market is efficient and inefficiencies will quickly be eliminated. At the risk of sounding like an old fogey, I believe a similar statement can be made for stats-utopian circles and their belief in ability to quantify every thing. As the post says: > One nice thing about sports is that they often have detailed play-by-play data and well-defined win criteria which lets us tell, on average, what the expected value of a decision is. So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value.
- 0xcde4c3db 9y ago
  > So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Exactly. The biggest problem with quantifying efficiency in particular is that you have to define the inputs and outputs. Your biases are embedded in that definition rather than in the calculation per se. You can make some pretty awful processes "efficient" if you assign zero value to some of the inputs, and conversely some pretty great processes "inefficient" if you assign zero value to some of the outputs.
  - Bartweiss 9y ago
    > You can make some pretty awful processes "efficient" if you assign zero value to some of the inputs, and conversely some pretty great processes "inefficient" if you assign zero value to some of the outputs. I'm halfway convinced that time-motion studies are some kind of demonic invention for exactly this reason. We can get 3% more productivity out of an assembly line, just by doubling the accident rate and giving every worker RSI! Goodhart's Law is dangerous even when we're making management and business decisions in good faith, but when ideas like "long term injuries are irrelevant" are implicitly added it becomes monstrous.
- pySSK 9y ago
  > So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Small aside but some including me would have a problem with even using sports as a proxy. There's a school of thought in soccer that maintains that the best match is a 0-0 draw [1][2][3]. While the data reflects a completely unproductive match, a 0-0 draw means that both sides performed admirably in defending and in countering the efforts of the other side. There is very detailed stats in soccer too now, but, that doesn't weaken the point. [1] http://lastwordonfootball.com/2017/09/09/five-famous-0-0-draws/ http://lastwordonfootball.com/2017/09/09/five-famous-0-0-dra... [2] https://www.theguardian.com/football/2007/aug/14/newsstory.sport14 https://www.theguardian.com/football/2007/aug/14/newsstory.s... [3] http://worldsoccertalk.com/2010/06/22/explaining-a-draw-to-the-non-soccer-crowd/ http://worldsoccertalk.com/2010/06/22/explaining-a-draw-to-t...
- andrioni 9y ago
  >So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Wouldn't it make his result even stronger? If the thesis fails to hold even in better conditions (like when there is a well-defined expected value), it should also fail to hold in worse conditions.
- Spooky23 9y ago
  Using sports, particularly baseball is a terrible proxy, as the core assumption is that each game is a finite event. That’s really not true at all — key decisions like whether to relieve a pitcher are not made within the context of a game. At a higher level, decisions to trade players are made that will affect outcome as well. As someone who manages projects, there are parallels from a context point of view to be drawn to baseball. Each project is a measure of success or failure, but my effectiveness isn’t defined by individual projects. I may choose to do a project less efficiently to control cash flow or prioritize something else.
- mcguire 9y ago
  Absolutely not. In sports, data-based, "rational" decisions are trivial. If poor decisions are made there, there is no chance that a decision in a less well defined field will not be hogwash.
ggambetta 9y ago
This looks super interesting. Unfortunately, I couldn't follow the explanation beyond the few first paragraphs because of so much baseball terminology and logic, which is completely alien to me :( The intro is confusing enough, but then there are terms that aren't defined, such as "stealing" - are readers supposed to be able to infer what this means just by context?
- csours 9y ago
  So the whole intro is to make the point that a certain type of play (base stealing) doesn't make sense, even for teams that do it well - and it took until 2012 to figure that out and get their players to stop doing that play.
  - AskewEgret 9y ago
    Ceteris paribus, base stealing doesn't make sense. But baseball is a game played by humans and even if it weren't, there are a large number of variables that don't go into the simple analysis. For example: The members of the defending team adopt fielding positions based on a maximum potential for fielding a hit ball. But if there is even a threat of a stolen base attempt, the defense must change their positioning accordingly, affecting their ability to field a ball put into play. If a player does make an attempt to steal a base, at least one defender must begin to change their positioning before the hitter has even had a chance to hit the ball. If they do not, they will not make it to the base in time to receive the throw from the catcher and either the throw will not occur or it will go out into the outfield and the runner may advance another base. Alternatively, the hitter may actually hit the ball and the ball may go to an area that it is statistically likely to go, where a defender would have been but for the stolen base attempt. What was to be an out is now a hit and the runner, having a head start because of their stolen base attempt, is able to advance further than they normally would have been able to. Because the hitter hit the ball, this situation does not get recorded in the statistics as a stolen base attempt! Baseball is really a fascinating game to think about!
    - ashark 9y ago
deleted 9y ago
[deleted]
Retric 9y ago
One issue with analysis like this is micro optimizations have more information than simplified abstractions. Consider, in the bottom of the 9th 1 or 2 runs may have identical value. So, reducing the total expected value of runs to increase your odds of a single run may be a very good trade off. And that's assuming you want to optimize total wins. Increasing your odds of winning the playoffs may reduce your odds of getting into them by having more higher paid people, but a lower average player. Further, maximizing making money over time is yet another option, which may promote showmanship over success.
- lodi 9y ago
  This is addressed in the article: "But there are situations where the approximation isn’t very good, such as when it’s the 9th inning and the game is tied. In that case, a decision that increases the probability of scoring 1 run but decreases the probability of scoring multiple runs is actually the right choice." I understood that the "full" model includes these considerations.
  - Retric 9y ago
    I was bringing up that edge case specifically because 'winning' is the obvious optimization, but not necessarily the correct one. Bottom of the 9th bases are loaded and the guy at bat hit's a home run, that's the kind of thing that sticks with people and makes more money in the long run than a bunt that get 1 run.
ihsw2 9y ago
The problem is externalities -- both positive and negative externalities affect the outcome and it is basically impossible to determine whether an outcome occurred due to management decision or due to an externality (eg: revenue growth due to successful execution or revenue growth due to changing business cycles).
- bweis 9y ago
  Couldn't agree more. It's really hard to pinpoint the exact cause of such an outcome.
jackgolding 9y ago
This article is quite interesting for the isolated baseball analysis but is ultimately a long-winded discussion (which goes down a sports rabbit hole which ultimately doesn't contribute to the argument.) The TLDR is: > If we want to look at the quality of decision making, it’s too simplistic to say that we expect a firm to make good decisions because they’re exposed to markets and there’s economic value in making good decisions and people within the firm will probably be rewarded greatly if they make good decisions. Is there actually anything a reader here can take from this writing other than the assertion I quoted above (which is the author's point of view and isn't proven in nearly as much rigour as the baseball analysis is.)
- geofft 9y ago
  I think the argument holds up: it is that, even in a field where management decisions can be clearly quantified, the data is collected, and people are doing the analysis, bad decisions persist (management makes bad decisions when they could make good decisions, and the market does not value good management decisions). So in a field where it is much less clearly quantified, why would we expect the market to ensure high-quality management decisions? I don't think the post proves that the market fails to incentivize good decisions. That's a hard thing to prove. It does certainly give us serious reason to doubt the common claim that the market succeeds in incentivizing good decisions on a short timescale. (It might still be true, but we should have much less confidence in that statement being true than before we read this post.)
  - jackgolding 9y ago
    > It does certainly give us serious reason to doubt the common claim that the market succeeds in incentivizing good decisions on a short timescale. How does the article suggest this?
    - geofft 9y ago
      The claim "the market succeeds in incentivizing good decisions on a short timescale" is at odds with the article's demonstrated conclusion "the market fails to incentivize good decisions on a short timescale in the field of baseball". So you're left with two possible ways to resolve this: either baseball is unique and the market doesn't work properly there for reasons that don't generalize (but the article gives some reasons to believe that this isn't the right way to resolve the contradiction), or baseball isn't special and non-baseball markets very likely do similar things.
jerf 9y ago
"A statement I commonly hear in tech-utopian circles is that some seeming inefficiency can’t actually be inefficient because the market is efficient and inefficiencies will quickly be eliminated." There's equivocation in that statement; what market efficiency is and what people generally mean by efficiency are not the same thing. This is not an accusation of danluu, though, as I believe it is an accurate statement of an error made by a lot of people. Market efficiency is really a rather boring statement about the difficulty of performing arbitrage in the long term. Plus, given that the market is not absolutely efficient we should expect errors to be made. But we can do some numerical analysis here. Consider the range of possibilities. Suppose population X is paid 0% of what everyone else is paid in the market, for the same output. It would take an astonishingly-large, an implausibly large, countervailing force to prevent some company from swooping in and claiming that free labor. (Remember, even if 99.9% of the firms on the market choose not to, it just means that the .1% is going have that much easier of a time.) However, for group X to be paid 99.9% of what everyone else is paid on average doesn't take much at all; indeed, mere noise could produce that result. Rather than it being a question of "is or is not discrimination", this sort of analysis can put numbers on how much, from which you can start analyzing things like the value of certain changes vs. their costs. No conclusion; I'm just musing here. Except that I think that people generally do a lot of binary thinking here when quantitative might be more useful.
- rossdavidh 9y ago
  "There's equivocation in that statement; what market efficiency is and what people generally mean by efficiency are not the same thing" True, but: it's no coincidence that the terms are used for different things. What we really care about as humans is "market efficiency" in the ordinary English sense of the word. That's really hard to define, though, so economists redefine it in a very precise, technical, and yes, 'boring' statement. Now it is easier to see if we have it, but also not very useful. However, it can still be used to argue that any intervention in the market will be a mistake. The confusion about what the term "market efficiency" means is not just a coincidence. Redefine it as something easily proven, and then let people mistake that for a proof of what they actually care about, and you get the same political/lobbying result as the (rather more difficult, perhaps impossible) task of proving that markets are efficient in the way we actually care about as humans, rather than just economists.
monkeypizza 9y ago
This is the same topic of Eliezer Yudkowsky's recent book "Inadequate Equilibria" [0], reviewed by Scott Aaronson here [1]. That book does a good job explaining why not every inefficiency can be eliminated - because getting out of a bad nash equilibrium requires coordinated changes, and for large enough groups you can't convince enough people to make a change at the same time. 0: https://equilibriabook.com/toc/ https://equilibriabook.com/toc/ 1: https://www.scottaaronson.com/blog/?p=3535 https://www.scottaaronson.com/blog/?p=3535
- madhadron 9y ago
  Or you could read short book on game theory, think for ten minutes about perverse incentives, and get to the same results without the conceptual diarrhea that Yudkowsky insists on. Why he had to try to drag a concept of free energy into it with no justification, I don't know.
  - daveguy 9y ago
    "Read a short book on..." Is a pretty high bar for conceptual simplicity. Why wouldn't those who have had exposure to the concept of free energy prefer that metaphor to yours? You may be biased having exposure to more game theory than free energy. One person's conceptual diarrhea is another person's cleanest explanation.
    - alasdair_ 9y ago
      >"Read a short book on..." Is a pretty high bar for conceptual simplicity. Take five minutes and read the wikipedia page on the prisoner's dilemma then. Even in a two player game, it's clear why individually optimizing for personal gain leads to poor outcomes for everyone, yet if there was a way to force people to act in a particular way, the total outcome for everyone would be better for all.
      - ucaetano 9y ago
        Except that the one forcing people will inevitably force people to act in a way that maximizes his/her personal gain.
inputcoffee 9y ago
There are three issues I have with this: 1. "Quality" for management decisions are notoriously ill defined for the same reason that we never know if we did the right thing with the economy: the experiments can never be repeated. 2. The example serves to explain what a decision is and what a good decision would be, but that is not the concept that needs explanation! 3. "some seeming inefficiency can’t actually be inefficient because the market is efficient and inefficiencies will quickly be eliminated." is not the right statement of efficient markets. Rather, the act of trying to make it more efficient is exactly what makes it efficient. So if you see an opportunity, someone exploits it and thus makes it more efficient.
- philipov 9y ago
  4. Markets aren't actually efficient because they are stacked in favor of incumbents through their ability to lobby for anti-competitive legislation, as can be seen with the perversion of the copyright and patent systems.
  - mulmen 9y ago
    The mistake you made in this example is attributing an action in the legislative system as a failure in the market.
    - philipov 9y ago
      I am saying the opposite: the market becomes less efficient as a result of legislative capture. But it is a feedback loop, so attributing the dynamics to simple cause and effect is not appropriate.
  - Spooky23 9y ago
    Markets aren’t going to deliver best value to anyone without being forced to. Eventually, every company wants to own the market, and convincing government to do so is a key part of that strategy. The invisible hand sometimes is holding a visible club.
- lucas_membrane 9y ago
scott_s 9y ago
An aside to the main point: the author claims that one can do a similar analysis to other sports. My intuition is that you can't do quite as accurate an analysis of, say, soccer, than you can baseball. As a sport, baseball is unusually discretized when compared to a much more continuous sport like soccer or hockey. Being discrete, it makes it much easier to represent as a state machine, which makes it easier to simulate, and easier to suss out cause and effect. American football, I think, may be somewhere inbetween. I know, of course, that other sports do significant statistical analysis, but my intuition is that it is harder to attribute cause and effect at the fine-grain level that one can with baseball.
ksk 9y ago
>A less contentious example is that when you see a big company doing something that seems bizarrely inefficient, maybe it’s not inefficient and you just lack the information necessary to understand why the decision was efficient. This pretty much sums up a lot of tech news and the associated comments on such news.
- analog31 9y ago
  Indeed, and likewise when someone does something that seems irrational.
  - taneq 9y ago
    I've long said that if the way someone's behaving doesn't make sense, you don't understand their motivations. Now, it may be that their motivations are all messed up and they're acting the way they are because sabotaging their relationships reaffirms their core belief that they're unlovable. But it does make sense once you understand the framework they're working in.
hatcherdogg 9y ago
Outstanding read.
deleted 9y ago
[deleted]
Joeri 9y ago
From what I can gather making good management decisions is a lot like making a good estimate for a software project. There are two ways to go about it: (A) do a lot of work to break down the problem to manageable parts, put a well-reasoned number on each part, then sum up the parts with the necessary tolerances and risk factors, or (B) just go with your gut. I see a lot of (B). Hilarity ensues.
- deepsun 9y ago
  Speed is very important in business. How much time would it take to estimate how much time it would take to analyze and estimate the management decision in question?
- kinkrtyavimoodh 9y ago
  What is gut if not an accumulation of experiences? If an engineering-manager-with-10-years-experience's gut produces the same accuracy as a fresh grad's, then it's not of much use, and there's a good chance that the manager would generate a bad estimate even following (A). That's not to say that people shouldn't analyze. Just saying that gut is not random guesswork.
  - mcguire 9y ago
    Mostly, it's answering a simpler question than was asked. See Thinking, Fast and Slow.
- abhiyerra 9y ago
  I feel like 80/20 Rule works here. Describe the why, what, and 20% of the how that answers 80% of how to do what. The other 80% that answers the last 20% should rest on employee. That way you are not micromanaging and give way too human creativity.
- mcguire 9y ago
  "There are two ways to make decisions. One is bad, the other is worse. No one knows which is which."
bluetwo 9y ago
All companies make both good and bad decisions. They don't fall apart when they make a single bad decision. They fall apart when he sum total of the impact of their bad decisions outweighs the sum total of the impact of their good decisions. When this happens, the last bad decision usually takes the blame, however it is rarely the true cause.
sokoloff 9y ago
> we see that the average stolen base is worth -.467 runs and the average successful steal is worth .175 runs. First clause should be "average caught stealing"
marcus_holmes 9y ago
Betteridge's Law strikes again [0]. You could replace that entire post with "No. Unlike baseball." Not that the post wasn't interesting, of course. To me, it leads to the inevitable conclusion that we need to record failures as well as wins in order to get accurate data on management decisions. 0: https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
mcguire 9y ago
Tl;dr: Many organizations make decisions like a drunken madman whose seeing-eye dog just ate his hearing aid. For reasons why, see Thinking, Fast and Slow (just reading it now; it's quite good) and Thaler's Misbehaving.
ataturk 9y ago
If there's one thing about management it's that success is fully theirs and failures are trickle down to everyone else. I worked on a major and successful product launch that made a big company big bucks and watched as none of the promotions and bonuses trickled down past management despite the untold hours and sacrifices by the development team. When the company decided there were too many smart people, they created disincentives to staying around and subsequently all those bright people got the hint and found other jobs. It was more humane than a layoff, I suppose, but so so deceitful and cynical. And so it goes.

178 points by luu 9y ago

> A statement I commonly hear in tech-utopian circles is that some seeming inefficiency can’t actually be inefficient because the market is efficient and inefficiencies will quickly be eliminated. At the risk of sounding like an old fogey, I believe a similar statement can be made for stats-utopian circles and their belief in ability to quantify every thing. As the post says: > One nice thing about sports is that they often have detailed play-by-play data and well-defined win criteria which lets us tell, on average, what the expected value of a decision is. So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value.

0xcde4c3db 9y ago
> So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Exactly. The biggest problem with quantifying efficiency in particular is that you have to define the inputs and outputs. Your biases are embedded in that definition rather than in the calculation per se. You can make some pretty awful processes "efficient" if you assign zero value to some of the inputs, and conversely some pretty great processes "inefficient" if you assign zero value to some of the outputs.
- Bartweiss 9y ago
  > You can make some pretty awful processes "efficient" if you assign zero value to some of the inputs, and conversely some pretty great processes "inefficient" if you assign zero value to some of the outputs. I'm halfway convinced that time-motion studies are some kind of demonic invention for exactly this reason. We can get 3% more productivity out of an assembly line, just by doubling the accident rate and giving every worker RSI! Goodhart's Law is dangerous even when we're making management and business decisions in good faith, but when ideas like "long term injuries are irrelevant" are implicitly added it becomes monstrous.
pySSK 9y ago
> So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Small aside but some including me would have a problem with even using sports as a proxy. There's a school of thought in soccer that maintains that the best match is a 0-0 draw [1][2][3]. While the data reflects a completely unproductive match, a 0-0 draw means that both sides performed admirably in defending and in countering the efforts of the other side. There is very detailed stats in soccer too now, but, that doesn't weaken the point. [1] http://lastwordonfootball.com/2017/09/09/five-famous-0-0-draws/ http://lastwordonfootball.com/2017/09/09/five-famous-0-0-dra... [2] https://www.theguardian.com/football/2007/aug/14/newsstory.sport14 https://www.theguardian.com/football/2007/aug/14/newsstory.s... [3] http://worldsoccertalk.com/2010/06/22/explaining-a-draw-to-the-non-soccer-crowd/ http://worldsoccertalk.com/2010/06/22/explaining-a-draw-to-t...
andrioni 9y ago
>So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value. Wouldn't it make his result even stronger? If the thesis fails to hold even in better conditions (like when there is a well-defined expected value), it should also fail to hold in worse conditions.
Spooky23 9y ago
Using sports, particularly baseball is a terrible proxy, as the core assumption is that each game is a finite event. That’s really not true at all — key decisions like whether to relieve a pitcher are not made within the context of a game. At a higher level, decisions to trade players are made that will affect outcome as well. As someone who manages projects, there are parallels from a context point of view to be drawn to baseball. Each project is a measure of success or failure, but my effectiveness isn’t defined by individual projects. I may choose to do a project less efficiently to control cash flow or prioritize something else.
mcguire 9y ago
Absolutely not. In sports, data-based, "rational" decisions are trivial. If poor decisions are made there, there is no chance that a decision in a less well defined field will not be hogwash.

ggambetta 9y ago

This looks super interesting. Unfortunately, I couldn't follow the explanation beyond the few first paragraphs because of so much baseball terminology and logic, which is completely alien to me :( The intro is confusing enough, but then there are terms that aren't defined, such as "stealing" - are readers supposed to be able to infer what this means just by context?

csours 9y ago
So the whole intro is to make the point that a certain type of play (base stealing) doesn't make sense, even for teams that do it well - and it took until 2012 to figure that out and get their players to stop doing that play.
- AskewEgret 9y ago
  Ceteris paribus, base stealing doesn't make sense. But baseball is a game played by humans and even if it weren't, there are a large number of variables that don't go into the simple analysis. For example: The members of the defending team adopt fielding positions based on a maximum potential for fielding a hit ball. But if there is even a threat of a stolen base attempt, the defense must change their positioning accordingly, affecting their ability to field a ball put into play. If a player does make an attempt to steal a base, at least one defender must begin to change their positioning before the hitter has even had a chance to hit the ball. If they do not, they will not make it to the base in time to receive the throw from the catcher and either the throw will not occur or it will go out into the outfield and the runner may advance another base. Alternatively, the hitter may actually hit the ball and the ball may go to an area that it is statistically likely to go, where a defender would have been but for the stolen base attempt. What was to be an out is now a hit and the runner, having a head start because of their stolen base attempt, is able to advance further than they normally would have been able to. Because the hitter hit the ball, this situation does not get recorded in the statistics as a stolen base attempt! Baseball is really a fascinating game to think about!
  - ashark 9y ago

deleted 9y ago

[deleted]

Retric 9y ago

One issue with analysis like this is micro optimizations have more information than simplified abstractions. Consider, in the bottom of the 9th 1 or 2 runs may have identical value. So, reducing the total expected value of runs to increase your odds of a single run may be a very good trade off. And that's assuming you want to optimize total wins. Increasing your odds of winning the playoffs may reduce your odds of getting into them by having more higher paid people, but a lower average player. Further, maximizing making money over time is yet another option, which may promote showmanship over success.

lodi 9y ago
This is addressed in the article: "But there are situations where the approximation isn’t very good, such as when it’s the 9th inning and the game is tied. In that case, a decision that increases the probability of scoring 1 run but decreases the probability of scoring multiple runs is actually the right choice." I understood that the "full" model includes these considerations.
- Retric 9y ago
  I was bringing up that edge case specifically because 'winning' is the obvious optimization, but not necessarily the correct one. Bottom of the 9th bases are loaded and the guy at bat hit's a home run, that's the kind of thing that sticks with people and makes more money in the long run than a bunt that get 1 run.

ihsw2 9y ago

The problem is externalities -- both positive and negative externalities affect the outcome and it is basically impossible to determine whether an outcome occurred due to management decision or due to an externality (eg: revenue growth due to successful execution or revenue growth due to changing business cycles).

bweis 9y ago
Couldn't agree more. It's really hard to pinpoint the exact cause of such an outcome.

jackgolding 9y ago

This article is quite interesting for the isolated baseball analysis but is ultimately a long-winded discussion (which goes down a sports rabbit hole which ultimately doesn't contribute to the argument.) The TLDR is: > If we want to look at the quality of decision making, it’s too simplistic to say that we expect a firm to make good decisions because they’re exposed to markets and there’s economic value in making good decisions and people within the firm will probably be rewarded greatly if they make good decisions. Is there actually anything a reader here can take from this writing other than the assertion I quoted above (which is the author's point of view and isn't proven in nearly as much rigour as the baseball analysis is.)

geofft 9y ago
I think the argument holds up: it is that, even in a field where management decisions can be clearly quantified, the data is collected, and people are doing the analysis, bad decisions persist (management makes bad decisions when they could make good decisions, and the market does not value good management decisions). So in a field where it is much less clearly quantified, why would we expect the market to ensure high-quality management decisions? I don't think the post proves that the market fails to incentivize good decisions. That's a hard thing to prove. It does certainly give us serious reason to doubt the common claim that the market succeeds in incentivizing good decisions on a short timescale. (It might still be true, but we should have much less confidence in that statement being true than before we read this post.)
- jackgolding 9y ago
  > It does certainly give us serious reason to doubt the common claim that the market succeeds in incentivizing good decisions on a short timescale. How does the article suggest this?
  - geofft 9y ago
    The claim "the market succeeds in incentivizing good decisions on a short timescale" is at odds with the article's demonstrated conclusion "the market fails to incentivize good decisions on a short timescale in the field of baseball". So you're left with two possible ways to resolve this: either baseball is unique and the market doesn't work properly there for reasons that don't generalize (but the article gives some reasons to believe that this isn't the right way to resolve the contradiction), or baseball isn't special and non-baseball markets very likely do similar things.

jerf 9y ago

"A statement I commonly hear in tech-utopian circles is that some seeming inefficiency can’t actually be inefficient because the market is efficient and inefficiencies will quickly be eliminated." There's equivocation in that statement; what market efficiency is and what people generally mean by efficiency are not the same thing. This is not an accusation of danluu, though, as I believe it is an accurate statement of an error made by a lot of people. Market efficiency is really a rather boring statement about the difficulty of performing arbitrage in the long term. Plus, given that the market is not absolutely efficient we should expect errors to be made. But we can do some numerical analysis here. Consider the range of possibilities. Suppose population X is paid 0% of what everyone else is paid in the market, for the same output. It would take an astonishingly-large, an implausibly large, countervailing force to prevent some company from swooping in and claiming that free labor. (Remember, even if 99.9% of the firms on the market choose not to, it just means that the .1% is going have that much easier of a time.) However, for group X to be paid 99.9% of what everyone else is paid on average doesn't take much at all; indeed, mere noise could produce that result. Rather than it being a question of "is or is not discrimination", this sort of analysis can put numbers on how much, from which you can start analyzing things like the value of certain changes vs. their costs. No conclusion; I'm just musing here. Except that I think that people generally do a lot of binary thinking here when quantitative might be more useful.

rossdavidh 9y ago
"There's equivocation in that statement; what market efficiency is and what people generally mean by efficiency are not the same thing" True, but: it's no coincidence that the terms are used for different things. What we really care about as humans is "market efficiency" in the ordinary English sense of the word. That's really hard to define, though, so economists redefine it in a very precise, technical, and yes, 'boring' statement. Now it is easier to see if we have it, but also not very useful. However, it can still be used to argue that any intervention in the market will be a mistake. The confusion about what the term "market efficiency" means is not just a coincidence. Redefine it as something easily proven, and then let people mistake that for a proof of what they actually care about, and you get the same political/lobbying result as the (rather more difficult, perhaps impossible) task of proving that markets are efficient in the way we actually care about as humans, rather than just economists.

monkeypizza 9y ago

This is the same topic of Eliezer Yudkowsky's recent book "Inadequate Equilibria" [0], reviewed by Scott Aaronson here [1]. That book does a good job explaining why not every inefficiency can be eliminated - because getting out of a bad nash equilibrium requires coordinated changes, and for large enough groups you can't convince enough people to make a change at the same time. 0: https://equilibriabook.com/toc/ https://equilibriabook.com/toc/ 1: https://www.scottaaronson.com/blog/?p=3535 https://www.scottaaronson.com/blog/?p=3535

madhadron 9y ago
Or you could read short book on game theory, think for ten minutes about perverse incentives, and get to the same results without the conceptual diarrhea that Yudkowsky insists on. Why he had to try to drag a concept of free energy into it with no justification, I don't know.
- daveguy 9y ago
  "Read a short book on..." Is a pretty high bar for conceptual simplicity. Why wouldn't those who have had exposure to the concept of free energy prefer that metaphor to yours? You may be biased having exposure to more game theory than free energy. One person's conceptual diarrhea is another person's cleanest explanation.
  - alasdair_ 9y ago
    >"Read a short book on..." Is a pretty high bar for conceptual simplicity. Take five minutes and read the wikipedia page on the prisoner's dilemma then. Even in a two player game, it's clear why individually optimizing for personal gain leads to poor outcomes for everyone, yet if there was a way to force people to act in a particular way, the total outcome for everyone would be better for all.
    - ucaetano 9y ago
      Except that the one forcing people will inevitably force people to act in a way that maximizes his/her personal gain.

inputcoffee 9y ago

There are three issues I have with this: 1. "Quality" for management decisions are notoriously ill defined for the same reason that we never know if we did the right thing with the economy: the experiments can never be repeated. 2. The example serves to explain what a decision is and what a good decision would be, but that is not the concept that needs explanation! 3. "some seeming inefficiency can’t actually be inefficient because the market is efficient and inefficiencies will quickly be eliminated." is not the right statement of efficient markets. Rather, the act of trying to make it more efficient is exactly what makes it efficient. So if you see an opportunity, someone exploits it and thus makes it more efficient.

philipov 9y ago
4. Markets aren't actually efficient because they are stacked in favor of incumbents through their ability to lobby for anti-competitive legislation, as can be seen with the perversion of the copyright and patent systems.
- mulmen 9y ago
  The mistake you made in this example is attributing an action in the legislative system as a failure in the market.
  - philipov 9y ago
    I am saying the opposite: the market becomes less efficient as a result of legislative capture. But it is a feedback loop, so attributing the dynamics to simple cause and effect is not appropriate.
- Spooky23 9y ago
  Markets aren’t going to deliver best value to anyone without being forced to. Eventually, every company wants to own the market, and convincing government to do so is a key part of that strategy. The invisible hand sometimes is holding a visible club.
lucas_membrane 9y ago

scott_s 9y ago

An aside to the main point: the author claims that one can do a similar analysis to other sports. My intuition is that you can't do quite as accurate an analysis of, say, soccer, than you can baseball. As a sport, baseball is unusually discretized when compared to a much more continuous sport like soccer or hockey. Being discrete, it makes it much easier to represent as a state machine, which makes it easier to simulate, and easier to suss out cause and effect. American football, I think, may be somewhere inbetween. I know, of course, that other sports do significant statistical analysis, but my intuition is that it is harder to attribute cause and effect at the fine-grain level that one can with baseball.

ksk 9y ago

>A less contentious example is that when you see a big company doing something that seems bizarrely inefficient, maybe it’s not inefficient and you just lack the information necessary to understand why the decision was efficient. This pretty much sums up a lot of tech news and the associated comments on such news.

analog31 9y ago
Indeed, and likewise when someone does something that seems irrational.
- taneq 9y ago
  I've long said that if the way someone's behaving doesn't make sense, you don't understand their motivations. Now, it may be that their motivations are all messed up and they're acting the way they are because sabotaging their relationships reaffirms their core belief that they're unlovable. But it does make sense once you understand the framework they're working in.

hatcherdogg 9y ago

Outstanding read.

deleted 9y ago

[deleted]

Joeri 9y ago

From what I can gather making good management decisions is a lot like making a good estimate for a software project. There are two ways to go about it: (A) do a lot of work to break down the problem to manageable parts, put a well-reasoned number on each part, then sum up the parts with the necessary tolerances and risk factors, or (B) just go with your gut. I see a lot of (B). Hilarity ensues.

deepsun 9y ago
Speed is very important in business. How much time would it take to estimate how much time it would take to analyze and estimate the management decision in question?
kinkrtyavimoodh 9y ago
What is gut if not an accumulation of experiences? If an engineering-manager-with-10-years-experience's gut produces the same accuracy as a fresh grad's, then it's not of much use, and there's a good chance that the manager would generate a bad estimate even following (A). That's not to say that people shouldn't analyze. Just saying that gut is not random guesswork.
- mcguire 9y ago
  Mostly, it's answering a simpler question than was asked. See Thinking, Fast and Slow.
abhiyerra 9y ago
I feel like 80/20 Rule works here. Describe the why, what, and 20% of the how that answers 80% of how to do what. The other 80% that answers the last 20% should rest on employee. That way you are not micromanaging and give way too human creativity.
mcguire 9y ago
"There are two ways to make decisions. One is bad, the other is worse. No one knows which is which."

bluetwo 9y ago

All companies make both good and bad decisions. They don't fall apart when they make a single bad decision. They fall apart when he sum total of the impact of their bad decisions outweighs the sum total of the impact of their good decisions. When this happens, the last bad decision usually takes the blame, however it is rarely the true cause.

sokoloff 9y ago

> we see that the average stolen base is worth -.467 runs and the average successful steal is worth .175 runs. First clause should be "average caught stealing"

marcus_holmes 9y ago

Betteridge's Law strikes again [0]. You could replace that entire post with "No. Unlike baseball." Not that the post wasn't interesting, of course. To me, it leads to the inevitable conclusion that we need to record failures as well as wins in order to get accurate data on management decisions. 0: https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...

mcguire 9y ago

Tl;dr: Many organizations make decisions like a drunken madman whose seeing-eye dog just ate his hearing aid. For reasons why, see Thinking, Fast and Slow (just reading it now; it's quite good) and Thaler's Misbehaving.

ataturk 9y ago

If there's one thing about management it's that success is fully theirs and failures are trickle down to everyone else. I worked on a major and successful product launch that made a big company big bucks and watched as none of the promotions and bonuses trickled down past management despite the untold hours and sacrifices by the development team. When the company decided there were too many smart people, they created disincentives to staying around and subsequently all those bright people got the hint and found other jobs. It was more humane than a layoff, I suppose, but so so deceitful and cynical. And so it goes.