3 ms·

This is one of the intractable problems of human society. There is no optimal solution: Consolidation gives you consistency, network effects, and economies of

This is one of the intractable problems of human society. There is no optimal solution: Consolidation gives you consistency, network effects, and economies of scale. But it also gives you monocultures, stagnation, and overfitting. Heterogeneity and competition gives you incentives to optimize, innovation, and robustness. But it also gives you redundancy, knowledge loss, and inefficiency.

api 5y ago
I think it all boils down to overfitting. “Efficiency is overfitting.”
fastball 5y ago
Technically there probably is an optimal solution, if you decide on a system for valuing each of those positives/negatives. Once you manage that, there's almost certainly a point on the curve that gives you the highest total value. And yes, it is probably impossible to be objective here, but a subjective optimal solution can be helpful too.
- angry_octet 5y ago
  And I'd argue that we have chosen such an optimal solution, where robustness is given a low weighting, and now we are experiencing a predicted downtime event. The cost of this event is spread across the whole economy, and the executives (in industry and govt) who made the decisions which led to this will experience no downside personally. Executives at rivals who have just as bad an IT problem will receive bonuses. Essentially, we have a very weak correction incentive. Until capable hackers are endemic and ransomware hits everyone, there is little likelihood of change. Next week it could be RJR Nabisco, or Synnex/TurboTax/DuPont, or just 10,000 variants of {tiny $20m company that makes software for auditing sewerage pipe and is used by 370 of the 3000 counties in the US, and every client gets the DB corrupted}.
  - munificent 5y ago
    I think we are constantly tweaking things over time. For much of human history, consolidation was natural but the relative simplicity of technology kept barriers of entry low which enabled competition and new entrants. Also, poorer logistics and transportation meant you have many smaller semi-independent markets instead of fewer monolithic ones. Industrialization and powered transportation upended that and gave a lot of power to consolidation. That led to the era of robber barons. That power concentration was so bad for society that eventually the labor and antitrust movements came around to somewhat correct it. I think now we're seeing another oscillation. Software creates huge network effects and economies of scale. If a business writes a single program once, they can run it on as many servers or sell it to as many users as possible. Services that let users interact benefit exponentially from network effects. AI generates a lot of value, but requires extremely large datasets that only the largest corporations have access to. We are essentially in the era of digital robber barons right now. Six of the top ten richest people in the world according to Forbes got there through software. (Arguably seven if you consider Musk's wealth to be software-driven.) We are clearly nowhere near the optimal point on the consolidation continuum. With luck, we'll get something like an "information labor movement" and more teeth in our antitrust regulation to correct that.
- munificent 5y ago
  Sure, there is an optimal meta-solution if you're able to define which set of trade-offs represent a "winning" solution. But that's different from other problems where you can simply try to optimize the result itself directly. For example, we probably don't need to have hard discussions of trade-offs when it comes to, I don't know, infant mortality. There's almost no upside to babies dying (assuming you aren't heartless enough to consider less overpopulation to be an "upside"). But with business size, there are many desirable factors and improving any of them reduces another directly opposed but also desireable factor. Competition is good. Cooperation is also good. Increasing one by definition lowers the other.
SkyPuncher 5y ago
> Consolidation gives you consistency, network effects, and economies of scale. But it also gives you monocultures, stagnation, and overfitting. I very much enjoy the fact that I can go to nearly any city in America and they'll have a few dozen well-known stores with recognizable product. There are certainly bad forms of consolidation, but it's easy to overlook the good forms.
- Red_Leaves_Flyy 5y ago
  I disagree. I prefer the variety of different stores. I prefer the unique floor plans, esthetics, and in house products. I don’t like having Walmart, target, Dunkin, and Starbucks in every city I go to. I may be near poor but I go out of my way to avoid big box stores whenever and wherever I can. I like stores that have been run by the same owner or manager for decades. I’ve seen multiple local shops balloon into chains and in every single case their service, product quality, and in store experience got markedly worse. If I’m going to support Big Box Store then I’m going to nickel and dime every transaction I make with them because that’s how they treat me, their customer. Take the personality out of the experience and all that’s left is the transaction, no loyalty, no good feelings, just the exchange of cash for product. No amount of Twitter wokeisms, virtue signaling, or advertising will replace their dehumanization.

api 5y ago

I think it all boils down to overfitting. “Efficiency is overfitting.”

fastball 5y ago

Technically there probably is an optimal solution, if you decide on a system for valuing each of those positives/negatives. Once you manage that, there's almost certainly a point on the curve that gives you the highest total value. And yes, it is probably impossible to be objective here, but a subjective optimal solution can be helpful too.

angry_octet 5y ago
And I'd argue that we have chosen such an optimal solution, where robustness is given a low weighting, and now we are experiencing a predicted downtime event. The cost of this event is spread across the whole economy, and the executives (in industry and govt) who made the decisions which led to this will experience no downside personally. Executives at rivals who have just as bad an IT problem will receive bonuses. Essentially, we have a very weak correction incentive. Until capable hackers are endemic and ransomware hits everyone, there is little likelihood of change. Next week it could be RJR Nabisco, or Synnex/TurboTax/DuPont, or just 10,000 variants of {tiny $20m company that makes software for auditing sewerage pipe and is used by 370 of the 3000 counties in the US, and every client gets the DB corrupted}.
- munificent 5y ago
  I think we are constantly tweaking things over time. For much of human history, consolidation was natural but the relative simplicity of technology kept barriers of entry low which enabled competition and new entrants. Also, poorer logistics and transportation meant you have many smaller semi-independent markets instead of fewer monolithic ones. Industrialization and powered transportation upended that and gave a lot of power to consolidation. That led to the era of robber barons. That power concentration was so bad for society that eventually the labor and antitrust movements came around to somewhat correct it. I think now we're seeing another oscillation. Software creates huge network effects and economies of scale. If a business writes a single program once, they can run it on as many servers or sell it to as many users as possible. Services that let users interact benefit exponentially from network effects. AI generates a lot of value, but requires extremely large datasets that only the largest corporations have access to. We are essentially in the era of digital robber barons right now. Six of the top ten richest people in the world according to Forbes got there through software. (Arguably seven if you consider Musk's wealth to be software-driven.) We are clearly nowhere near the optimal point on the consolidation continuum. With luck, we'll get something like an "information labor movement" and more teeth in our antitrust regulation to correct that.
munificent 5y ago
Sure, there is an optimal meta-solution if you're able to define which set of trade-offs represent a "winning" solution. But that's different from other problems where you can simply try to optimize the result itself directly. For example, we probably don't need to have hard discussions of trade-offs when it comes to, I don't know, infant mortality. There's almost no upside to babies dying (assuming you aren't heartless enough to consider less overpopulation to be an "upside"). But with business size, there are many desirable factors and improving any of them reduces another directly opposed but also desireable factor. Competition is good. Cooperation is also good. Increasing one by definition lowers the other.

SkyPuncher 5y ago

> Consolidation gives you consistency, network effects, and economies of scale. But it also gives you monocultures, stagnation, and overfitting. I very much enjoy the fact that I can go to nearly any city in America and they'll have a few dozen well-known stores with recognizable product. There are certainly bad forms of consolidation, but it's easy to overlook the good forms.

Red_Leaves_Flyy 5y ago
I disagree. I prefer the variety of different stores. I prefer the unique floor plans, esthetics, and in house products. I don’t like having Walmart, target, Dunkin, and Starbucks in every city I go to. I may be near poor but I go out of my way to avoid big box stores whenever and wherever I can. I like stores that have been run by the same owner or manager for decades. I’ve seen multiple local shops balloon into chains and in every single case their service, product quality, and in store experience got markedly worse. If I’m going to support Big Box Store then I’m going to nickel and dime every transaction I make with them because that’s how they treat me, their customer. Take the personality out of the experience and all that’s left is the transaction, no loyalty, no good feelings, just the exchange of cash for product. No amount of Twitter wokeisms, virtue signaling, or advertising will replace their dehumanization.