4 ms·

Death of the Dark Room: How Generative AI Broke Enterprise It's Political Cover

2 points by chris-d 8mo ago

chris-d 8mo ago
For thirty years, a failed ERP could be buried inside an architecture diagram. A failed AI chatbot gets screenshotted and forwarded to the board before lunch. That's the shift this piece is about. But the part I find more interesting is the second fear – not that AI fails visibly, but that it succeeds visibly. A system that genuinely compresses legal review time or automates reporting doesn't just create efficiency. It makes existing roles and empires contestable. And that's when the corporate immune system kicks in: it won't be compatible, it can't be done here, it will take years, it's probably illegal, who takes responsibility. The result is what I call the controlled mediocrity trap – organisations deploying AI that's good enough to avoid embarrassment but not so effective that it threatens anyone. 74% of CIOs in Dataiku's 2026 survey regretted at least one major AI platform decision. I think a lot of that regret is political, not technical. I suspect this is more common than people admit...
Terr_ 8mo ago
> Generative AI goes further. It changes evaluative capacity. A director who cannot read an architecture diagram can still form a direct judgement about a summary, a search result, a recommendation, a draft, or a chatbot answer. It may technically be a "direct" judgement... except it's judging a very indirect artifact, one that likely has absolutely no review or validation by any human with any domain-knowledge or involvement. We should be careful to distinguish between the capabilities it actually gives people versus the capabilities they assume they've obtained. > That objection misses the political point. Governance action is triggered by confidence, not by methodological purity. The board does not need a technically valid comparison to demand an explanation. It only needs a comparison that feels legible. Right, whether that "directly evaluated summary" is itself correct matters less than we'd hope: Its mere presence can affect what happens. "Yes Minister" is a classic British comedy series, and in it certain manipulative bureaucratic characters are very aware of this phenomenon, cynically penning or revealing "reports" and "cautions" in order to achieve a certain policy outcome.
- deleted 7mo ago
  [deleted]
- chris-d 7mo ago
  Right, but that's basically my point. I'm not saying generative AI gives boards a better way to judge the underlying work. Often it doesn't. I'm saying it makes the output visible enough that non-technical leadership can compare it, form a view, and intervene without the old technical layer in the middle. That matters even when the judgment is shallow or wrong...
beardyw 8mo ago
It's should read IT's Worth fixing if you can. I read the headline several times and didn't get it.
- chris-d 7mo ago
  You're right, thanks for the tip. Unfortunately, the series only allows title changes for the first two hours after publication…

2 points by chris-d 8mo ago

For thirty years, a failed ERP could be buried inside an architecture diagram. A failed AI chatbot gets screenshotted and forwarded to the board before lunch. That's the shift this piece is about. But the part I find more interesting is the second fear – not that AI fails visibly, but that it succeeds visibly. A system that genuinely compresses legal review time or automates reporting doesn't just create efficiency. It makes existing roles and empires contestable. And that's when the corporate immune system kicks in: it won't be compatible, it can't be done here, it will take years, it's probably illegal, who takes responsibility. The result is what I call the controlled mediocrity trap – organisations deploying AI that's good enough to avoid embarrassment but not so effective that it threatens anyone. 74% of CIOs in Dataiku's 2026 survey regretted at least one major AI platform decision. I think a lot of that regret is political, not technical. I suspect this is more common than people admit...

Terr_ 8mo ago

> Generative AI goes further. It changes evaluative capacity. A director who cannot read an architecture diagram can still form a direct judgement about a summary, a search result, a recommendation, a draft, or a chatbot answer. It may technically be a "direct" judgement... except it's judging a very indirect artifact, one that likely has absolutely no review or validation by any human with any domain-knowledge or involvement. We should be careful to distinguish between the capabilities it actually gives people versus the capabilities they assume they've obtained. > That objection misses the political point. Governance action is triggered by confidence, not by methodological purity. The board does not need a technically valid comparison to demand an explanation. It only needs a comparison that feels legible. Right, whether that "directly evaluated summary" is itself correct matters less than we'd hope: Its mere presence can affect what happens. "Yes Minister" is a classic British comedy series, and in it certain manipulative bureaucratic characters are very aware of this phenomenon, cynically penning or revealing "reports" and "cautions" in order to achieve a certain policy outcome.

deleted 7mo ago
[deleted]
chris-d 7mo ago
Right, but that's basically my point. I'm not saying generative AI gives boards a better way to judge the underlying work. Often it doesn't. I'm saying it makes the output visible enough that non-technical leadership can compare it, form a view, and intervene without the old technical layer in the middle. That matters even when the judgment is shallow or wrong...

beardyw 8mo ago

It's should read IT's Worth fixing if you can. I read the headline several times and didn't get it.

chris-d 7mo ago
You're right, thanks for the tip. Unfortunately, the series only allows title changes for the first two hours after publication…