7 ms·

The Five Levels: From spicy autocomplete to the dark factory

74 points by benwerd 9mo ago

ekidd 8mo ago
Having actually run some of the software produced by nearly "dark software factories," a lot of that software is completely shit. Yegge's Beads is a genuinely good design, for example, but it's flakier and more broken the Unix vendor Motif implementations in 1993, and it eats itself more often than Windows 98 would blue screen. I can actually run a bunch of orchestrated agents, and get code which isn't complete shit. But it's an extremely skill-intensive process, because I'm acting as product manager, lead engineer, and the backstop for the holes in the cognition of a bunch of different Claudes. So far, the people promising completely dark software factories are either high on their own supply, or grifting to sell books (or occasionally crypto). Or so I judge from using the programs they ship.
- xg15 8mo ago
  I found it kind of fitting that didn't even describe what a human would still do at level 5 nor why it would be desirable. It's just the "natural" progression of a 5 step ladder and that seems to be reason enough.
  - thenfcm 8mo ago
    Well isnt the point humans wouldn't need to do basically anything? It would be 'desirable' because the value is in the product of the labour not the labour itself. (Of course the resulting dystopian hellscape might be considered undesirable)
    - ekidd 8mo ago
      As I keep pointing out, if the model ever stops needing you to complete ambitious goals, then what does the model actually need you for? People somehow imagine an agent that can crush the competition with minimal human oversight. And then they somehow think that they'll be in charge, and not Sam Altman, a government, or possibly the model itself. If the model's that good, nobody's going to sell it to you.
      - handoflixue 8mo ago
        A Dark Factory is a lot more work than the model, and often perpendicular to the goal of general model improvement. A Dark Factory specializes in building one particular thing, whereas the AI labs care about generalization and what you can do absent such advanced scaffolding. It is so named because we have literal Dark Factories in the real world, run by robotics instead of AI, producing cellphones without any need for humans. None the less, said literal Dark Factory that actually exists, in the real world, is still owned by the corporation that built it. The robots did not take over, the government did not seize it.
        darkwater 8mo ago
        > None the less, said literal Dark Factory that actually exists, in the real world, is still owned by the corporation that built it. The robots did not take over, the government did not seize it. It's probably worth pointing out that hardware and software are two completely different things. At least until the day we have robots that can create and put to work other robots with 0 or minimal human intervention.
saulpw 8mo ago
One of other authors he links to[0] brags that he's released 10 projects in the past month, like "Super Xtreme Mapper, a high-end, professional MIDI mapping software for professional DJs", which has 4 stars on Github. Despite the "high-end, professional...for professional" description, literally no one is going to use it, because this guy can't [be trusted to] maintain this software. Even if Claude Code is doing all the work, adding all the features, and fixing all the bugs, someone has to issue the command to do that work, and to foot the bill. This guy is just spraying code around and snorting digital coke. There is plausibly something here with AI-generated code but as always, the value is not in the first release but in the years of maintenance and maturation that makes it something you can use and invest in. The problem with AI is that it's giving these people hyper-ADHD, they can't commit to anything, and no one will use vibe-coded tools--I'm betting not even themselves after a month. [0] https://nraford7.github.io/road-runner-economy/ https://nraford7.github.io/road-runner-economy/
- bigfishrunning 8mo ago
  > snorting digital coke What an apt description -- the website on the other side of that link is the most coked-out design I've ever seen.
- Dr_Birdbrain 8mo ago
  My feeling is that AI-generated code is disposable code. It’s great if you can quickly stand up a tool that scratches an itch for you, but there is minimal value in it for other people, and it probably doesn’t make sense to share it in a repo. Other people could just quickly vibe-code something of equal quality.
  - thewebguyd 8mo ago
    That's how I've been using and treating it, though I'm not primarily a developer. I work in ops, and LLMs write all sorts of disposable code for me. Primarily one-off scripts or little personal utilities. These don't get shared with anyone else, or put on github, etc. but have been incredibly helpful. SQL queries, some python to clean up or dig through some data sets, log files, etc. to spit out a quick result when something more robust or permanent isn't needed. Plus, so far, LLMs seem better at writing code to do a thing over directly doing the thing, where it's more likely to hallucinate, especially when it comes to working with large CSV or Json files. "Re-order this CSV file to be in Alphabetical order by the Name field" will make up fake data, but "Write a python script to order the Name filed in this CSV to be alphabetical" will succeed.
simonw 8mo ago
I've talked to a team that's doing the dark factory pattern hinted at here. It was fascinating. The key characteristics: - Nobody reviews AI-produced code, ever. They don't even look at it. - The goal of the system is to prove that the system works. A huge amount of the coding agent work goes into testing and tooling and simulating related systems and running demos. - The role of the humans is to design that system - to find new patterns that can help the agents work more effectively and demonstrate that the software they are building is robust and effective. It was a tiny team and they stuff they had built in just a few months looked very convincing to me. Some of them had 20+ years of experience as software developers working on systems with high reliability requirements, so they were not approaching this from a naive perspective. I'm hoping they come out of stealth soon because I can't really share more details than this.
- observationist 8mo ago
  You'd think at some point it'll be enough to tell the AI "ok, now do a thorough security audit, highlight all the potential issues, come up with a best practices design document, and fix all the vulnerabilities and bugs. Repeat until the codebase is secure and meets all the requisite protocol standards and industry best practices." We're not there yet, but at some point, AI is gonna be able to blitz through things like that the way they blitz through making haikus or rewriting news articles. At some point AI will just be reliably competent. Definitely not there yet. The dark factory pattern is terrifying, lol.
  - simonw 8mo ago
    That's definitely a pattern people are already starting to have good results from - using multiple "agents" (aka multiple system prompts) where one of them is a security reviewer that audits for problems and files issues for other coding agents to then fix. I don't think this worked at all well six months ago. GPT-5.2 and Opus 4.5 might just be good enough for this pattern to start being effective.
    - FEELmyAGI 8mo ago
      My current dark factory stack is using a Cyber Elon [0] at CEO with a dev team consisting of Gilfoyle, 2x Mr Robots, and Pickle Rick, with Alan Turing as dev manager, easily 5x'd my output in raw performance metrics with this, and considering I had already easily achieved a 10x over baseline dev performance using vanilla agents and other mainstream AI techniques. Whenever people say AI is just glorified auto complete I know they haven't been using the latest model versions. [0] Basically an immortal version of ELon musk with his mind fused cybernetically with Grok AI
deleted 8mo ago
[deleted]
badgersnake 8mo ago
These hype articles are getting very boring.
- Animats 8mo ago
  This is a meta-hype article. It's an article about the hype.
pphysch 8mo ago
The autopilot analogy is good because level 4-5 are essentially vaporware outside of success in controlled environments backed by massive investment and engineering.
hbarka 8mo ago
What is the AI analog for Tesla's level of robotaxi, where there's a "safety monitor" in the passenger seat or sans safety monitor there's a trailing guide car[1] and remote driver in Mumbai[2]? [1] https://electrek.co/2026/01/22/tesla-didnt-remove-the-robotaxi-safety-monitor-it-just-moved-them-to-a-trailing-car/ https://electrek.co/2026/01/22/tesla-didnt-remove-the-robota... [2] https://insideevs.com/news/760863/tesla-hiring-humans-to-control-robotaxis/ https://insideevs.com/news/760863/tesla-hiring-humans-to-con...
renjimen 8mo ago
We're going to need to become a lot more creative about what and how we test if we're ever to reach dark factory levels. Unit tests and integration tests are one thing, but truly testing against everything in a typical project requirements document is another thing.
- simonw 8mo ago
  The team I saw doing this had a fake Slack channel full of fake users, each of which was constantly hammering away trying out different things against a staging environment version of the system. That was just one of the tricks they were using, and this was a couple of months ago so they've no-doubt come up with a bunch more testing methods since then.
  - stuaxo 8mo ago
    I dread to imagine tbe state lf the code, there are some antipatterns that LLMs come back to again and again.
deadbabe 8mo ago
Level Six: knowledge on how to build products deteriorates, more high level thinking is outsourced to AI. AI are asked to simply put out several versions and possibilities of products and testers go through harvesting candidates that are the most usable and have the least bugs, good enough for production. It could take a long time or it could happen very quick. Level Seven: no one even knows what software is anymore, they just pray to AI to solve their problems and hope for the best. Some priests occasionally do random stuff that seems to affect outcomes, but no one knows for sure.
- PaulDavisThe1st 8mo ago
  Level Eight: so few people do any paid labor any more, and society failed to figure out any sort of distributive income system such as UBI, so increasing chronic and endemic poverty is slowly eating away at revenue generation from AI designed and coded products and services.
  - naruhodo 8mo ago
    Pitchforks and killbots.
6510 8mo ago
The analogy is a good fit. I'm at level 0 because no way in hell I'm going to die from cruise control. I imagine there should be two levels above: 6: The AI designs the product and 7: A market where AI (now completely autonomous) sells incomprehensible products to other AI's. Like a project Dwain factor enhancer where Dwain is a fictional character coined by an onlyfax DND bot.
radu_floricica 8mo ago
People are very pessimistic here in the comments, but I see no fundamental, long term reason why AI generated code can't be refactored, maintained and tested by AI just as well (or better) than average-quality human generated code. Especially because things are evolving - by the time the projects will need to be maintained, there will likely already be better tools to do that. So while I wouldn't vibecode drivers for life support systems yet, there is significant runway of tech debt for most use cases.

74 points by benwerd 9mo ago

ekidd 8mo ago

Having actually run some of the software produced by nearly "dark software factories," a lot of that software is completely shit. Yegge's Beads is a genuinely good design, for example, but it's flakier and more broken the Unix vendor Motif implementations in 1993, and it eats itself more often than Windows 98 would blue screen. I can actually run a bunch of orchestrated agents, and get code which isn't complete shit. But it's an extremely skill-intensive process, because I'm acting as product manager, lead engineer, and the backstop for the holes in the cognition of a bunch of different Claudes. So far, the people promising completely dark software factories are either high on their own supply, or grifting to sell books (or occasionally crypto). Or so I judge from using the programs they ship.

xg15 8mo ago
I found it kind of fitting that didn't even describe what a human would still do at level 5 nor why it would be desirable. It's just the "natural" progression of a 5 step ladder and that seems to be reason enough.
- thenfcm 8mo ago
  Well isnt the point humans wouldn't need to do basically anything? It would be 'desirable' because the value is in the product of the labour not the labour itself. (Of course the resulting dystopian hellscape might be considered undesirable)
  - ekidd 8mo ago
    As I keep pointing out, if the model ever stops needing you to complete ambitious goals, then what does the model actually need you for? People somehow imagine an agent that can crush the competition with minimal human oversight. And then they somehow think that they'll be in charge, and not Sam Altman, a government, or possibly the model itself. If the model's that good, nobody's going to sell it to you.
    - handoflixue 8mo ago
      A Dark Factory is a lot more work than the model, and often perpendicular to the goal of general model improvement. A Dark Factory specializes in building one particular thing, whereas the AI labs care about generalization and what you can do absent such advanced scaffolding. It is so named because we have literal Dark Factories in the real world, run by robotics instead of AI, producing cellphones without any need for humans. None the less, said literal Dark Factory that actually exists, in the real world, is still owned by the corporation that built it. The robots did not take over, the government did not seize it.
      - darkwater 8mo ago
        > None the less, said literal Dark Factory that actually exists, in the real world, is still owned by the corporation that built it. The robots did not take over, the government did not seize it. It's probably worth pointing out that hardware and software are two completely different things. At least until the day we have robots that can create and put to work other robots with 0 or minimal human intervention.

saulpw 8mo ago

One of other authors he links to[0] brags that he's released 10 projects in the past month, like "Super Xtreme Mapper, a high-end, professional MIDI mapping software for professional DJs", which has 4 stars on Github. Despite the "high-end, professional...for professional" description, literally no one is going to use it, because this guy can't [be trusted to] maintain this software. Even if Claude Code is doing all the work, adding all the features, and fixing all the bugs, someone has to issue the command to do that work, and to foot the bill. This guy is just spraying code around and snorting digital coke. There is plausibly something here with AI-generated code but as always, the value is not in the first release but in the years of maintenance and maturation that makes it something you can use and invest in. The problem with AI is that it's giving these people hyper-ADHD, they can't commit to anything, and no one will use vibe-coded tools--I'm betting not even themselves after a month. [0] https://nraford7.github.io/road-runner-economy/ https://nraford7.github.io/road-runner-economy/

bigfishrunning 8mo ago
> snorting digital coke What an apt description -- the website on the other side of that link is the most coked-out design I've ever seen.
Dr_Birdbrain 8mo ago
My feeling is that AI-generated code is disposable code. It’s great if you can quickly stand up a tool that scratches an itch for you, but there is minimal value in it for other people, and it probably doesn’t make sense to share it in a repo. Other people could just quickly vibe-code something of equal quality.
- thewebguyd 8mo ago
  That's how I've been using and treating it, though I'm not primarily a developer. I work in ops, and LLMs write all sorts of disposable code for me. Primarily one-off scripts or little personal utilities. These don't get shared with anyone else, or put on github, etc. but have been incredibly helpful. SQL queries, some python to clean up or dig through some data sets, log files, etc. to spit out a quick result when something more robust or permanent isn't needed. Plus, so far, LLMs seem better at writing code to do a thing over directly doing the thing, where it's more likely to hallucinate, especially when it comes to working with large CSV or Json files. "Re-order this CSV file to be in Alphabetical order by the Name field" will make up fake data, but "Write a python script to order the Name filed in this CSV to be alphabetical" will succeed.

simonw 8mo ago

I've talked to a team that's doing the dark factory pattern hinted at here. It was fascinating. The key characteristics: - Nobody reviews AI-produced code, ever. They don't even look at it. - The goal of the system is to prove that the system works. A huge amount of the coding agent work goes into testing and tooling and simulating related systems and running demos. - The role of the humans is to design that system - to find new patterns that can help the agents work more effectively and demonstrate that the software they are building is robust and effective. It was a tiny team and they stuff they had built in just a few months looked very convincing to me. Some of them had 20+ years of experience as software developers working on systems with high reliability requirements, so they were not approaching this from a naive perspective. I'm hoping they come out of stealth soon because I can't really share more details than this.

observationist 8mo ago
You'd think at some point it'll be enough to tell the AI "ok, now do a thorough security audit, highlight all the potential issues, come up with a best practices design document, and fix all the vulnerabilities and bugs. Repeat until the codebase is secure and meets all the requisite protocol standards and industry best practices." We're not there yet, but at some point, AI is gonna be able to blitz through things like that the way they blitz through making haikus or rewriting news articles. At some point AI will just be reliably competent. Definitely not there yet. The dark factory pattern is terrifying, lol.
- simonw 8mo ago
  That's definitely a pattern people are already starting to have good results from - using multiple "agents" (aka multiple system prompts) where one of them is a security reviewer that audits for problems and files issues for other coding agents to then fix. I don't think this worked at all well six months ago. GPT-5.2 and Opus 4.5 might just be good enough for this pattern to start being effective.
  - FEELmyAGI 8mo ago
    My current dark factory stack is using a Cyber Elon [0] at CEO with a dev team consisting of Gilfoyle, 2x Mr Robots, and Pickle Rick, with Alan Turing as dev manager, easily 5x'd my output in raw performance metrics with this, and considering I had already easily achieved a 10x over baseline dev performance using vanilla agents and other mainstream AI techniques. Whenever people say AI is just glorified auto complete I know they haven't been using the latest model versions. [0] Basically an immortal version of ELon musk with his mind fused cybernetically with Grok AI

deleted 8mo ago

[deleted]

badgersnake 8mo ago

These hype articles are getting very boring.

Animats 8mo ago
This is a meta-hype article. It's an article about the hype.

pphysch 8mo ago

The autopilot analogy is good because level 4-5 are essentially vaporware outside of success in controlled environments backed by massive investment and engineering.

hbarka 8mo ago

What is the AI analog for Tesla's level of robotaxi, where there's a "safety monitor" in the passenger seat or sans safety monitor there's a trailing guide car[1] and remote driver in Mumbai[2]? [1] https://electrek.co/2026/01/22/tesla-didnt-remove-the-robotaxi-safety-monitor-it-just-moved-them-to-a-trailing-car/ https://electrek.co/2026/01/22/tesla-didnt-remove-the-robota... [2] https://insideevs.com/news/760863/tesla-hiring-humans-to-control-robotaxis/ https://insideevs.com/news/760863/tesla-hiring-humans-to-con...

renjimen 8mo ago

We're going to need to become a lot more creative about what and how we test if we're ever to reach dark factory levels. Unit tests and integration tests are one thing, but truly testing against everything in a typical project requirements document is another thing.

simonw 8mo ago
The team I saw doing this had a fake Slack channel full of fake users, each of which was constantly hammering away trying out different things against a staging environment version of the system. That was just one of the tricks they were using, and this was a couple of months ago so they've no-doubt come up with a bunch more testing methods since then.
- stuaxo 8mo ago
  I dread to imagine tbe state lf the code, there are some antipatterns that LLMs come back to again and again.

deadbabe 8mo ago

Level Six: knowledge on how to build products deteriorates, more high level thinking is outsourced to AI. AI are asked to simply put out several versions and possibilities of products and testers go through harvesting candidates that are the most usable and have the least bugs, good enough for production. It could take a long time or it could happen very quick. Level Seven: no one even knows what software is anymore, they just pray to AI to solve their problems and hope for the best. Some priests occasionally do random stuff that seems to affect outcomes, but no one knows for sure.

PaulDavisThe1st 8mo ago
Level Eight: so few people do any paid labor any more, and society failed to figure out any sort of distributive income system such as UBI, so increasing chronic and endemic poverty is slowly eating away at revenue generation from AI designed and coded products and services.
- naruhodo 8mo ago
  Pitchforks and killbots.

6510 8mo ago

The analogy is a good fit. I'm at level 0 because no way in hell I'm going to die from cruise control. I imagine there should be two levels above: 6: The AI designs the product and 7: A market where AI (now completely autonomous) sells incomprehensible products to other AI's. Like a project Dwain factor enhancer where Dwain is a fictional character coined by an onlyfax DND bot.

radu_floricica 8mo ago

People are very pessimistic here in the comments, but I see no fundamental, long term reason why AI generated code can't be refactored, maintained and tested by AI just as well (or better) than average-quality human generated code. Especially because things are evolving - by the time the projects will need to be maintained, there will likely already be better tools to do that. So while I wouldn't vibecode drivers for life support systems yet, there is significant runway of tech debt for most use cases.