4 ms·

> The AI had simply told me a good story. Like vibewriting a novel, the agent showed me a good couple paragraphs that sure enough made sense and were structural

by ncruces 9mo ago

> The AI had simply told me a good story. Like vibewriting a novel, the agent showed me a good couple paragraphs that sure enough made sense and were structurally and syntactically correct. Hell, it even picked up on the idiosyncrasies of the various characters. But for whatever reason, when you read the whole chapter, it’s a mess. It makes no sense in the overall context of the book and the preceding and proceeding chapters. This is the bit I think enthusiasts need to argue doesn't apply. Have you ever read a 200 page vibewritten novel and found it satisfying? So why do you think a 10 kLoC vibecoded codebase will be any good engineering-wise?

ashikns 9mo ago
Because a novel is about creative output, and engineering is about understanding a lot of rules and requirements and then writing logic to satisfy that. The latter has a much more explicitly defined output.
- therealdrag0 9mo ago
  Said another way, a novel is about the experience of reading every word of implementation, whereas software is sufficient to be a black box, the functional output is all that matters. No one is reading assembly for example. We’re moving into a world where suboptimal code doesn’t matter that much because it’s so cheap to produce.
- ModernMech 9mo ago
  The lesson of UML is that software engineering is not a process of refining rules and requirements into logic. Software engineering is lucrative because it very much is a creative process.
mrtesthah 9mo ago
I wrote this a day ago but I find it even more relevant to your observation: — I would never use, let alone pay for, a fully vibe-coded app whose implementation no human understands. Whether you’re reading a book or using an app, you’re communicating with the author by way of your shared humanity in how they anticipate what you’re thinking as you explore the work. The author incorporates and plans for those predicted reactions and thoughts where it makes sense. Ultimately the author is conveying an implicit mental model (or even evoking emotional states or sensations) to the reader. The first problem is that many of these pathways and edge cases aren’t apparent until the actual implementation, and sometimes in the process the author realizes that the overall product would work better if it were re-specified from the start. This opportunity is lost without a hands on approach. The second problem is that, the less human touch is there, the less consistent the mental model conveyed to the user is going to be, because a specification and collection of prompts does not constitute a mental model. This can create subconscious confusion and cognitive friction when interacting with the work.
- charcircuit 9mo ago
  No human understands how Windows works. The number of products where a human understands the whole thing is small.
  - mrtesthah 9mo ago
    That's a false analogy. Product managers, designers, API implementers, kernel developers, etc. all understand what they're building and how that fits into a larger picture.
    - charcircuit 9mo ago
      They may know the area they are responsible for, but they don't know all of the details of everything else and just have to trust that other people are doing the right thing and following contracts correctly. It doesn't require anyone to have full global understanding. Having local experts is good enough.
      - mrtesthah 9mo ago
        Local experts still need to have a shared mental model of how what they’re building fits into the overall system.
fsloth 9mo ago
"So why do you think a 10 kLoC vibecoded codebase will be any good engineering-wise?" I've been coding a side-project for a year with full LLM assistance (the project is quite a bit older than that). Basically I spent over a decade developing CAD software at Trimble and now have pivoted to a different role and different company. So like an addict, I of course wanted to continue developing CAD technology. I pretty much know how CAD software is supposed to work. But it's _a lot of work_ to put together. With LLMs I can basically speedrun through my requirements that require tons of boilerplate. The velocity is incredible compared to if I would be doing this by hand. Sometimes the LLM outputs total garbage. Then you don't accept the output, and start again. The hardest parts are never coding but design. The engineer does the design. Sometimes I pain weeks or months over a difficult detail (it's a sideproject, I have a family etc). Once the design is crystal clear, it's fairly obvious if the LLM output is aligned with the design or not. Once I have good design, I can just start the feature / boilerplate speedrun. If you have a Windows box you can try my current public alpha. The bugs are on me, not on the LLM: https://github.com/AdaShape/adashape-open-testing/releases/tag/adashape-alpha-0.1.4 https://github.com/AdaShape/adashape-open-testing/releases/t...
- mattjhall 9mo ago
  It’s amazing how often these miracle codebases that an AI has generated are always not open source.
  - fsloth 9mo ago
    If you doubt it’s real just run it, man. I shared the app because it’s not confidential and it’s concrete - I can’t really discuss work stuff without stressing out what I can share and what not. At least in my workplace everyone I know is using Claude Code or Cursor. Now, I don’t know why some people are productive with tools and some aren’t. But the code generation capabilities are for real.
- 0xffff2 9mo ago
  Neat project, and your experience mirrors mine when writing hobby projects. About the project itself, do you plan to open source if eventually? LLM discussion aside, I've long been frustrated by the lack of a good free desktop 3D CAD software.
causal 9mo ago
I like this way of framing the problem, and it might even be a good way to self-evaluate your use of AI: Try vibe-writing a novel and see how coherent it is. I suspect part of the reason we see such a wide range of testimonies about vibe-coding is some people are actually better at it, and it would be useful to have some way of measuring that effectiveness.
rahimnathwani 9mo ago
Have you ever read a 200 page vibewritten novel and found it satisfying? I haven't, but my son has. For two separate novels authored by GPT 4.5. (The model was asked to generate a chapter at a time. At each step, it was given the full outline of the novel, the characters, and a summary of each chapter so far.)
- andai 9mo ago
  Interesting. I heard that model was significantly better than what we ended up with (at least for writing), and they shut it down because it was huge and expensive. Did the model also come up with the idea for the novel, the characters, the outline?
  - rahimnathwani 9mo ago
    For one novel, I gave the model a sentence about the idea, and the names and a few words about each of the characters. For the other, my son wrote ~200 words total describing the story idea and the characters. In each case, the model created the detailed outline and did all the writing.
lopatin 9mo ago
I don’t get the analogy because novel is supposed to be interesting. Code isn’t supposed to be interesting, it’s supposed to work. If you’re writing novel algorithms all day, then I get your point. But are you? Or have you ever delegated work? If you find the AI losing its train of thought all it takes is to try again with better high level instructions.

by ncruces 9mo ago

ashikns 9mo ago

Because a novel is about creative output, and engineering is about understanding a lot of rules and requirements and then writing logic to satisfy that. The latter has a much more explicitly defined output.

therealdrag0 9mo ago
Said another way, a novel is about the experience of reading every word of implementation, whereas software is sufficient to be a black box, the functional output is all that matters. No one is reading assembly for example. We’re moving into a world where suboptimal code doesn’t matter that much because it’s so cheap to produce.
ModernMech 9mo ago
The lesson of UML is that software engineering is not a process of refining rules and requirements into logic. Software engineering is lucrative because it very much is a creative process.

mrtesthah 9mo ago

I wrote this a day ago but I find it even more relevant to your observation: — I would never use, let alone pay for, a fully vibe-coded app whose implementation no human understands. Whether you’re reading a book or using an app, you’re communicating with the author by way of your shared humanity in how they anticipate what you’re thinking as you explore the work. The author incorporates and plans for those predicted reactions and thoughts where it makes sense. Ultimately the author is conveying an implicit mental model (or even evoking emotional states or sensations) to the reader. The first problem is that many of these pathways and edge cases aren’t apparent until the actual implementation, and sometimes in the process the author realizes that the overall product would work better if it were re-specified from the start. This opportunity is lost without a hands on approach. The second problem is that, the less human touch is there, the less consistent the mental model conveyed to the user is going to be, because a specification and collection of prompts does not constitute a mental model. This can create subconscious confusion and cognitive friction when interacting with the work.

charcircuit 9mo ago
No human understands how Windows works. The number of products where a human understands the whole thing is small.
- mrtesthah 9mo ago
  That's a false analogy. Product managers, designers, API implementers, kernel developers, etc. all understand what they're building and how that fits into a larger picture.
  - charcircuit 9mo ago
    They may know the area they are responsible for, but they don't know all of the details of everything else and just have to trust that other people are doing the right thing and following contracts correctly. It doesn't require anyone to have full global understanding. Having local experts is good enough.
    - mrtesthah 9mo ago
      Local experts still need to have a shared mental model of how what they’re building fits into the overall system.

fsloth 9mo ago

"So why do you think a 10 kLoC vibecoded codebase will be any good engineering-wise?" I've been coding a side-project for a year with full LLM assistance (the project is quite a bit older than that). Basically I spent over a decade developing CAD software at Trimble and now have pivoted to a different role and different company. So like an addict, I of course wanted to continue developing CAD technology. I pretty much know how CAD software is supposed to work. But it's _a lot of work_ to put together. With LLMs I can basically speedrun through my requirements that require tons of boilerplate. The velocity is incredible compared to if I would be doing this by hand. Sometimes the LLM outputs total garbage. Then you don't accept the output, and start again. The hardest parts are never coding but design. The engineer does the design. Sometimes I pain weeks or months over a difficult detail (it's a sideproject, I have a family etc). Once the design is crystal clear, it's fairly obvious if the LLM output is aligned with the design or not. Once I have good design, I can just start the feature / boilerplate speedrun. If you have a Windows box you can try my current public alpha. The bugs are on me, not on the LLM: https://github.com/AdaShape/adashape-open-testing/releases/tag/adashape-alpha-0.1.4 https://github.com/AdaShape/adashape-open-testing/releases/t...

mattjhall 9mo ago
It’s amazing how often these miracle codebases that an AI has generated are always not open source.
- fsloth 9mo ago
  If you doubt it’s real just run it, man. I shared the app because it’s not confidential and it’s concrete - I can’t really discuss work stuff without stressing out what I can share and what not. At least in my workplace everyone I know is using Claude Code or Cursor. Now, I don’t know why some people are productive with tools and some aren’t. But the code generation capabilities are for real.
0xffff2 9mo ago
Neat project, and your experience mirrors mine when writing hobby projects. About the project itself, do you plan to open source if eventually? LLM discussion aside, I've long been frustrated by the lack of a good free desktop 3D CAD software.

causal 9mo ago

I like this way of framing the problem, and it might even be a good way to self-evaluate your use of AI: Try vibe-writing a novel and see how coherent it is. I suspect part of the reason we see such a wide range of testimonies about vibe-coding is some people are actually better at it, and it would be useful to have some way of measuring that effectiveness.

rahimnathwani 9mo ago

Have you ever read a 200 page vibewritten novel and found it satisfying? I haven't, but my son has. For two separate novels authored by GPT 4.5. (The model was asked to generate a chapter at a time. At each step, it was given the full outline of the novel, the characters, and a summary of each chapter so far.)

andai 9mo ago
Interesting. I heard that model was significantly better than what we ended up with (at least for writing), and they shut it down because it was huge and expensive. Did the model also come up with the idea for the novel, the characters, the outline?
- rahimnathwani 9mo ago
  For one novel, I gave the model a sentence about the idea, and the names and a few words about each of the characters. For the other, my son wrote ~200 words total describing the story idea and the characters. In each case, the model created the detailed outline and did all the writing.

lopatin 9mo ago

I don’t get the analogy because novel is supposed to be interesting. Code isn’t supposed to be interesting, it’s supposed to work. If you’re writing novel algorithms all day, then I get your point. But are you? Or have you ever delegated work? If you find the AI losing its train of thought all it takes is to try again with better high level instructions.