5 ms·

I guess they really do eat their own dogfood and vibe code their way through it without care for technical debt? In a way, it’s a good challenge, but it’s fairl

I guess they really do eat their own dogfood and vibe code their way through it without care for technical debt? In a way, it’s a good challenge, but it’s fairly painful to watch the current state of the project (which is about a year old now, so it should be in prime shape).

coldtrait 7mo ago
Boris Cherny, the creator of Claude Code said he uses CC to build CC.
- Cthulhu_ 7mo ago
  Which makes for an interesting thought / discussion; code is written to be read by humans first, executed by computers second. What would code look like if it was written to be read by LLMs? The way they work now (or, how they're trained) is on human language and code, but there might be a style that's better for LLMs. Whatever metric of "better" you may use. Just a thought experiment, I very much doubt I'm the first one to think of it. It's probably in the same line of "why doesn't an LLM just write assembly directly"
  - syphia 7mo ago
    LLMs read and write human-code because humans have been reading and writing human-code. The sample size of assembly problems is, in my estimate, too small for LLMs to efficiently read and write it for common use cases. I liken it to the problem of applying machine learning to hard video games (e.g. Starcraft). When trained to mimic human strategies, it can be extremely effective, but machine learning will not discover broadly effective strategies on a reasonable timescale. If you convert "human strategies" to "human theory, programming languages, and design patterns", perhaps the point will be clear. But: could the ouroboric cycle of LLM use decay the common strategies and design patterns we use into inexplicable blobs of assembly? Can LLMs improve at programming if humans do not advance the theory or invent new languages, patterns, etc?
    - Mentlo 7mo ago
      But starcraft training is not through mimicking human strategies - it was pure RL with a reward function shaped around winning, which allows it to emerge non-human and eventually super-human strategies (such as the worker oversaturation). The current training loop for coding is RL as well - so a departure from human coding patterns is not unexpected (even if departure from human coding structure is unexpected, as that would require development of a new coding language).
      - syphia 7mo ago
        AlphaStar (2019) refined through self-play but was initially trained on human data. I don't know of any other high-level Starcraft AI, but if you do let me know.
  - tempay 7mo ago
    > It's probably in the same line of "why doesn't an LLM just write assembly directly" My suspicion is that the "language" part of LLMs means they tend to prefer languages which are closer to human languages than assembly and benefit from much of the same abstractions and tooling (hence the recent acquisition of bun and astral).
  - fragmede 7mo ago
    The problem with that is that assembly isn't portable, and x86 isn't as dominant as it once was, so then you've got arm and x86(_64). But you could target the LLVM machine if you wanted.
  - coldtrait 7mo ago
    I hope whoever seeks to replace humans with AIs to write code never find this comment.
- stingraycharles 7mo ago
  Yes but my point was that they seem to explicitly not care about code quality and/or the insane amount of bloat, and seem to just want the LLM to be able to deal with it.
  - lukaslalinsky 7mo ago
    I've heard somewhere that they have roughly 100% code churn every few months, so yes, they unfortunately don't care about code quality. It's a shame, because it's still the best coding agent, in my experience.
    - stingraycharles 7mo ago
      Yes, but as I said, it’s in a way the ultimate form of dogfooding: ideally they’ll be able to get the LLM smart enough to keep the codebase working well long-term. Now whether that’s actually possible is a second topic.
    - drakezone 7mo ago
      [dead]
    - menaerus 7mo ago
      > they unfortunately don't care about code quality. > It's a shame, because it's still the best coding agent, in my experience. If it is the best, and if it delivers the value users are asking for, then why would they have an incentive to make further $$$ investments to make it of a "higher" quality if the value this difference could make is not substantial or hurts the ROI? On many projects I found this "higher quality" not only to be false of delivering more substantial value but actually I found it was hurting the project to deliver the value that matters. Maybe we are after all entering the era of SWE where all this bike-shedding is gone and only type of engineers who will be able to survive in it will be the ones who are capable of delivering the actual value (IME very few per project).
      - troupo 7mo ago
        Is this why they ran into a bug with people hitting usage limits even on very short sessions and had to cease all communications for over a day after a week of gaslighting users because they couldn't find the root cause in the "quality doesn't matter" code base? Or that's why tgey had to buy bun with actual engineers to work on Claude Code to reduce memory peaks from 68 GB (yes, 68 gigabytes) to a "measely" 1.7? Because code quality doesn't matter? Or that a year later they still cannot figure out how to render anything in the terminal without flickering? The only reason people use Claude Code is because it's the only way to use Anthropic's heavily subsidized subscription. You get banned if you use it through other, better, tools.
brabel 7mo ago
> is about a year old now, so it should be in prime shape A 1yo project may be in good shape if written by just one dev, maybe a few. But if you have many devs, I can guarantee it will be messy and buggy. If anything, at 1yo it is probably still full of bugs because not enough time has elapsed for people to run into them.
- mattmanser 7mo ago
  It's only 510k LoC, at ~100 lines of code a day for a year, this code base would take 23 engineers a year to write. That's for 220 working days in somewhere civilized. And I'm sure we all know that when working on a greenfield project you can produce a lot more LoC per day than maintaining a legacy one. Given that vibe code is significantly more verbose, you're probably talking about ~15 engineers worth of code? I know that's all silly numbers, but this is just attempting to give people some context here, this isn't a massive code base. I've not read a lot of it, so maybe it's better than the verbose code I see Claude put out sometimes.
  - cududa 7mo ago
    When you say it’s not a massive codebase, I’m curious, what are you comparing it to?
    - mattmanser 7mo ago
      The previous poster was making out that in a year the code base would be a mess if people had done it. This is a two-pizza team sized project, so it's not a project that the code quality would inevitably spiral out of control due to communication problems. A single senior architect COULD have kept the code quality under control.
    - kordlessagain 7mo ago
      Splunk.
  - lelanthran 7mo ago
    > It's only 510k LoC, at ~100 lines of code a day for a year, this code base would take 23 engineers a year to write. Correction: a code base of 500kLoC would take 23 engineers a year to write. There is no indication that the functionality needed in a TUI app that does what this app does needs 500kLoC.
troupo 7mo ago
They explicitly boast about using claude code to write code: https://x.com/bcherny/status/2007179836704600237 https://x.com/bcherny/status/2007179836704600237 That's how you get "oh this TUI API wrapper needs 68GB of RAM" https://x.com/jarredsumner/status/2026497606575398987 https://x.com/jarredsumner/status/2026497606575398987 or "we need 16ms to lay out a few hundred characters on screen that's why it's a small game engine": https://x.com/trq212/status/2014051501786931427 https://x.com/trq212/status/2014051501786931427
- 000ooo000 7mo ago
  Just finished looking at Ink here.. frontend world has no shame. Love the gloating about 40x less RAM as if that amount of memory for a text REPL even approaches defensible. "CC built CC" is not the flex people seem to suggest it is.
  - johnisgood 7mo ago
    Indeed a sad state of affairs.
  - anthk 7mo ago
    Frontend losers not realizing the turds they are releasing. An LLM client fits under netcat+echo+awk+jq runnable under a 486 if there's no SSL/TLS on its way, Pentium II could drive fast TLS connections like nothing and under 32MB of RAM with NetBSD for a simple terminal install, maybe with X and a simple WM with RXVT if you care.
    - ggregoire 7mo ago
      Any loser is a "full stack software engineer" nowadays thanks to claude.
mikkupikku 7mo ago
Put yourself in their shoes; either the quality of Claude's coding continues to improve or else their business is probably doomed if it stagnates, so for them it makes sense to punt technical debt to the future when more capable versions of their models will be able to better fix it. This is why I personally don't take technical debt arguments about how LLM maintained code bases deteriorate with size/age seriously; it presumes that at some point I'll give up with the LLM and be left with a mess to clean up by hand, but that's not going to happen, future maintenance is to be left to LLMs and if that isn't possible for some reason then the project is as good as dead anyway. When you start a project with a LLM the plan should be to see it through with LLMs, planning to have unaided humans take over maintenance at some point is a mistake.
- openfoliage 7mo ago
  I am more worried that we are moving toward creating black boxes and this might turn software "development" into a field as confused as philosophy and dialectics.
- blanched 7mo ago
  Doesn't this contradict the popular wisdom that "what's good for a human engineer is good for an LLM"? e.g. documentation, separation of concerns, organized files, DRY. I find LLMs very useful and capable, but in my experience they definitely perform worse when things are unorganized. Maintenance isn't just aesthetics, it's a direct input to correctness.
  - mikkupikku 7mo ago
    Maybe a little. I don't hold fast to that popular wisdom, e.g. I think comments are not always a net positive for LLMs. With respect to technical debt, how much debt is too much debt before it gums up the works and arrests forward progress on the software? It probably depends on the individual programmer. LLMs do seem to have a higher tolerance for technical debt than myself personally at least.
    - blanched 7mo ago

coldtrait 7mo ago

Boris Cherny, the creator of Claude Code said he uses CC to build CC.

Cthulhu_ 7mo ago
Which makes for an interesting thought / discussion; code is written to be read by humans first, executed by computers second. What would code look like if it was written to be read by LLMs? The way they work now (or, how they're trained) is on human language and code, but there might be a style that's better for LLMs. Whatever metric of "better" you may use. Just a thought experiment, I very much doubt I'm the first one to think of it. It's probably in the same line of "why doesn't an LLM just write assembly directly"
- syphia 7mo ago
  LLMs read and write human-code because humans have been reading and writing human-code. The sample size of assembly problems is, in my estimate, too small for LLMs to efficiently read and write it for common use cases. I liken it to the problem of applying machine learning to hard video games (e.g. Starcraft). When trained to mimic human strategies, it can be extremely effective, but machine learning will not discover broadly effective strategies on a reasonable timescale. If you convert "human strategies" to "human theory, programming languages, and design patterns", perhaps the point will be clear. But: could the ouroboric cycle of LLM use decay the common strategies and design patterns we use into inexplicable blobs of assembly? Can LLMs improve at programming if humans do not advance the theory or invent new languages, patterns, etc?
  - Mentlo 7mo ago
    But starcraft training is not through mimicking human strategies - it was pure RL with a reward function shaped around winning, which allows it to emerge non-human and eventually super-human strategies (such as the worker oversaturation). The current training loop for coding is RL as well - so a departure from human coding patterns is not unexpected (even if departure from human coding structure is unexpected, as that would require development of a new coding language).
    - syphia 7mo ago
      AlphaStar (2019) refined through self-play but was initially trained on human data. I don't know of any other high-level Starcraft AI, but if you do let me know.
- tempay 7mo ago
  > It's probably in the same line of "why doesn't an LLM just write assembly directly" My suspicion is that the "language" part of LLMs means they tend to prefer languages which are closer to human languages than assembly and benefit from much of the same abstractions and tooling (hence the recent acquisition of bun and astral).
- fragmede 7mo ago
  The problem with that is that assembly isn't portable, and x86 isn't as dominant as it once was, so then you've got arm and x86(_64). But you could target the LLVM machine if you wanted.
- coldtrait 7mo ago
  I hope whoever seeks to replace humans with AIs to write code never find this comment.
stingraycharles 7mo ago
Yes but my point was that they seem to explicitly not care about code quality and/or the insane amount of bloat, and seem to just want the LLM to be able to deal with it.
- lukaslalinsky 7mo ago
  I've heard somewhere that they have roughly 100% code churn every few months, so yes, they unfortunately don't care about code quality. It's a shame, because it's still the best coding agent, in my experience.
  - stingraycharles 7mo ago
    Yes, but as I said, it’s in a way the ultimate form of dogfooding: ideally they’ll be able to get the LLM smart enough to keep the codebase working well long-term. Now whether that’s actually possible is a second topic.
  - drakezone 7mo ago
    [dead]
  - menaerus 7mo ago
    > they unfortunately don't care about code quality. > It's a shame, because it's still the best coding agent, in my experience. If it is the best, and if it delivers the value users are asking for, then why would they have an incentive to make further $$$ investments to make it of a "higher" quality if the value this difference could make is not substantial or hurts the ROI? On many projects I found this "higher quality" not only to be false of delivering more substantial value but actually I found it was hurting the project to deliver the value that matters. Maybe we are after all entering the era of SWE where all this bike-shedding is gone and only type of engineers who will be able to survive in it will be the ones who are capable of delivering the actual value (IME very few per project).
    - troupo 7mo ago
      Is this why they ran into a bug with people hitting usage limits even on very short sessions and had to cease all communications for over a day after a week of gaslighting users because they couldn't find the root cause in the "quality doesn't matter" code base? Or that's why tgey had to buy bun with actual engineers to work on Claude Code to reduce memory peaks from 68 GB (yes, 68 gigabytes) to a "measely" 1.7? Because code quality doesn't matter? Or that a year later they still cannot figure out how to render anything in the terminal without flickering? The only reason people use Claude Code is because it's the only way to use Anthropic's heavily subsidized subscription. You get banned if you use it through other, better, tools.

brabel 7mo ago

> is about a year old now, so it should be in prime shape A 1yo project may be in good shape if written by just one dev, maybe a few. But if you have many devs, I can guarantee it will be messy and buggy. If anything, at 1yo it is probably still full of bugs because not enough time has elapsed for people to run into them.

mattmanser 7mo ago
It's only 510k LoC, at ~100 lines of code a day for a year, this code base would take 23 engineers a year to write. That's for 220 working days in somewhere civilized. And I'm sure we all know that when working on a greenfield project you can produce a lot more LoC per day than maintaining a legacy one. Given that vibe code is significantly more verbose, you're probably talking about ~15 engineers worth of code? I know that's all silly numbers, but this is just attempting to give people some context here, this isn't a massive code base. I've not read a lot of it, so maybe it's better than the verbose code I see Claude put out sometimes.
- cududa 7mo ago
  When you say it’s not a massive codebase, I’m curious, what are you comparing it to?
  - mattmanser 7mo ago
    The previous poster was making out that in a year the code base would be a mess if people had done it. This is a two-pizza team sized project, so it's not a project that the code quality would inevitably spiral out of control due to communication problems. A single senior architect COULD have kept the code quality under control.
  - kordlessagain 7mo ago
    Splunk.
- lelanthran 7mo ago
  > It's only 510k LoC, at ~100 lines of code a day for a year, this code base would take 23 engineers a year to write. Correction: a code base of 500kLoC would take 23 engineers a year to write. There is no indication that the functionality needed in a TUI app that does what this app does needs 500kLoC.

troupo 7mo ago

They explicitly boast about using claude code to write code: https://x.com/bcherny/status/2007179836704600237 https://x.com/bcherny/status/2007179836704600237 That's how you get "oh this TUI API wrapper needs 68GB of RAM" https://x.com/jarredsumner/status/2026497606575398987 https://x.com/jarredsumner/status/2026497606575398987 or "we need 16ms to lay out a few hundred characters on screen that's why it's a small game engine": https://x.com/trq212/status/2014051501786931427 https://x.com/trq212/status/2014051501786931427

000ooo000 7mo ago
Just finished looking at Ink here.. frontend world has no shame. Love the gloating about 40x less RAM as if that amount of memory for a text REPL even approaches defensible. "CC built CC" is not the flex people seem to suggest it is.
- johnisgood 7mo ago
  Indeed a sad state of affairs.
- anthk 7mo ago
  Frontend losers not realizing the turds they are releasing. An LLM client fits under netcat+echo+awk+jq runnable under a 486 if there's no SSL/TLS on its way, Pentium II could drive fast TLS connections like nothing and under 32MB of RAM with NetBSD for a simple terminal install, maybe with X and a simple WM with RXVT if you care.
  - ggregoire 7mo ago
    Any loser is a "full stack software engineer" nowadays thanks to claude.

mikkupikku 7mo ago

Put yourself in their shoes; either the quality of Claude's coding continues to improve or else their business is probably doomed if it stagnates, so for them it makes sense to punt technical debt to the future when more capable versions of their models will be able to better fix it. This is why I personally don't take technical debt arguments about how LLM maintained code bases deteriorate with size/age seriously; it presumes that at some point I'll give up with the LLM and be left with a mess to clean up by hand, but that's not going to happen, future maintenance is to be left to LLMs and if that isn't possible for some reason then the project is as good as dead anyway. When you start a project with a LLM the plan should be to see it through with LLMs, planning to have unaided humans take over maintenance at some point is a mistake.

openfoliage 7mo ago
I am more worried that we are moving toward creating black boxes and this might turn software "development" into a field as confused as philosophy and dialectics.
blanched 7mo ago
Doesn't this contradict the popular wisdom that "what's good for a human engineer is good for an LLM"? e.g. documentation, separation of concerns, organized files, DRY. I find LLMs very useful and capable, but in my experience they definitely perform worse when things are unorganized. Maintenance isn't just aesthetics, it's a direct input to correctness.
- mikkupikku 7mo ago
  Maybe a little. I don't hold fast to that popular wisdom, e.g. I think comments are not always a net positive for LLMs. With respect to technical debt, how much debt is too much debt before it gums up the works and arrests forward progress on the software? It probably depends on the individual programmer. LLMs do seem to have a higher tolerance for technical debt than myself personally at least.
  - blanched 7mo ago