5 ms·

I hate to be that guy, but... one week old project, clear signs of vibing. I will be shocked if the remaining work listed (cpython test suite) proceeds in any r

I hate to be that guy, but... one week old project, clear signs of vibing. I will be shocked if the remaining work listed (cpython test suite) proceeds in any reasonable timeline. This is a pretty hard problem to just solve in a week. EDIT: and man, these kind of comments LLM created comments are really starting to grind my gears as my job slowly turns into reviewing LLM PRs: > Known gaps at the language level are burned down through the ratcheted floors above — the committed floor files, not this README, are the authoritative compatibility baseline.

baq 3mo ago
of course it is vibed. it doesn't matter as long as it works.
- ActionHank 3mo ago
  That's the neat part, when it's vibed it works, until it doesn't and then it's really hard to make it work again.
  - coldtea 3mo ago
    >when it's vibed it works, until it doesn't and then it's really hard to make it work again Is it? People have solved AI bugs with AI. If some vibe project eventually hits some bug and stops working, what exactly stops using AI to fix it? Is the idea that bugs will go beyond the limits of AI capability? If you meant to say that when an AI vibe coded project beyond some complexity it's difficult for a human coder to manually go through all the code they didn't write, understand it, and find the issue, sure.
    - ubercore 3mo ago
      The problem is the _way_ AI will solve an AI bug. I've seen the loop countless times. There's a creeping complexity and brittleness that creeps in over time as more and more complexity is left purely to the LLM agent. It will become unsustainable without a human understanding and making course corrections at some point.
      - coldtea 3mo ago
        In my experience, it just needs some high level guidance. And it's quite easy to ask an AI to refactor a certain way too.
        ubercore 3mo ago
        Been there done that. My point is that even with Fable being a big improvement, it still needs constant feedback. The loops themselves are a lot better, but it still needs judgement calls, and Fable will often take an odd direction, and if you don't catch it, that odd choice will compound as it continues to layer on top.
        coldtea 3mo ago
        >Been there done that. My point is that even with Fable being a big improvement, it still needs constant feedback. Even so, if it does 80% of the work itself, that's still a 5x improvement. Plus it keeps the human coder in control and in the loop (and in a job).
      - int_19h 3mo ago
        If you just keep throwing feature requests at an LLM, then yes, this happens. However it can self-correct if you specifically give it engineering debt / code cleanup as a task. And Fable in particular is very good at this exact thing.
    - LtWorf 3mo ago
      AI companies are unable to fix the bugs in their own text editors for years… no AI cannot fix bugs, clearly.
      - coldtea 3mo ago
        Doesn't matter what AI companies do, since AI companies just "move fast and break things" not caring for bug fixing but for iterating quickly on their agents. That's a business decision, not an AI limitation. If you use AI yourself, with a focus on bug fixing and stability, you'll find that AI can fix bugs just fine.
        LtWorf 3mo ago
        It matters, it shows the limits of the technology, and they have all the interest to showcase how good it is (and are failing to show it can fix bugs)
        coldtea 3mo ago
        They have little interest to "showcase how good it is with doing that" since (a) people already see it's good and are hooked, (b) they don't want to stop the pace of changes and fall behind on features by focusing on stability and bug fixing.
        LtWorf 3mo ago
        They literally could use unlimited tokens and focus on both… it's telling that they cannot. And they are TRYING to fix the bugs, they just keep failing over and over, so your reply is entirely incorrect. Nice try though.
        coldtea 3mo ago
        >They literally could use unlimited tokens and focus on both… it's telling that they cannot. If you ship updates fast, you can't just 'focus on both'. You focus on one or another, doesn't matter if you use "unlimited tokens", same way 9 pregnant women can't make a baby in a month. >And they are TRYING to fix the bugs, they just keep failing over and over, so your reply is entirely incorrect. That they "keep failing over and over" is a huge overstatement, it just has some bugs like other software has, so your point can be simply dismissed.
    - timacles 3mo ago
      AI will simply code you into an architectural corner where you can’t get out of without a refactor.
      - nielsbot 3mo ago
        to be fair that happens with code i write too…
      - coldtea 3mo ago
        Not if you give it guidance for the architecture and don't just blindly let it one-shot after one-shot of huge chunks of the program. Besides, AI can also be told to do the refactor.
    - deleted 3mo ago
      [deleted]
- kameit00 3mo ago
  In 12 months… vibe code mess. Or discontinued. Or both.
  - ttul 3mo ago
    How much time have you spent with Fable? We're in new territory here. It does not create messes.
    - ubercore 3mo ago
      Anecdote, yes, but I am _right now in the middle of helping Fable clean up a mess_. Complex code is hard and Fable still makes mistakes.
    - what 3mo ago
      >this time it’s different! Same thing people claim every time a new model is released, yet never seems to be true.
      - int_19h 3mo ago
        It was true every time though. The capacity of frontier models to tackle complicated issues has improved immensely. I still remember the first time I saw a model do a non-trivial issue end to end, and that was less than two years ago. Now they can genuinely do whole projects with human only as a supervisor / quality checker. Do they still make mistakes? Sure. So do humans, though, so it would be unrealistic to expect perfection. The question is: does Fable make fewer mistakes than the median human coder? And at this point I'm genuinely not sure anymore.
    - nozzlegear 3mo ago
      > We're in new territory here. > It does not create messes. ?
- mcphage 3mo ago
  Given the stdlib modules listed as "explicitly not done yet", I'm going to say: it doesn't yet, in any meaningful sense. The question then becomes: how confident do we feel that it will work in the near future?
  - ubercore 3mo ago
    I was trying to say "not confident at all" but hedged a bit too much. I see this as a case of the "quick to get to a POC that falls apart after sustained development for the same reasons it didn't work pre-Fable" problem.
- nozzlegear 3mo ago
  > it doesn't matter as long as it works. I think the clankers would call this a "load bearing statement".
  - nielsbot 3mo ago
    it reads like marketing copy…
- getpokedagain 3mo ago
  Something working is pointless if there are no users and no need is being addressed.
himata4113 3mo ago
This is written by fable with the guidance of a very experienced, highly skilled person. See their previous work.
- throwaway27448 3mo ago
  Experience doesn't change the fundamental problem. I don't see this project going anywhere for general use beyond their needs.
- Dilettante_ 3mo ago
  "Very experienced" might mean different things to you. The oldest repo on their GH is from 2017. As for highly skilled: Could you point closer to which parts of their portfolio we are supposed to be awestruck by?
  - not_a9 3mo ago
    https://github.com/vtil-project/VTIL-Core https://github.com/vtil-project/VTIL-Core https://github.com/can1357/selene https://github.com/can1357/selene
- roger_ 3mo ago
  This guy is behind the awesome Oh My Pi agent, so I’d give him a chance.
thx67 3mo ago
These tics are fairly easy to remove via hooks and prompts, but once the codebase is infected, it is 10x as much work to get the agents to stop.

baq 3mo ago

of course it is vibed. it doesn't matter as long as it works.

ActionHank 3mo ago
That's the neat part, when it's vibed it works, until it doesn't and then it's really hard to make it work again.
- coldtea 3mo ago
  >when it's vibed it works, until it doesn't and then it's really hard to make it work again Is it? People have solved AI bugs with AI. If some vibe project eventually hits some bug and stops working, what exactly stops using AI to fix it? Is the idea that bugs will go beyond the limits of AI capability? If you meant to say that when an AI vibe coded project beyond some complexity it's difficult for a human coder to manually go through all the code they didn't write, understand it, and find the issue, sure.
  - ubercore 3mo ago
    The problem is the _way_ AI will solve an AI bug. I've seen the loop countless times. There's a creeping complexity and brittleness that creeps in over time as more and more complexity is left purely to the LLM agent. It will become unsustainable without a human understanding and making course corrections at some point.
    - coldtea 3mo ago
      In my experience, it just needs some high level guidance. And it's quite easy to ask an AI to refactor a certain way too.
      - ubercore 3mo ago
        Been there done that. My point is that even with Fable being a big improvement, it still needs constant feedback. The loops themselves are a lot better, but it still needs judgement calls, and Fable will often take an odd direction, and if you don't catch it, that odd choice will compound as it continues to layer on top.
        coldtea 3mo ago
        >Been there done that. My point is that even with Fable being a big improvement, it still needs constant feedback. Even so, if it does 80% of the work itself, that's still a 5x improvement. Plus it keeps the human coder in control and in the loop (and in a job).
    - int_19h 3mo ago
      If you just keep throwing feature requests at an LLM, then yes, this happens. However it can self-correct if you specifically give it engineering debt / code cleanup as a task. And Fable in particular is very good at this exact thing.
  - LtWorf 3mo ago
    AI companies are unable to fix the bugs in their own text editors for years… no AI cannot fix bugs, clearly.
    - coldtea 3mo ago
      Doesn't matter what AI companies do, since AI companies just "move fast and break things" not caring for bug fixing but for iterating quickly on their agents. That's a business decision, not an AI limitation. If you use AI yourself, with a focus on bug fixing and stability, you'll find that AI can fix bugs just fine.
      - LtWorf 3mo ago
        It matters, it shows the limits of the technology, and they have all the interest to showcase how good it is (and are failing to show it can fix bugs)
        coldtea 3mo ago
        They have little interest to "showcase how good it is with doing that" since (a) people already see it's good and are hooked, (b) they don't want to stop the pace of changes and fall behind on features by focusing on stability and bug fixing.
        LtWorf 3mo ago
        They literally could use unlimited tokens and focus on both… it's telling that they cannot. And they are TRYING to fix the bugs, they just keep failing over and over, so your reply is entirely incorrect. Nice try though.
        coldtea 3mo ago
        >They literally could use unlimited tokens and focus on both… it's telling that they cannot. If you ship updates fast, you can't just 'focus on both'. You focus on one or another, doesn't matter if you use "unlimited tokens", same way 9 pregnant women can't make a baby in a month. >And they are TRYING to fix the bugs, they just keep failing over and over, so your reply is entirely incorrect. That they "keep failing over and over" is a huge overstatement, it just has some bugs like other software has, so your point can be simply dismissed.
  - timacles 3mo ago
    AI will simply code you into an architectural corner where you can’t get out of without a refactor.
    - nielsbot 3mo ago
      to be fair that happens with code i write too…
    - coldtea 3mo ago
      Not if you give it guidance for the architecture and don't just blindly let it one-shot after one-shot of huge chunks of the program. Besides, AI can also be told to do the refactor.
  - deleted 3mo ago
    [deleted]
kameit00 3mo ago
In 12 months… vibe code mess. Or discontinued. Or both.
- ttul 3mo ago
  How much time have you spent with Fable? We're in new territory here. It does not create messes.
  - ubercore 3mo ago
    Anecdote, yes, but I am _right now in the middle of helping Fable clean up a mess_. Complex code is hard and Fable still makes mistakes.
  - what 3mo ago
    >this time it’s different! Same thing people claim every time a new model is released, yet never seems to be true.
    - int_19h 3mo ago
      It was true every time though. The capacity of frontier models to tackle complicated issues has improved immensely. I still remember the first time I saw a model do a non-trivial issue end to end, and that was less than two years ago. Now they can genuinely do whole projects with human only as a supervisor / quality checker. Do they still make mistakes? Sure. So do humans, though, so it would be unrealistic to expect perfection. The question is: does Fable make fewer mistakes than the median human coder? And at this point I'm genuinely not sure anymore.
  - nozzlegear 3mo ago
    > We're in new territory here. > It does not create messes. ?
mcphage 3mo ago
Given the stdlib modules listed as "explicitly not done yet", I'm going to say: it doesn't yet, in any meaningful sense. The question then becomes: how confident do we feel that it will work in the near future?
- ubercore 3mo ago
  I was trying to say "not confident at all" but hedged a bit too much. I see this as a case of the "quick to get to a POC that falls apart after sustained development for the same reasons it didn't work pre-Fable" problem.
nozzlegear 3mo ago
> it doesn't matter as long as it works. I think the clankers would call this a "load bearing statement".
- nielsbot 3mo ago
  it reads like marketing copy…
getpokedagain 3mo ago
Something working is pointless if there are no users and no need is being addressed.

himata4113 3mo ago

This is written by fable with the guidance of a very experienced, highly skilled person. See their previous work.

throwaway27448 3mo ago
Experience doesn't change the fundamental problem. I don't see this project going anywhere for general use beyond their needs.
Dilettante_ 3mo ago
"Very experienced" might mean different things to you. The oldest repo on their GH is from 2017. As for highly skilled: Could you point closer to which parts of their portfolio we are supposed to be awestruck by?
- not_a9 3mo ago
  https://github.com/vtil-project/VTIL-Core https://github.com/vtil-project/VTIL-Core https://github.com/can1357/selene https://github.com/can1357/selene
roger_ 3mo ago
This guy is behind the awesome Oh My Pi agent, so I’d give him a chance.

thx67 3mo ago

These tics are fairly easy to remove via hooks and prompts, but once the codebase is infected, it is 10x as much work to get the agents to stop.