3 ms·

i find something a bit funny in an ai project, written by ai, requiring human-written text with specific guidance to not use ai. "Given that coding agents writ

i find something a bit funny in an ai project, written by ai, requiring human-written text with specific guidance to not use ai. "Given that coding agents write most underlying code now, we'd prefer PRs in the form of human-written text. [...] Please do not have AI artificially expand what you'd like to do into a formal proposal."

warkdarrior 3mo ago
They specifically ask not to create/submit a formal proposal. I do not see any restriction on the use of AI otherwise.
- john_strinlai 3mo ago
  i read the emphasized "human-written" part, in combination with that last line, as a blanket restriction (for contributing). but perhaps i read it wrong.
  - embedding-shape 3mo ago
    List of commits already include at least two contributors who are openly working with Claude on their code: https://github.com/yc-software/qm/commits/main/ https://github.com/yc-software/qm/commits/main/
  - tptacek 3mo ago
    The point is that they just want your prompts, not your code.
    - deleted 3mo ago
      [deleted]
    - john_strinlai 3mo ago
      yes, specifically they want purely human-written ones. i understand why they want their own agent to do the code, and i can see the reasoning behind it. but not allowing ai to format/tidy up/flesh out the proposal is the part that i thought was a little funny.
      - tptacek 3mo ago
        It's definitely a little bit funny. I think it's a good and interesting rule though!
ronsor 3mo ago
Coding agents are extremely useful but often extremely dumb with design. If you do not design the software yourself, you will probably get slop. This was always the core issue with vibe coding.
- j45 3mo ago
  AI Averages, and is inclined to do average designs and implementations, which in some cases might be an improvement, but long term it creates more to deal with.
  - bityard 3mo ago
    Hm. If "AI averages," then why doesn't it create an average amount to deal with, instead of more?
    - ronsor 3mo ago
      Because competent people are evaluating it. In places where the standards are rock-bottom, the AI is leagues ahead of anything they'd produce normally.
    - freeone3000 3mo ago
      Because the “average” developer is incompetent.
sigbottle 3mo ago
XY problem, don't expand the design doc before you have the idea for the design. Don't compile down into lower abstraction levels until necessary
stefan_ 3mo ago
Yet the README is generated ("Two skills maintain the boundary in both directions.") and the demo is .. a Mobius strip? I'd prefer if you explain what it is you are building in the form of human-written text.
- jaggederest 3mo ago
  Well, if they're consistent, the ADR directory contains only human written text that drives the underlying development... Aaand it's empty. Well. I personally would probably have the readme generated based on that directory as the primary document, probably with another `readme_generation_rules.md` in the ADRs directory, and I would be pretty ruthless about disallowing all the slop-adjacent wording.
  - titanomachy 3mo ago
    > I would be pretty ruthless about disallowing all the slop-adjacent wording. I’d think that trying to play whack-a-mole with slop like that would result in a lengthy prompt, as well as being brittle to future changes. If you want the prompt to be the source of truth, you’re probably going to have to accept prose that feels like AI, at least until you can regenerate with some future smarter model.
    - jaggederest 3mo ago
      Not even close :) It's really, really good at making natural-sounding prose if you demand it, set clear guidelines (ideally a script it can run with grading, but...), Fable in particular is eerie about mimicking prose style. Try RFC style if you want really clear architecture documents with MUST/SHOULD etc. It will, of course, be slightly worse at the non-style task (i.e. make the readme worse), but if you care about the style, that's probably an acceptable tradeoff.
petesergeant 3mo ago
That's not surprising. I have much the same thing in my latest open-source work[0]. I have a workflow that's working nicely with agents, it's more of a pain to review someone else's code than it is to get a well-written description of their problem. 0: https://github.com/pjlsergeant/byre?tab=contributing-ov-file https://github.com/pjlsergeant/byre?tab=contributing-ov-file
2001zhaozhao 3mo ago
i think it makes perfect sense and I'll probably adopt the same posture for my open source project. The idea is that the core dev team is the one with the AI harness. If someone contributes an idea they can just feed it to their AI to implement it and it probably costs 10 minutes of human time, because they trust that their harness will be able to implement the feature competently. So, because the implementation effort is so low, the only aspect that matters if the quality of the idea. It's way easier to screen human written text than a bunch of code in a PR. If you just give them code, they would not know if you made it with a competent harness. If you gave them a AI written design they would have a lot more to read through to decide whether it's slop. If you just give them an idea it's a lot easier to determine whether it's high quality.

warkdarrior 3mo ago

They specifically ask not to create/submit a formal proposal. I do not see any restriction on the use of AI otherwise.

john_strinlai 3mo ago
i read the emphasized "human-written" part, in combination with that last line, as a blanket restriction (for contributing). but perhaps i read it wrong.
- embedding-shape 3mo ago
  List of commits already include at least two contributors who are openly working with Claude on their code: https://github.com/yc-software/qm/commits/main/ https://github.com/yc-software/qm/commits/main/
- tptacek 3mo ago
  The point is that they just want your prompts, not your code.
  - deleted 3mo ago
    [deleted]
  - john_strinlai 3mo ago
    yes, specifically they want purely human-written ones. i understand why they want their own agent to do the code, and i can see the reasoning behind it. but not allowing ai to format/tidy up/flesh out the proposal is the part that i thought was a little funny.
    - tptacek 3mo ago
      It's definitely a little bit funny. I think it's a good and interesting rule though!

ronsor 3mo ago

Coding agents are extremely useful but often extremely dumb with design. If you do not design the software yourself, you will probably get slop. This was always the core issue with vibe coding.

j45 3mo ago
AI Averages, and is inclined to do average designs and implementations, which in some cases might be an improvement, but long term it creates more to deal with.
- bityard 3mo ago
  Hm. If "AI averages," then why doesn't it create an average amount to deal with, instead of more?
  - ronsor 3mo ago
    Because competent people are evaluating it. In places where the standards are rock-bottom, the AI is leagues ahead of anything they'd produce normally.
  - freeone3000 3mo ago
    Because the “average” developer is incompetent.

sigbottle 3mo ago

XY problem, don't expand the design doc before you have the idea for the design. Don't compile down into lower abstraction levels until necessary

stefan_ 3mo ago

Yet the README is generated ("Two skills maintain the boundary in both directions.") and the demo is .. a Mobius strip? I'd prefer if you explain what it is you are building in the form of human-written text.

jaggederest 3mo ago
Well, if they're consistent, the ADR directory contains only human written text that drives the underlying development... Aaand it's empty. Well. I personally would probably have the readme generated based on that directory as the primary document, probably with another `readme_generation_rules.md` in the ADRs directory, and I would be pretty ruthless about disallowing all the slop-adjacent wording.
- titanomachy 3mo ago
  > I would be pretty ruthless about disallowing all the slop-adjacent wording. I’d think that trying to play whack-a-mole with slop like that would result in a lengthy prompt, as well as being brittle to future changes. If you want the prompt to be the source of truth, you’re probably going to have to accept prose that feels like AI, at least until you can regenerate with some future smarter model.
  - jaggederest 3mo ago
    Not even close :) It's really, really good at making natural-sounding prose if you demand it, set clear guidelines (ideally a script it can run with grading, but...), Fable in particular is eerie about mimicking prose style. Try RFC style if you want really clear architecture documents with MUST/SHOULD etc. It will, of course, be slightly worse at the non-style task (i.e. make the readme worse), but if you care about the style, that's probably an acceptable tradeoff.

petesergeant 3mo ago

That's not surprising. I have much the same thing in my latest open-source work[0]. I have a workflow that's working nicely with agents, it's more of a pain to review someone else's code than it is to get a well-written description of their problem. 0: https://github.com/pjlsergeant/byre?tab=contributing-ov-file https://github.com/pjlsergeant/byre?tab=contributing-ov-file

2001zhaozhao 3mo ago

i think it makes perfect sense and I'll probably adopt the same posture for my open source project. The idea is that the core dev team is the one with the AI harness. If someone contributes an idea they can just feed it to their AI to implement it and it probably costs 10 minutes of human time, because they trust that their harness will be able to implement the feature competently. So, because the implementation effort is so low, the only aspect that matters if the quality of the idea. It's way easier to screen human written text than a bunch of code in a PR. If you just give them code, they would not know if you made it with a competent harness. If you gave them a AI written design they would have a lot more to read through to decide whether it's slop. If you just give them an idea it's a lot easier to determine whether it's high quality.