3 ms·

> - Negative: Submitters just add stylistic markers to make their accounts and output seem human-generated. This is like syntactic sugar: the core content and t

by ivorius 3mo ago

> - Negative: Submitters just add stylistic markers to make their accounts and output seem human-generated. This is like syntactic sugar: the core content and the size of contributions stay the same, but the style gets quirkier. From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much. > Positive: Submitters actually provide to-the-point, no-bullshit commits and comments That would be a dream.

watwut 3mo ago
> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much. The policy allows the reviewer to reject it on the "AI" grounds.
- dspillett 3mo ago
  > allows the reviewer to reject it on the "AI" grounds … but still unfortunately leaves reviewers having to spend time checking submissions and rejecting them.
  - fendy3002 3mo ago
    oh but it'll be very2 helpful and the time spent will be short. It's easy to verify: * new contributor? * more than 10 files affected (higher count are more valid)? * wall of text on description without screenshots, etc? just close the PR as AI, and then the contributor can challenge it if they feel it should not.
    - HelloNurse 3mo ago
      A contributor in good faith is going to accept criticism and resubmit an improved change: less files modified, more explanation, more focus, references to actual tickets and discussion with actual developers.
  - jon-wood 3mo ago
    At least half the people firing off LLM generated PRs will have left the "Coauthored-by: Claude" line on it allowing automated rejections.
    - ivorius 3mo ago
      Unfortunately, only a single PR like this comes to mind. Most AI authors we've seen were identifiable mainly by overly verbose PR descriptions, meaningless code changes and copy-pasting more AI output when questioned.
QuantumNomad_ 3mo ago
> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much. True. At least with a policy about it, the project maintainers can unilaterally close such PRs without further internal or external discussion on any case-by-case basis.
- maybewhenthesun 3mo ago
  Dingdingding, we have a winner. The main use of such a policy is to be able to just close those giant wall-of-text PRs and have something to point to when people start to scream it's not fair.
  - mcphage 3mo ago
    > when people start to scream it's not fair Or LLMs, as we have seen.
    - chrisjj 3mo ago
      Or, who knows, the "AI" might gain sufficient intelligence to read the policy...
      - julianlam 3mo ago
        Then we'll amend the policy to instruct LLMs to author PRs as Fred Flintstone, yabba dabba doo!
  - tayo42 3mo ago
    Why is a policy necessary. you were never entitled to have your pr merged in the first place? If pr wasn't reviewable pre AI I'd expect it to be closed or ignored too
    - the_hoser 3mo ago
      The policy isn't necessary to close the PR. The policy just helps to shut down the ensuing discussion after closing the PR. It helps in quickly dealing with well-meaning onlookers asking for clarification when you block PRs from the account.
whateverboat 3mo ago
But now with AI, this should be "easier" for some definition of easy. In the sense that in the past, this might have taken 15 minutes to write, now with AI, this can take 5 minutes to write by first getting AI to produce a summary and then using human judgement to make it better. So, it's a good idea now to actually demand the dream.
- ivorius 3mo ago
  If people knew how to get AI to write terse, focused summaries, sure, that might help. I haven't seen many that do (well, ignoring the toupee fallacy). Though the most important aspect is that we need to know the motivation and thought process, and all AI can do is fabricate a 'plausible' one.
  - adalacelove 3mo ago
    Reading AI PRs reminds me of Monty Python's holy grenade: "And the Lord spake, saying, ''First shalt thou take out the Holy Pin. Then shalt thou count to three, no more, no less. Three shall be the number thou shalt count, and the number of the counting shall be three. Four shalt thou not count, neither count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thy foe, who, being naughty in My sight, shall snuff it.'
    - dsign 3mo ago
      I wouldn't mind reading that and having a good chuckle while processing an MR, as long as the comment had been crafted by a person. But now I think writing in grunts is gonna become the thing. "pete? you good boy? we won't hire you/ paper you gave HR girl with older gigs? remember it pete my boy? too many words/ words were too long/ dots you used dots/ you scratched long dash with knife, but baby saw scratch/we don't ai pete/ask if they have job in next cave/good luck."
  - ray_kay777 3mo ago
    I've found that the instructions "be extremely concise" gets me much closer to output that's actually sensible/helpful rather than another wall of text.
snarfy 3mo ago
They could allow AI PRs, but then have another AI PR reviewer reject them if they do not match the definitions for `to-the-point` `no-bullshit` commits.
- mort96 3mo ago
  And who pays for the (likely significant, and controllable by everyone) tokens such a system would use?
  - julianlam 3mo ago
    A small 4-9B model would be able to run cheaply for this sort of work.
    - krainboltgreene 3mo ago
      What question do you think you're answering?
      - Cpoll 3mo ago
        The Godot Foundation pays for it if it furthers their mission.
        mort96 3mo ago
        Does it?
- skeeter2020 3mo ago
  Please provide 3 examples where layering on MORE of the offending technology has solved the problem. Spam? Malware & Viruses? Customer Service? Hiring & Recruitment?
paulddraper 3mo ago
> That would be a dream. “Mission. Fucking. Accomplished.” https://xkcd.com/810/ https://xkcd.com/810/
stronglikedan 3mo ago
> That would be a dream. We just recently started that policy so we'll see how it goes. If anything, having it stated as policy lets us filter out these requests without spending brain tokens on them.
DonsDiscountGas 3mo ago
I've always instructed Claude to check the policies first, frankly I'm surprised it's not smart enough to do that already. Would be easy to add to a system prompt. But usually it doesn't matter because many projects have no policies, or maybe they exist but only in hidden forum posts issues or something.

by ivorius 3mo ago

watwut 3mo ago

> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much. The policy allows the reviewer to reject it on the "AI" grounds.

dspillett 3mo ago
> allows the reviewer to reject it on the "AI" grounds … but still unfortunately leaves reviewers having to spend time checking submissions and rejecting them.
- fendy3002 3mo ago
  oh but it'll be very2 helpful and the time spent will be short. It's easy to verify: * new contributor? * more than 10 files affected (higher count are more valid)? * wall of text on description without screenshots, etc? just close the PR as AI, and then the contributor can challenge it if they feel it should not.
  - HelloNurse 3mo ago
    A contributor in good faith is going to accept criticism and resubmit an improved change: less files modified, more explanation, more focus, references to actual tickets and discussion with actual developers.
- jon-wood 3mo ago
  At least half the people firing off LLM generated PRs will have left the "Coauthored-by: Claude" line on it allowing automated rejections.
  - ivorius 3mo ago
    Unfortunately, only a single PR like this comes to mind. Most AI authors we've seen were identifiable mainly by overly verbose PR descriptions, meaningless code changes and copy-pasting more AI output when questioned.

QuantumNomad_ 3mo ago

> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much. True. At least with a policy about it, the project maintainers can unilaterally close such PRs without further internal or external discussion on any case-by-case basis.

maybewhenthesun 3mo ago
Dingdingding, we have a winner. The main use of such a policy is to be able to just close those giant wall-of-text PRs and have something to point to when people start to scream it's not fair.
- mcphage 3mo ago
  > when people start to scream it's not fair Or LLMs, as we have seen.
  - chrisjj 3mo ago
    Or, who knows, the "AI" might gain sufficient intelligence to read the policy...
    - julianlam 3mo ago
      Then we'll amend the policy to instruct LLMs to author PRs as Fred Flintstone, yabba dabba doo!
- tayo42 3mo ago
  Why is a policy necessary. you were never entitled to have your pr merged in the first place? If pr wasn't reviewable pre AI I'd expect it to be closed or ignored too
  - the_hoser 3mo ago
    The policy isn't necessary to close the PR. The policy just helps to shut down the ensuing discussion after closing the PR. It helps in quickly dealing with well-meaning onlookers asking for clarification when you block PRs from the account.

whateverboat 3mo ago

But now with AI, this should be "easier" for some definition of easy. In the sense that in the past, this might have taken 15 minutes to write, now with AI, this can take 5 minutes to write by first getting AI to produce a summary and then using human judgement to make it better. So, it's a good idea now to actually demand the dream.

ivorius 3mo ago
If people knew how to get AI to write terse, focused summaries, sure, that might help. I haven't seen many that do (well, ignoring the toupee fallacy). Though the most important aspect is that we need to know the motivation and thought process, and all AI can do is fabricate a 'plausible' one.
- adalacelove 3mo ago
  Reading AI PRs reminds me of Monty Python's holy grenade: "And the Lord spake, saying, ''First shalt thou take out the Holy Pin. Then shalt thou count to three, no more, no less. Three shall be the number thou shalt count, and the number of the counting shall be three. Four shalt thou not count, neither count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thy foe, who, being naughty in My sight, shall snuff it.'
  - dsign 3mo ago
    I wouldn't mind reading that and having a good chuckle while processing an MR, as long as the comment had been crafted by a person. But now I think writing in grunts is gonna become the thing. "pete? you good boy? we won't hire you/ paper you gave HR girl with older gigs? remember it pete my boy? too many words/ words were too long/ dots you used dots/ you scratched long dash with knife, but baby saw scratch/we don't ai pete/ask if they have job in next cave/good luck."
- ray_kay777 3mo ago
  I've found that the instructions "be extremely concise" gets me much closer to output that's actually sensible/helpful rather than another wall of text.

snarfy 3mo ago

They could allow AI PRs, but then have another AI PR reviewer reject them if they do not match the definitions for `to-the-point` `no-bullshit` commits.

mort96 3mo ago
And who pays for the (likely significant, and controllable by everyone) tokens such a system would use?
- julianlam 3mo ago
  A small 4-9B model would be able to run cheaply for this sort of work.
  - krainboltgreene 3mo ago
    What question do you think you're answering?
    - Cpoll 3mo ago
      The Godot Foundation pays for it if it furthers their mission.
      - mort96 3mo ago
        Does it?
skeeter2020 3mo ago
Please provide 3 examples where layering on MORE of the offending technology has solved the problem. Spam? Malware & Viruses? Customer Service? Hiring & Recruitment?

paulddraper 3mo ago

> That would be a dream. “Mission. Fucking. Accomplished.” https://xkcd.com/810/ https://xkcd.com/810/

stronglikedan 3mo ago

> That would be a dream. We just recently started that policy so we'll see how it goes. If anything, having it stated as policy lets us filter out these requests without spending brain tokens on them.

DonsDiscountGas 3mo ago

I've always instructed Claude to check the policies first, frankly I'm surprised it's not smart enough to do that already. Would be easy to add to a system prompt. But usually it doesn't matter because many projects have no policies, or maybe they exist but only in hidden forum posts issues or something.