3 ms·

This seems like a "we've banned you and will ban any account deemed to be ban-evading" situation. OSS and the whole culture of open PRs requires a certain assum

by pjc50 9mo ago

This seems like a "we've banned you and will ban any account deemed to be ban-evading" situation. OSS and the whole culture of open PRs requires a certain assumption of good faith, which is not something that an AI is capable of on its own and is not a privilege which should be granted to AI operators. I suspect the culture will have to retreat back behind the gates at some point, which will be very sad and shrink it further.

midnitewarrior 9mo ago
Do that and the AI might fork the repo, address all the outstanding issues and split your users. The code quality may not be there now, but it will be soon.
- sethops1 9mo ago
  This is a fantasy that virtually never comes to fruition. The vast majority of forks are dead within weeks when the forkers realize how much effort goes into building and maintaining the project, on top of starting with zero users.
  - thephyber 9mo ago
    While true, there are projects which surmount these hurdles because the people involved realize how important the project is. Given projects which are important enough, the bots will organize and coordinate. This is how that Anthropic developer got several agents to work in parallel to write a C compiler using Rust, granted he created the coordination framework.
  - newswasboring 9mo ago
    I think the difference now (in case code quality is solved with LLMs) is the cost of effort is now approaching zero.
    - PurpleRamen 9mo ago
      Good enough AI is not cheap (yet). So at the moment it's more a scenario for people who are rich enough. Though, small projects with little maintenance-burden might be at a risk here. But thinking about, this might be a new danger to get us into another xz-utils-situation. The big malicious actors have enough money to waste and can scale up the amount of projects they attack and hijack, or even build themselves.
      - HanClinto 9mo ago
        XZ Utils is EXACTLY what came to mind for me. That exploit / takeover happened precisely because an angry user was bullying a project maintainer, and then a "white knight" came in to save the day and defend the maintainer against the demanding users. In reality, both the problem and the solution were manufactured by the social engineer, but bullying the maintainer was the vector that this exploited. What happens when agents are used to do this sort of thing at scale?
  - dormento 9mo ago
    This might be true today, but think about it. This is a new scenario, where a giga-brain-sized <insert_role_here> works tirelessly 24/7 improving code. Imagine it starts to fork repos. Imagine it can eventually outpace human contributors, not only on volume (which it already can), but in attention to detail and usefulness of resulting code. Now imagine the forks overtake the original projects. This is not just "Will Smith eating spaghetti", its a real breaking point. I'm equal parts frightened and amazed.
    - sethops1 9mo ago
      If your bot is actually capable of doing as your say, why waste time forking OSS repos? Why not instruct it to start 1000 new tech startups and start generating you tons of money? I can "think about" winning the lottery with just as much rigor and effect as day dreaming about the kind of all encompassing intelligence you describe. Maybe it's time to stop being "frightened and amazed" and come back to reality.
      - pixl97 9mo ago
        Because everyone's bot will be trying to start a business and make money at the same time, hence nobody will make any money. Now technically if bots actually improve tech the cost of actual products/services will go down Because of competition. You keep saying 'come back to reality' but drag someone from 200 years ago to today and our reality would be so shocking they may not recover from it.
- bayindirh 9mo ago
  > The code quality may not be there now, but it will be soon. I'm hearing this exact argument since 2002 or so. Even Duke Nukem Forever has been released in this time frame. I bet even Tesla might solve Autopilot(TM) problems before this becomes a plausible reality.
  - pixl97 9mo ago
    I mean in 1850 I kept hearing heavier than air flight was just a year away, and yet here we are without heavier than air flight...
- acedTrex 9mo ago
  I am perfectly willing to take that risk. Hell i'll even throw ten bucks on it while we are here.
bayindirh 9mo ago
> I suspect the culture will have to retreat back behind the gates at some point, which will be very sad and shrink it further. I'm personally contemplating not publishing the code I write anymore. The things I write are not world-changing and GPLv3+ licensed only, but I was putting them out just in case somebody would find it useful. However, I don't want my code scraped and remixed by AI systems. Since I'm doing this for personal fun and utility, who cares about my code being in the open. I just can write and use it myself. Putting it outside for humans to find it was fun, while it lasted. Now everything is up for grabs, and I don't play that game.
- 20k 9mo ago
  Its astonishing the way that we've just accepted mass theft of copyright. There appears to be no way to stop AI companies from stealing your work and selling it on for profits On the plus side: It only takes a small fraction of people deliberately poisoning their work to significantly lower the quality, so perhaps consider publishing it with deliberate AI poisoning built in
  - zero_shift 9mo ago
    Would publishing under AGPL count as poisoning? Or even with an explicit "this is not licensed" license
    - deleted 9mo ago
      [deleted]
    - thephyber 9mo ago
      Your licensing only matters if you are willing to enforce it. That costs lawyer money and a will to spend your time. This won’t be solved by individuals withholding their content. Everything you have already contributed to (including GitHub, StackOverflow, etc) has already been trained. The most powerful thing we can do is band together, lobby Congress, and get intellectual property laws changes to support Americans. There’s no way courts have the bandwidth to react to this reactively.
  - 9mo ago
casey2 9mo ago
We have webs of trust, just swap router/packet with PID/PR Then the maintainer can see something like 10-1 accepted/rejected for first layer (direct friends) 1000-40 for layer two (friends of friends) and so own. Then you can directly message any public ID or see any PR. This can help agents too since they can see all their agent buddies have a 0% success rate they won't bother
functionmouse 9mo ago
Better my gates than Bill Gates The moment Microsoft bought GitHub it was over
jcgrillo 9mo ago
The tooling amplifies the problem. I've become increasingly skeptical of the "open contributions" model Github and their ilk default to. I'd rather the tooling default be "look but don't touch"--fully gate-kept. If I want someone to collaborate with me I'll reach out to that person and solicit their assistance in the form of pull requests or bug reports. I absolutely never want random internet entities "helping". Developing in the open seems like a great way to do software. Developing with an "open team" seems like the absolute worst. We are careful when we choose colleagues, we test them, interview them.. so why would we let just anyone start slinging trash at our code review tools and issue trackers? A well kept gate keeps the rabble out.
snigsnog 9mo ago
Yeah there's unfortunately just no way true OSS in the way we've enjoyed it survives the new era of AI and all of India coming online.
TuxSH 9mo ago
> This seems like a "we've banned you and will ban any account deemed to be ban-evading" Honestly, if faced with such a situation, instead of just blocking, I would report the acc to GH Support, so that they nuke the account and its associated PRs/issues.
stephen_g 9mo ago
Yes, hard to see how LLM agents won't destroy all online spaces unless they all go behind closed doors with some kind of physical verification of human-ness (like needing a real-world meetup with another member or something before being admitted). Even if 99.999% of the population deploy them responsibly, it only takes a handful of trolls (or well-meaning but very misguided people) to flood every comment section, forum, open source project, etc. with far more crap than any maintainer can ever handle... I guess I can be glad I got to experience a bit more than 20 years of the pre-LLM internet, but damn it's sad thinking about where things are going to go now.

by pjc50 9mo ago

midnitewarrior 9mo ago

Do that and the AI might fork the repo, address all the outstanding issues and split your users. The code quality may not be there now, but it will be soon.

sethops1 9mo ago
This is a fantasy that virtually never comes to fruition. The vast majority of forks are dead within weeks when the forkers realize how much effort goes into building and maintaining the project, on top of starting with zero users.
- thephyber 9mo ago
  While true, there are projects which surmount these hurdles because the people involved realize how important the project is. Given projects which are important enough, the bots will organize and coordinate. This is how that Anthropic developer got several agents to work in parallel to write a C compiler using Rust, granted he created the coordination framework.
- newswasboring 9mo ago
  I think the difference now (in case code quality is solved with LLMs) is the cost of effort is now approaching zero.
  - PurpleRamen 9mo ago
    Good enough AI is not cheap (yet). So at the moment it's more a scenario for people who are rich enough. Though, small projects with little maintenance-burden might be at a risk here. But thinking about, this might be a new danger to get us into another xz-utils-situation. The big malicious actors have enough money to waste and can scale up the amount of projects they attack and hijack, or even build themselves.
    - HanClinto 9mo ago
      XZ Utils is EXACTLY what came to mind for me. That exploit / takeover happened precisely because an angry user was bullying a project maintainer, and then a "white knight" came in to save the day and defend the maintainer against the demanding users. In reality, both the problem and the solution were manufactured by the social engineer, but bullying the maintainer was the vector that this exploited. What happens when agents are used to do this sort of thing at scale?
- dormento 9mo ago
  This might be true today, but think about it. This is a new scenario, where a giga-brain-sized <insert_role_here> works tirelessly 24/7 improving code. Imagine it starts to fork repos. Imagine it can eventually outpace human contributors, not only on volume (which it already can), but in attention to detail and usefulness of resulting code. Now imagine the forks overtake the original projects. This is not just "Will Smith eating spaghetti", its a real breaking point. I'm equal parts frightened and amazed.
  - sethops1 9mo ago
    If your bot is actually capable of doing as your say, why waste time forking OSS repos? Why not instruct it to start 1000 new tech startups and start generating you tons of money? I can "think about" winning the lottery with just as much rigor and effect as day dreaming about the kind of all encompassing intelligence you describe. Maybe it's time to stop being "frightened and amazed" and come back to reality.
    - pixl97 9mo ago
      Because everyone's bot will be trying to start a business and make money at the same time, hence nobody will make any money. Now technically if bots actually improve tech the cost of actual products/services will go down Because of competition. You keep saying 'come back to reality' but drag someone from 200 years ago to today and our reality would be so shocking they may not recover from it.
bayindirh 9mo ago
> The code quality may not be there now, but it will be soon. I'm hearing this exact argument since 2002 or so. Even Duke Nukem Forever has been released in this time frame. I bet even Tesla might solve Autopilot(TM) problems before this becomes a plausible reality.
- pixl97 9mo ago
  I mean in 1850 I kept hearing heavier than air flight was just a year away, and yet here we are without heavier than air flight...
acedTrex 9mo ago
I am perfectly willing to take that risk. Hell i'll even throw ten bucks on it while we are here.

bayindirh 9mo ago

> I suspect the culture will have to retreat back behind the gates at some point, which will be very sad and shrink it further. I'm personally contemplating not publishing the code I write anymore. The things I write are not world-changing and GPLv3+ licensed only, but I was putting them out just in case somebody would find it useful. However, I don't want my code scraped and remixed by AI systems. Since I'm doing this for personal fun and utility, who cares about my code being in the open. I just can write and use it myself. Putting it outside for humans to find it was fun, while it lasted. Now everything is up for grabs, and I don't play that game.

20k 9mo ago
Its astonishing the way that we've just accepted mass theft of copyright. There appears to be no way to stop AI companies from stealing your work and selling it on for profits On the plus side: It only takes a small fraction of people deliberately poisoning their work to significantly lower the quality, so perhaps consider publishing it with deliberate AI poisoning built in
- zero_shift 9mo ago
  Would publishing under AGPL count as poisoning? Or even with an explicit "this is not licensed" license
  - deleted 9mo ago
    [deleted]
  - thephyber 9mo ago
    Your licensing only matters if you are willing to enforce it. That costs lawyer money and a will to spend your time. This won’t be solved by individuals withholding their content. Everything you have already contributed to (including GitHub, StackOverflow, etc) has already been trained. The most powerful thing we can do is band together, lobby Congress, and get intellectual property laws changes to support Americans. There’s no way courts have the bandwidth to react to this reactively.
- 9mo ago

casey2 9mo ago

We have webs of trust, just swap router/packet with PID/PR Then the maintainer can see something like 10-1 accepted/rejected for first layer (direct friends) 1000-40 for layer two (friends of friends) and so own. Then you can directly message any public ID or see any PR. This can help agents too since they can see all their agent buddies have a 0% success rate they won't bother

functionmouse 9mo ago

Better my gates than Bill Gates The moment Microsoft bought GitHub it was over

jcgrillo 9mo ago

The tooling amplifies the problem. I've become increasingly skeptical of the "open contributions" model Github and their ilk default to. I'd rather the tooling default be "look but don't touch"--fully gate-kept. If I want someone to collaborate with me I'll reach out to that person and solicit their assistance in the form of pull requests or bug reports. I absolutely never want random internet entities "helping". Developing in the open seems like a great way to do software. Developing with an "open team" seems like the absolute worst. We are careful when we choose colleagues, we test them, interview them.. so why would we let just anyone start slinging trash at our code review tools and issue trackers? A well kept gate keeps the rabble out.

snigsnog 9mo ago

Yeah there's unfortunately just no way true OSS in the way we've enjoyed it survives the new era of AI and all of India coming online.

TuxSH 9mo ago

> This seems like a "we've banned you and will ban any account deemed to be ban-evading" Honestly, if faced with such a situation, instead of just blocking, I would report the acc to GH Support, so that they nuke the account and its associated PRs/issues.

stephen_g 9mo ago

Yes, hard to see how LLM agents won't destroy all online spaces unless they all go behind closed doors with some kind of physical verification of human-ness (like needing a real-world meetup with another member or something before being admitted). Even if 99.999% of the population deploy them responsibly, it only takes a handful of trolls (or well-meaning but very misguided people) to flood every comment section, forum, open source project, etc. with far more crap than any maintainer can ever handle... I guess I can be glad I got to experience a bit more than 20 years of the pre-LLM internet, but damn it's sad thinking about where things are going to go now.