3 ms·

Aside from obviously malicious bulk spamming like this, I'm seeing project maintainers get increasingly exasperated at users asking for help to fix some nonsens

Aside from obviously malicious bulk spamming like this, I'm seeing project maintainers get increasingly exasperated at users asking for help to fix some nonsense code which they eventually admit was generated by ChatGPT after some prodding. One Discord server I'm in has already instituted a "thou shalt not use ChatGPT" rule because it's wasting so much of the regulars time on deciphering confidently-wrong AI gibberish that someone took at face value. Whoever said that generative AI was going to raise the internets noise floor was right on the money.

jstarfish 4y ago
Friendly PSA: If you're going to use ChatGPT to generate code, do everyone a favor and add something like "...and write a passing unit test" to the prompt. If you want to really do everyone a favor, consider that TDD has never been easier. Don't write a prompt, write a unit test-- and instruct GPT to write the code that makes it pass. (Don't ask AI to do your work...ask it to finish your work.)
- NhanH 4y ago
  ChatGPT tends to over mock the module to the point it will always pass the test, so except for utilities module, this isn't helping much either. This is especially common in JS environment. If you think about it, it does make sense why: the prompt tends to have no where near the context of how the module should be delineated. Normally the programmer knows how the abstraction within his program is separated, but we have no good way to tell it to chatGPT. That means unless your program structure is still following one of the common structure and hasn't morphed into your business domain structure, ChatGPT will mock the wrong layer/ abstraction in the test suite.
  - jstarfish 4y ago
    I don't know JS, but it works pretty well for me in Python. Where it goes astray, I'll put comments indicating what it's supposed to do and GPT tends to pick up on it. It's like instruction-based inpainting/outpainting. The biggest hurdle is context limit, especially from retained comments. The real benefit to my system is that it weeds out the zero-effort clowns by forcing them to validate their own work before I waste my time trying to do anything with it.
- ipaddr 4y ago
  Now you have a test that passes but checks something unrelated.
  - Vanit 4y ago
    You ask ChatGPT to write that test too (:
    - bheadmaster 4y ago
      Now you have a test that passes its test but the latter checks for something unrelated...
- webjunkie 4y ago
  This this this. You can ask it to write a test. You can ask it to fix it, if the test does not pass. You can ask it to check whether the code it wrote makes sense, and it will surely find most times that there is something to improve. Urgh, it's not "one prompt and done".
TallGuyShort 4y ago
Curious if you can share any examples of this happening?
- jhugo 4y ago
  Not OP, but I have also seen this happening across multiple forums, Discord servers and IRC channels, and also in fields outside of tech. Someone wants to look smart, so they get GPT to write a post for them and paste it in as if it was their own response to a question. For example, I'm in some aviation-related discussion groups, and people will often ask technical questions which are best answered from a manual. In the past, eventually someone with access to the relevant manual would post an excerpt, answering the question. Nowadays, commonly someone will paste something from ChatGPT, without disclosing that it's just language model output. It will have the form of a correct response, but with invented details, and a few pages of confusion will result while people try to understand the implications of whatever GPT dreamed up, until someone with access to the real manual comes along, after which the original poster will usually admit that they copy-pasted from ChatGPT. I'm not sure if these people genuinely believe that a language model is somehow a font of knowledge, or if, like GPT, they don't care about whether something is accurate or not, but just care about whether other people believe them.
  - skummetmaelk 4y ago
    The missile knows where it is at all times. It knows this because it knows where it isn't. By subtracting where it is from where it isn't, or where it isn't from where it is (whichever is greater), it obtains a difference, or deviation. The guidance subsystem uses deviations to generate corrective commands to drive the missile from a position where it is to a position where it isn't, and arriving at a position where it wasn't, it now is. Consequently, the position where it is, is now the position that it wasn't, and it follows that the position that it was, is now the position that it isn't. In the event that the position that it is in is not the position that it wasn't, the system has acquired a variation, the variation being the difference between where the missile is, and where it wasn't. If variation is considered to be a significant factor, it too may be corrected by the GEA. However, the missile must also know where it was. The missile guidance computer scenario works as follows. Because a variation has modified some of the information the missile has obtained, it is not sure just where it is. However, it is sure where it isn't, within reason, and it knows where it was. It now subtracts where it should be from where it wasn't, or vice-versa, and by differentiating this from the algebraic sum of where it shouldn't be, and where it was, it is able to obtain the deviation and its variation, which is called error.
BlackjackCF 4y ago
What we need is someone to troll respond with ChatGPT generated responses.
- AJ007 4y ago
  This is a good idea, and probably will be the most likely path for dealing with LLM generated spam in general. Additionally, you could flag a spam contributor and then cloak them in to a parallel hole of nonsense. Of course the Microsoft issue is raised here. Probably this means if Microsoft ignores it, open source projects are going to have to move off of GitHub on to a platform they control and is explicitly designed to deal with spam contributions.
waboremo 4y ago
The internet's noise floor rising was always going to be a given with generative AI, but man to see how low effort, lazy, and haphazard it has been... disappointing. Everyone's been so scared of intentionally malicious groups, we all forgot about the average lazy fraud who wanted to boost his github commit graph, make his subreddit seem alive, or whatever the dumb case may be.
- BHSPitMonkey 4y ago
  The malicious groups benefit from all of the added noise acting as a smokescreen (in fact, the smart ones probably have a part of their fleet intentionally contributing to it in their victim communities)
- wpietri 4y ago
  That's been at the top of my list of concerns for quite a while. In Frankfurt's definition, bullshit is "speech intended to persuade without regard for truth". That's exactly what LLMs are good at. With no concept of truth or ability to verify anything, they're just good at plausible words. Which means they're ideal tools for bullshitters of all sorts.
  - behringer 4y ago
    Woah it's called a hallucination!
AraceliHarker 4y ago
They can't say "don't use ChatGPT" out loud, since GitHub itself sells Copilot, which is based on GPT.
nico 4y ago
This will always happen when a massive amount of people suddenly get access to a new technology There will be a learning and adaptation period for everyone This is also an opportunity for the maintainers to automate their responses and figure out how to chatgptize themselves (at least in terms of answering previously answered issues/questions) Presumably there will always be a need for a person to answer the novel questions

jstarfish 4y ago

Friendly PSA: If you're going to use ChatGPT to generate code, do everyone a favor and add something like "...and write a passing unit test" to the prompt. If you want to really do everyone a favor, consider that TDD has never been easier. Don't write a prompt, write a unit test-- and instruct GPT to write the code that makes it pass. (Don't ask AI to do your work...ask it to finish your work.)

NhanH 4y ago
ChatGPT tends to over mock the module to the point it will always pass the test, so except for utilities module, this isn't helping much either. This is especially common in JS environment. If you think about it, it does make sense why: the prompt tends to have no where near the context of how the module should be delineated. Normally the programmer knows how the abstraction within his program is separated, but we have no good way to tell it to chatGPT. That means unless your program structure is still following one of the common structure and hasn't morphed into your business domain structure, ChatGPT will mock the wrong layer/ abstraction in the test suite.
- jstarfish 4y ago
  I don't know JS, but it works pretty well for me in Python. Where it goes astray, I'll put comments indicating what it's supposed to do and GPT tends to pick up on it. It's like instruction-based inpainting/outpainting. The biggest hurdle is context limit, especially from retained comments. The real benefit to my system is that it weeds out the zero-effort clowns by forcing them to validate their own work before I waste my time trying to do anything with it.
ipaddr 4y ago
Now you have a test that passes but checks something unrelated.
- Vanit 4y ago
  You ask ChatGPT to write that test too (:
  - bheadmaster 4y ago
    Now you have a test that passes its test but the latter checks for something unrelated...
webjunkie 4y ago
This this this. You can ask it to write a test. You can ask it to fix it, if the test does not pass. You can ask it to check whether the code it wrote makes sense, and it will surely find most times that there is something to improve. Urgh, it's not "one prompt and done".

TallGuyShort 4y ago

Curious if you can share any examples of this happening?

jhugo 4y ago
Not OP, but I have also seen this happening across multiple forums, Discord servers and IRC channels, and also in fields outside of tech. Someone wants to look smart, so they get GPT to write a post for them and paste it in as if it was their own response to a question. For example, I'm in some aviation-related discussion groups, and people will often ask technical questions which are best answered from a manual. In the past, eventually someone with access to the relevant manual would post an excerpt, answering the question. Nowadays, commonly someone will paste something from ChatGPT, without disclosing that it's just language model output. It will have the form of a correct response, but with invented details, and a few pages of confusion will result while people try to understand the implications of whatever GPT dreamed up, until someone with access to the real manual comes along, after which the original poster will usually admit that they copy-pasted from ChatGPT. I'm not sure if these people genuinely believe that a language model is somehow a font of knowledge, or if, like GPT, they don't care about whether something is accurate or not, but just care about whether other people believe them.
- skummetmaelk 4y ago
  The missile knows where it is at all times. It knows this because it knows where it isn't. By subtracting where it is from where it isn't, or where it isn't from where it is (whichever is greater), it obtains a difference, or deviation. The guidance subsystem uses deviations to generate corrective commands to drive the missile from a position where it is to a position where it isn't, and arriving at a position where it wasn't, it now is. Consequently, the position where it is, is now the position that it wasn't, and it follows that the position that it was, is now the position that it isn't. In the event that the position that it is in is not the position that it wasn't, the system has acquired a variation, the variation being the difference between where the missile is, and where it wasn't. If variation is considered to be a significant factor, it too may be corrected by the GEA. However, the missile must also know where it was. The missile guidance computer scenario works as follows. Because a variation has modified some of the information the missile has obtained, it is not sure just where it is. However, it is sure where it isn't, within reason, and it knows where it was. It now subtracts where it should be from where it wasn't, or vice-versa, and by differentiating this from the algebraic sum of where it shouldn't be, and where it was, it is able to obtain the deviation and its variation, which is called error.

BlackjackCF 4y ago

What we need is someone to troll respond with ChatGPT generated responses.

AJ007 4y ago
This is a good idea, and probably will be the most likely path for dealing with LLM generated spam in general. Additionally, you could flag a spam contributor and then cloak them in to a parallel hole of nonsense. Of course the Microsoft issue is raised here. Probably this means if Microsoft ignores it, open source projects are going to have to move off of GitHub on to a platform they control and is explicitly designed to deal with spam contributions.

waboremo 4y ago

The internet's noise floor rising was always going to be a given with generative AI, but man to see how low effort, lazy, and haphazard it has been... disappointing. Everyone's been so scared of intentionally malicious groups, we all forgot about the average lazy fraud who wanted to boost his github commit graph, make his subreddit seem alive, or whatever the dumb case may be.

BHSPitMonkey 4y ago
The malicious groups benefit from all of the added noise acting as a smokescreen (in fact, the smart ones probably have a part of their fleet intentionally contributing to it in their victim communities)
wpietri 4y ago
That's been at the top of my list of concerns for quite a while. In Frankfurt's definition, bullshit is "speech intended to persuade without regard for truth". That's exactly what LLMs are good at. With no concept of truth or ability to verify anything, they're just good at plausible words. Which means they're ideal tools for bullshitters of all sorts.
- behringer 4y ago
  Woah it's called a hallucination!

AraceliHarker 4y ago

They can't say "don't use ChatGPT" out loud, since GitHub itself sells Copilot, which is based on GPT.

nico 4y ago

This will always happen when a massive amount of people suddenly get access to a new technology There will be a learning and adaptation period for everyone This is also an opportunity for the maintainers to automate their responses and figure out how to chatgptize themselves (at least in terms of answering previously answered issues/questions) Presumably there will always be a need for a person to answer the novel questions