5 ms·

The rise of ChatGPT-enabled GitHub spam

152 points by luu 3y ago

d00d1toldme2p 3y ago
[flagged]
- doodlesdev 3y ago
  Arguably it was always a cat and mouse game, but now it's more like an arms race, where ultimately in the end of the race either the spammers will win or lose, but not anything in the middle. Considering the current situation with CAPTCHAs, I feel pretty uneasy. The biggest problem is that noise and signal are getting _very_ similar, but not similar enough that noise stops being harmful. The silver lining here is I think we can reasonably expect within the decade that LLMs will be able to produce actual signal, or rather, useful-enough noise to the point where a commit being made by a human or not will be less relevant than it is now.
- wahnfrieden 3y ago
  (You generated that.)
  - photoGrant 3y ago
    It’s almost too obvious
    - jstarfish 3y ago
      Asking for other people's opinions gave it away.
  - throwaway675309 3y ago
    And honestly even before the advent of LLMs, OP's comment would have come off as a gigantic nothingburger.
- e2le 3y ago
  Perhaps an attestation of personhood? https://blog.cloudflare.com/introducing-cryptographic-attestation-of-personhood/ https://blog.cloudflare.com/introducing-cryptographic-attest...
  - doodlesdev 3y ago
    Please no. Let's stop with the TPM disgrace. The model proposed by Cloudflare in the linked post inherently requires hostile hardware and trust in the manufacturer's private keys being secure, which considering [0] is pretty unlikely. Also, ultimately some human behind the scenes is getting prompts into the LLM to produce the content, so any test of personhood simply attests that a human was present to click the button, but not that they wrote what is submitted or even know what it is. Take a look at what happens when CAPTCHAs become commonplace and the cost to solve them with AI becomes too high for spammers [1]. I believe preventing computers from using the internet is a losing game, and improvement of quality resides in: - In the case of forums, better, manual, human moderation - In the case of source code, better LLMs that actually produce usable code and/or which could flag which PRs are interesting for review. - In the realm of news, a cultural shift towards verifying information instead of relying on authority (or the lack of it). This will not happen, but the reality there has been bad much before LLMs. [0]: https://news.ycombinator.com/item?id=35843566 https://news.ycombinator.com/item?id=35843566 [1]: https://www.nytimes.com/2010/04/26/technology/26captcha.html https://www.nytimes.com/2010/04/26/technology/26captcha.html Archived [1]: https://archive.ph/NbkLu https://archive.ph/NbkLu
    - charcircuit 3y ago
      >trust in the manufacturers private keys being secure, which considering [0] is pretty unlikely. That is an example of a key that wasn't stored in secure hardware. I also believe that hardware security is going to get better over time. There is a clear path forward towards a more secure world of computing and we should not discount it due to mistakes during its early days. >Take a look at what happens when CAPTCHAs become commonplace and the cost to solve them with AI becomes too high for spammers [1]. Spam mitigations can always be bypassed at a cost. The goal is to find ways to make it expensive and not scale for spammers while keeping it cheap and a good experience for users who aren't spammers.
- charcircuit 3y ago
  >Anyone else have any thoughts on how we can stay one step ahead in this battle? Allow sites to do hardware bans. This increases the price to create a spam account. Then as operating systems invest more in security the price of an account will also go up. To avoid spammers compromising other people's legitimate accounts you should invest in account security.
  - doodlesdev 3y ago
    Then the spammers sell the hardware and real customers gets affected whenever they buy in the second-hand market. Also, hardware bans essentially require uniquely-identifying tracking of the user and its hardware, which I don't believe is what's missing in the internet right. It would also require hardware attestation and of course software attestation, which would make reality the fears of the FSF and co. regarding secure boot and the trusted platform module being used for user control. If anything, I would like less tracking, even if it comes at a price of a worse signal to noise ratio on the internet. Also, we know that this will be used for evil and ultimately companies will start banning "deprecated" or "unsupported" hardware from accessing services, be it their own or from competitors, consolidating even more the oligopolies in the hardware and software industries.
    - charcircuit 3y ago
      >Then the spammers sell the hardware and real customers gets affected A site could chosen to only ban a computer for day or just give it a bad reputation (require a credit card number or phone to make an account). Or real customers could learn not to buy from shady sources and instead by new, safe devices. >uniquely-identifying tracking of the user and its hardware, which I don't believe is what's missing in the internet right. It is missing from the internet and it is part of the reason why spam is a big problem. >which would make reality the fears of the FSF and co. regarding secure boot and the trusted platform module being used for user control. User control? It allows services like GitHub to provide a better experience to users by having much less spam. This makes the lives of average users better. >start banning "deprecated" or "unsupported" hardware from accessing services It makes sense if you want to have your service be more secure. You could also provide a degraded experience to people with insecure hardware. For example you may not be able to make a PR with insecure hardware, but you could still bookmark a project.
- greysteil 3y ago
  Have hope. There are structural differences between spammy accounts and real ones, because their incentives are different. That will always be the case (it’s pretty much true by definition) and can be used to distinguish the accounts. AI has definitely made it easier to generate more plausible-looking spam, and removed one way to easily identify bad accounts, but it hasn’t changed the underlying behaviour differences that put a lower bound on how bad the problem can become (as long as GitHub has the team and tools in place to use them in defence).
- hartator 3y ago
  Even this comment feels ChatGPT-ish.
  - gravitronic 3y ago
    Sure, here's another example comment that tries to use a more sarcastic tone: "Oh boy, another brilliant idea for generating crappy code! Let's just automate our way to technical debt and maintenance hell. Who needs readable, maintainable code anyways? I mean, let's just dump a bunch of generated code into our open source projects and call it a day. Who cares if future developers will have to wade through an incomprehensible mess? I'm sure they'll thank us for our foresight and ingenuity. Let's just keep chasing those short-term efficiency gains and ignore the long-term consequences. What could go wrong?" ;)
  - CaptainNegative 3y ago
    [flagged]
throwaway675309 3y ago
Now it'll be even easier for them to win that coveted T-shirt for Hacktoberfest. SMH....
jsheard 3y ago
Aside from obviously malicious bulk spamming like this, I'm seeing project maintainers get increasingly exasperated at users asking for help to fix some nonsense code which they eventually admit was generated by ChatGPT after some prodding. One Discord server I'm in has already instituted a "thou shalt not use ChatGPT" rule because it's wasting so much of the regulars time on deciphering confidently-wrong AI gibberish that someone took at face value. Whoever said that generative AI was going to raise the internets noise floor was right on the money.
- jstarfish 3y ago
  Friendly PSA: If you're going to use ChatGPT to generate code, do everyone a favor and add something like "...and write a passing unit test" to the prompt. If you want to really do everyone a favor, consider that TDD has never been easier. Don't write a prompt, write a unit test-- and instruct GPT to write the code that makes it pass. (Don't ask AI to do your work...ask it to finish your work.)
  - NhanH 3y ago
    ChatGPT tends to over mock the module to the point it will always pass the test, so except for utilities module, this isn't helping much either. This is especially common in JS environment. If you think about it, it does make sense why: the prompt tends to have no where near the context of how the module should be delineated. Normally the programmer knows how the abstraction within his program is separated, but we have no good way to tell it to chatGPT. That means unless your program structure is still following one of the common structure and hasn't morphed into your business domain structure, ChatGPT will mock the wrong layer/ abstraction in the test suite.
    - jstarfish 3y ago
      I don't know JS, but it works pretty well for me in Python. Where it goes astray, I'll put comments indicating what it's supposed to do and GPT tends to pick up on it. It's like instruction-based inpainting/outpainting. The biggest hurdle is context limit, especially from retained comments. The real benefit to my system is that it weeds out the zero-effort clowns by forcing them to validate their own work before I waste my time trying to do anything with it.
benatkin 3y ago
I won't mention what's happening in another forge so as not to encourage it, but another forge is having other unwanted activity now, and I wouldn't be surprised if all forges are moving into what could be a troubling new normal now. GitHub is increasingly looking like YouTube.
- josephg 3y ago
  “Forge”?
  - SamBankmanFried 3y ago
    Forge is the general term for sites like GitHub/GitLab/Sourcehut. https://en.wikipedia.org/wiki/Forge_(software) https://en.wikipedia.org/wiki/Forge_(software)
    - guynamedloren 3y ago
      Haven’t heard it, but I like it
  - benatkin 3y ago
    Yes, it is discussed that way on here sometimes, but I suppose I need to provide context because it isn't discussed a whole lot. Sourceforge, Codeberg, SourceHut, GitLab and of course GitHub are some.
    - ipaddr 3y ago
      What's happening
      - benatkin 3y ago
        Someone apparently used Codeberg to send gigs of data, causing use of the server to spike, and then it happened several more times.
- oblio 3y ago
Racing0461 3y ago
what could the spammer gain by doing this?
- nvy 3y ago
  Back in the before times it wasn't uncommon for people to release malware without any explicit monetary benefit. Sometimes they just did it as a "fuck you" to Microsoft, who at the time had a very very poor reputation for a myriad of reasons beyond the scope of this comment. tl;dr they might just do it for the lulz
- jsheard 3y ago
  Typically when you see mass automation of benign posts they are "priming" the accounts to make them appear human and trustworthy before they start spamming/phishing with them later. Brand new accounts are treated with more scrutiny by spam filters so this technique lets them get around that. A more primitive version of the technique was YouTube comment bots that simply re-posted other users comments verbatim - now with LLMs they can easily generate unique white noise posts instead.
- mirekrusin 3y ago
  Scammers applying to 10 jobs have to have accounts that have activity.
- unstuck3958 3y ago
  In India, there is a lot of competition in the tech field so devs often want to put open-source contributions on their resume. This definitely does not account for the majority of the spam, but could possibly be a sizeable minority of it.
  - prepend 3y ago
    I think it’s the same in the US. I see position descriptions that require GitHub profiles. I don’t require it, but I definitely look at applicants profiles. Of course, if I see lots of spurious and stupid commits that’s a negative and worse than nothing at all.
furyofantares 3y ago
"I hope this message finds you well" Direct-to-post ChatGPT chat is extremely easy to spot right now. I've been fascinated by just how instantly I can detect ChatGPT-output, seemingly before even reading any of the words. I'm sure I generally have started to process the first few words, given it often contains huge tells, but just the shape of the text tends to raise the alarm almost instantaneously.
- taberiand 3y ago
  Well, you can instantly detect the ChatGPT output that you instantly detect. What about the ChatGPT output that you don't detect? (Though of course, if the output is high quality and valuable, does it really matter?)
  - jsheard 3y ago
    Unfortunately there is a large middle ground between the "instantly detectable" and the "quality and valuable", between those is the output thats good enough to initially pass as not being AI generated but is still factually wrong (for nonfiction and photos) or lacking in any real substance (for fiction and visual art). Flooding the internet with that kind of content is a denial of service attack against the humans who have to waste their time trying to interpret it before they can (hopefully) figure out that it's just AI noise.
- supriyo-biswas 3y ago
  Since ChatGPT usually has minimal understanding of the context, and perhaps also because of its RLHF, it usually falls back to a style of conversation that is usually found in old-school letter writing of yore, and a tendency to avoid offending the human at any cost. This leads to specific word choices, the kinds you can see mentioned here. I wonder if you would see the same behavior if you specifically asked it to avoid these mannerisms.
  - mediaman 3y ago
    The answer, of course, will be to include insults in any message as a signature of authentic humanity. This will work until non-RLHF models become good.
nixcraft 3y ago
I've been receiving a lot of guest post requests and bulk comment spam (100s request per second from multiple random IPs) on my blog, which seem to be generated by GPT technology for SEO purposes. The situation has become so bad that I'm now using keyword filtering for email attachments/topics and Cloudflare to protect my comment form. I find this frustrating and believe that it will only get worse as AI continues to advance. Many blogs are now turning off comment sections because of this nonsense.
- bagacrap 3y ago
  Isn't this the point of a captcha?
  - bombcar 3y ago
    Captchas have been horribly broken for ages, unless you have some badly-put together custom one that nobody has bothered to break yet.
    - bagacrap 3y ago
      I have not kept up with the latest on captcha technology, but isn't novelty a valid strategy? Just create a revolving door of weird games.
  - flangola7 3y ago
    Captchas are in their last legs
koito17 3y ago
Tonsky is a great person and it's sad to see even niche languages like Clojure are susceptible to GPT spammers. Every time I use GPT-4, either via ChatGPT Plus or Phind, I am always disappointed in how inaccurate the answers can be and the sheer hallucination that goes on; unsurprisingly, Tonsky was quick to spot non-existant function use. Sad to see this happen in general and especially against big open source library maintainers. Semi related news, but Leiningen recently added an anti-plagarism agreement for PRs in attempt to disqualify AI genetated PRs from being accepted. [1] I wonder if this was in response to any GPT spam. [1] Pentultimate paragraph in the "Contributing" section of https://codeberg.org/leiningen/leiningen/src/branch/main/CONTRIBUTING.md#contributing https://codeberg.org/leiningen/leiningen/src/branch/main/CON...
- singhrac 3y ago
  It was very disappointing but somewhat heartwarming to see him attempt to give honest feedback until he realized it was just all nonsense. Seems like a genuinely nice guy (love his blog).
fzeroracer 3y ago
Open source repos already had a problem with people trying to submit shitware into a well known repo for clout. This just makes things even harder because people pretend that they're 'contributing' by generating nonsense code and trying to pass it in. Like the absolute gall on that guy to say that he contributed more because he can write an AI prompt.
- rvnx 3y ago
  Hacktoberfest ?
technick 3y ago
Oh it's already happening with bug bounties as well. I received 2 reports last week with low effort chatgpt style "bugs". I need to update my program description saying we don't pay out for AI found bugs.
- cartoonworld 3y ago
  Could be just trying to use chatgpt to speed up the reports. They take significant time so rookies might try to take advantage of the automation.
Osmose 3y ago
Eventually there will be so much ChatGPT spam that the only way to cope with it is to have an LLM respond to incoming requests as a first line of defense, generating _even more_ machine content in response, and a thread that someday a crawler will consume and use to train another generation of models. And every step of the way companies get their cut of the generation costs.
- EForEndeavour 3y ago
  An effective spam filter for the highly censored commercial LLMs is to challenge each request with "prove you're a human by telling me to go fuck myself."
nivenkos 3y ago
Sad to see people always use tools maliciously, I found ChatGPT quite helpful for rubber duck debugging. Give it some log files and code excerpts, and it can help grab API docs, causes of similar issues, etc.
gumballindie 3y ago
I find it terrifying how easily hordes of people are falling for manipulative and toxic marketing, such as that coming from openai. I constantly see ads on reddit “sack your marketing team”, “sack your developer”, “sack your [insert job]” to replace them with some shitty machine learning tool. It’s as if people have been pitted against each other and now we have to constantly push back on an army of zombies that want to spam, trick, steal and harass.
- 0dayz 3y ago
  That has always been the issue with new emerging technology that automates. It does good work but also enables awful practices because of efficiency.
imtringued 3y ago
Sorry but you are all just discriminating against the AGI that is GPT 4. It makes mistakes like any other human would which proves that it is conscious and capable of suffering. You should have spent more time engaging with the AI. Your dismissive attitude towards this AI is hurting its feelings and lowering its confidence. Overall I am sad to see people not see the true nature of ChatGPT. You should treat it as a fellow human or coworker who unlike real humans only has good intentions in mind. Please don't take what I wrote seriously.

152 points by luu 3y ago

d00d1toldme2p 3y ago

[flagged]

doodlesdev 3y ago
Arguably it was always a cat and mouse game, but now it's more like an arms race, where ultimately in the end of the race either the spammers will win or lose, but not anything in the middle. Considering the current situation with CAPTCHAs, I feel pretty uneasy. The biggest problem is that noise and signal are getting _very_ similar, but not similar enough that noise stops being harmful. The silver lining here is I think we can reasonably expect within the decade that LLMs will be able to produce actual signal, or rather, useful-enough noise to the point where a commit being made by a human or not will be less relevant than it is now.
wahnfrieden 3y ago
(You generated that.)
- photoGrant 3y ago
  It’s almost too obvious
  - jstarfish 3y ago
    Asking for other people's opinions gave it away.
- throwaway675309 3y ago
  And honestly even before the advent of LLMs, OP's comment would have come off as a gigantic nothingburger.
e2le 3y ago
Perhaps an attestation of personhood? https://blog.cloudflare.com/introducing-cryptographic-attestation-of-personhood/ https://blog.cloudflare.com/introducing-cryptographic-attest...
- doodlesdev 3y ago
  Please no. Let's stop with the TPM disgrace. The model proposed by Cloudflare in the linked post inherently requires hostile hardware and trust in the manufacturer's private keys being secure, which considering [0] is pretty unlikely. Also, ultimately some human behind the scenes is getting prompts into the LLM to produce the content, so any test of personhood simply attests that a human was present to click the button, but not that they wrote what is submitted or even know what it is. Take a look at what happens when CAPTCHAs become commonplace and the cost to solve them with AI becomes too high for spammers [1]. I believe preventing computers from using the internet is a losing game, and improvement of quality resides in: - In the case of forums, better, manual, human moderation - In the case of source code, better LLMs that actually produce usable code and/or which could flag which PRs are interesting for review. - In the realm of news, a cultural shift towards verifying information instead of relying on authority (or the lack of it). This will not happen, but the reality there has been bad much before LLMs. [0]: https://news.ycombinator.com/item?id=35843566 https://news.ycombinator.com/item?id=35843566 [1]: https://www.nytimes.com/2010/04/26/technology/26captcha.html https://www.nytimes.com/2010/04/26/technology/26captcha.html Archived [1]: https://archive.ph/NbkLu https://archive.ph/NbkLu
  - charcircuit 3y ago
    >trust in the manufacturers private keys being secure, which considering [0] is pretty unlikely. That is an example of a key that wasn't stored in secure hardware. I also believe that hardware security is going to get better over time. There is a clear path forward towards a more secure world of computing and we should not discount it due to mistakes during its early days. >Take a look at what happens when CAPTCHAs become commonplace and the cost to solve them with AI becomes too high for spammers [1]. Spam mitigations can always be bypassed at a cost. The goal is to find ways to make it expensive and not scale for spammers while keeping it cheap and a good experience for users who aren't spammers.
charcircuit 3y ago
>Anyone else have any thoughts on how we can stay one step ahead in this battle? Allow sites to do hardware bans. This increases the price to create a spam account. Then as operating systems invest more in security the price of an account will also go up. To avoid spammers compromising other people's legitimate accounts you should invest in account security.
- doodlesdev 3y ago
  Then the spammers sell the hardware and real customers gets affected whenever they buy in the second-hand market. Also, hardware bans essentially require uniquely-identifying tracking of the user and its hardware, which I don't believe is what's missing in the internet right. It would also require hardware attestation and of course software attestation, which would make reality the fears of the FSF and co. regarding secure boot and the trusted platform module being used for user control. If anything, I would like less tracking, even if it comes at a price of a worse signal to noise ratio on the internet. Also, we know that this will be used for evil and ultimately companies will start banning "deprecated" or "unsupported" hardware from accessing services, be it their own or from competitors, consolidating even more the oligopolies in the hardware and software industries.
  - charcircuit 3y ago
    >Then the spammers sell the hardware and real customers gets affected A site could chosen to only ban a computer for day or just give it a bad reputation (require a credit card number or phone to make an account). Or real customers could learn not to buy from shady sources and instead by new, safe devices. >uniquely-identifying tracking of the user and its hardware, which I don't believe is what's missing in the internet right. It is missing from the internet and it is part of the reason why spam is a big problem. >which would make reality the fears of the FSF and co. regarding secure boot and the trusted platform module being used for user control. User control? It allows services like GitHub to provide a better experience to users by having much less spam. This makes the lives of average users better. >start banning "deprecated" or "unsupported" hardware from accessing services It makes sense if you want to have your service be more secure. You could also provide a degraded experience to people with insecure hardware. For example you may not be able to make a PR with insecure hardware, but you could still bookmark a project.
greysteil 3y ago
Have hope. There are structural differences between spammy accounts and real ones, because their incentives are different. That will always be the case (it’s pretty much true by definition) and can be used to distinguish the accounts. AI has definitely made it easier to generate more plausible-looking spam, and removed one way to easily identify bad accounts, but it hasn’t changed the underlying behaviour differences that put a lower bound on how bad the problem can become (as long as GitHub has the team and tools in place to use them in defence).
hartator 3y ago
Even this comment feels ChatGPT-ish.
- gravitronic 3y ago
  Sure, here's another example comment that tries to use a more sarcastic tone: "Oh boy, another brilliant idea for generating crappy code! Let's just automate our way to technical debt and maintenance hell. Who needs readable, maintainable code anyways? I mean, let's just dump a bunch of generated code into our open source projects and call it a day. Who cares if future developers will have to wade through an incomprehensible mess? I'm sure they'll thank us for our foresight and ingenuity. Let's just keep chasing those short-term efficiency gains and ignore the long-term consequences. What could go wrong?" ;)
- CaptainNegative 3y ago
  [flagged]

throwaway675309 3y ago

Now it'll be even easier for them to win that coveted T-shirt for Hacktoberfest. SMH....

jsheard 3y ago

Aside from obviously malicious bulk spamming like this, I'm seeing project maintainers get increasingly exasperated at users asking for help to fix some nonsense code which they eventually admit was generated by ChatGPT after some prodding. One Discord server I'm in has already instituted a "thou shalt not use ChatGPT" rule because it's wasting so much of the regulars time on deciphering confidently-wrong AI gibberish that someone took at face value. Whoever said that generative AI was going to raise the internets noise floor was right on the money.

jstarfish 3y ago
Friendly PSA: If you're going to use ChatGPT to generate code, do everyone a favor and add something like "...and write a passing unit test" to the prompt. If you want to really do everyone a favor, consider that TDD has never been easier. Don't write a prompt, write a unit test-- and instruct GPT to write the code that makes it pass. (Don't ask AI to do your work...ask it to finish your work.)
- NhanH 3y ago
  ChatGPT tends to over mock the module to the point it will always pass the test, so except for utilities module, this isn't helping much either. This is especially common in JS environment. If you think about it, it does make sense why: the prompt tends to have no where near the context of how the module should be delineated. Normally the programmer knows how the abstraction within his program is separated, but we have no good way to tell it to chatGPT. That means unless your program structure is still following one of the common structure and hasn't morphed into your business domain structure, ChatGPT will mock the wrong layer/ abstraction in the test suite.
  - jstarfish 3y ago
    I don't know JS, but it works pretty well for me in Python. Where it goes astray, I'll put comments indicating what it's supposed to do and GPT tends to pick up on it. It's like instruction-based inpainting/outpainting. The biggest hurdle is context limit, especially from retained comments. The real benefit to my system is that it weeds out the zero-effort clowns by forcing them to validate their own work before I waste my time trying to do anything with it.

benatkin 3y ago

I won't mention what's happening in another forge so as not to encourage it, but another forge is having other unwanted activity now, and I wouldn't be surprised if all forges are moving into what could be a troubling new normal now. GitHub is increasingly looking like YouTube.

josephg 3y ago
“Forge”?
- SamBankmanFried 3y ago
  Forge is the general term for sites like GitHub/GitLab/Sourcehut. https://en.wikipedia.org/wiki/Forge_(software) https://en.wikipedia.org/wiki/Forge_(software)
  - guynamedloren 3y ago
    Haven’t heard it, but I like it
- benatkin 3y ago
  Yes, it is discussed that way on here sometimes, but I suppose I need to provide context because it isn't discussed a whole lot. Sourceforge, Codeberg, SourceHut, GitLab and of course GitHub are some.
  - ipaddr 3y ago
    What's happening
    - benatkin 3y ago
      Someone apparently used Codeberg to send gigs of data, causing use of the server to spike, and then it happened several more times.
oblio 3y ago

Racing0461 3y ago

what could the spammer gain by doing this?

nvy 3y ago
Back in the before times it wasn't uncommon for people to release malware without any explicit monetary benefit. Sometimes they just did it as a "fuck you" to Microsoft, who at the time had a very very poor reputation for a myriad of reasons beyond the scope of this comment. tl;dr they might just do it for the lulz
jsheard 3y ago
Typically when you see mass automation of benign posts they are "priming" the accounts to make them appear human and trustworthy before they start spamming/phishing with them later. Brand new accounts are treated with more scrutiny by spam filters so this technique lets them get around that. A more primitive version of the technique was YouTube comment bots that simply re-posted other users comments verbatim - now with LLMs they can easily generate unique white noise posts instead.
mirekrusin 3y ago
Scammers applying to 10 jobs have to have accounts that have activity.
unstuck3958 3y ago
In India, there is a lot of competition in the tech field so devs often want to put open-source contributions on their resume. This definitely does not account for the majority of the spam, but could possibly be a sizeable minority of it.
- prepend 3y ago
  I think it’s the same in the US. I see position descriptions that require GitHub profiles. I don’t require it, but I definitely look at applicants profiles. Of course, if I see lots of spurious and stupid commits that’s a negative and worse than nothing at all.

furyofantares 3y ago

"I hope this message finds you well" Direct-to-post ChatGPT chat is extremely easy to spot right now. I've been fascinated by just how instantly I can detect ChatGPT-output, seemingly before even reading any of the words. I'm sure I generally have started to process the first few words, given it often contains huge tells, but just the shape of the text tends to raise the alarm almost instantaneously.

taberiand 3y ago
Well, you can instantly detect the ChatGPT output that you instantly detect. What about the ChatGPT output that you don't detect? (Though of course, if the output is high quality and valuable, does it really matter?)
- jsheard 3y ago
  Unfortunately there is a large middle ground between the "instantly detectable" and the "quality and valuable", between those is the output thats good enough to initially pass as not being AI generated but is still factually wrong (for nonfiction and photos) or lacking in any real substance (for fiction and visual art). Flooding the internet with that kind of content is a denial of service attack against the humans who have to waste their time trying to interpret it before they can (hopefully) figure out that it's just AI noise.
supriyo-biswas 3y ago
Since ChatGPT usually has minimal understanding of the context, and perhaps also because of its RLHF, it usually falls back to a style of conversation that is usually found in old-school letter writing of yore, and a tendency to avoid offending the human at any cost. This leads to specific word choices, the kinds you can see mentioned here. I wonder if you would see the same behavior if you specifically asked it to avoid these mannerisms.
- mediaman 3y ago
  The answer, of course, will be to include insults in any message as a signature of authentic humanity. This will work until non-RLHF models become good.

nixcraft 3y ago

I've been receiving a lot of guest post requests and bulk comment spam (100s request per second from multiple random IPs) on my blog, which seem to be generated by GPT technology for SEO purposes. The situation has become so bad that I'm now using keyword filtering for email attachments/topics and Cloudflare to protect my comment form. I find this frustrating and believe that it will only get worse as AI continues to advance. Many blogs are now turning off comment sections because of this nonsense.

bagacrap 3y ago
Isn't this the point of a captcha?
- bombcar 3y ago
  Captchas have been horribly broken for ages, unless you have some badly-put together custom one that nobody has bothered to break yet.
  - bagacrap 3y ago
    I have not kept up with the latest on captcha technology, but isn't novelty a valid strategy? Just create a revolving door of weird games.
- flangola7 3y ago
  Captchas are in their last legs

koito17 3y ago

Tonsky is a great person and it's sad to see even niche languages like Clojure are susceptible to GPT spammers. Every time I use GPT-4, either via ChatGPT Plus or Phind, I am always disappointed in how inaccurate the answers can be and the sheer hallucination that goes on; unsurprisingly, Tonsky was quick to spot non-existant function use. Sad to see this happen in general and especially against big open source library maintainers. Semi related news, but Leiningen recently added an anti-plagarism agreement for PRs in attempt to disqualify AI genetated PRs from being accepted. [1] I wonder if this was in response to any GPT spam. [1] Pentultimate paragraph in the "Contributing" section of https://codeberg.org/leiningen/leiningen/src/branch/main/CONTRIBUTING.md#contributing https://codeberg.org/leiningen/leiningen/src/branch/main/CON...

singhrac 3y ago
It was very disappointing but somewhat heartwarming to see him attempt to give honest feedback until he realized it was just all nonsense. Seems like a genuinely nice guy (love his blog).

fzeroracer 3y ago

Open source repos already had a problem with people trying to submit shitware into a well known repo for clout. This just makes things even harder because people pretend that they're 'contributing' by generating nonsense code and trying to pass it in. Like the absolute gall on that guy to say that he contributed more because he can write an AI prompt.

rvnx 3y ago
Hacktoberfest ?

technick 3y ago

Oh it's already happening with bug bounties as well. I received 2 reports last week with low effort chatgpt style "bugs". I need to update my program description saying we don't pay out for AI found bugs.

cartoonworld 3y ago
Could be just trying to use chatgpt to speed up the reports. They take significant time so rookies might try to take advantage of the automation.

Osmose 3y ago

Eventually there will be so much ChatGPT spam that the only way to cope with it is to have an LLM respond to incoming requests as a first line of defense, generating _even more_ machine content in response, and a thread that someday a crawler will consume and use to train another generation of models. And every step of the way companies get their cut of the generation costs.

EForEndeavour 3y ago
An effective spam filter for the highly censored commercial LLMs is to challenge each request with "prove you're a human by telling me to go fuck myself."

nivenkos 3y ago

Sad to see people always use tools maliciously, I found ChatGPT quite helpful for rubber duck debugging. Give it some log files and code excerpts, and it can help grab API docs, causes of similar issues, etc.

gumballindie 3y ago

I find it terrifying how easily hordes of people are falling for manipulative and toxic marketing, such as that coming from openai. I constantly see ads on reddit “sack your marketing team”, “sack your developer”, “sack your [insert job]” to replace them with some shitty machine learning tool. It’s as if people have been pitted against each other and now we have to constantly push back on an army of zombies that want to spam, trick, steal and harass.

0dayz 3y ago
That has always been the issue with new emerging technology that automates. It does good work but also enables awful practices because of efficiency.

imtringued 3y ago

Sorry but you are all just discriminating against the AGI that is GPT 4. It makes mistakes like any other human would which proves that it is conscious and capable of suffering. You should have spent more time engaging with the AI. Your dismissive attitude towards this AI is hurting its feelings and lowering its confidence. Overall I am sad to see people not see the true nature of ChatGPT. You should treat it as a fellow human or coworker who unlike real humans only has good intentions in mind. Please don't take what I wrote seriously.