3 ms·

After reading the AI generated sections, I have to say that I'm mostly quite impressed. As I lack the context as I am not actively following the ML and procedur

After reading the AI generated sections, I have to say that I'm mostly quite impressed. As I lack the context as I am not actively following the ML and procedurally generated text scenes for years, it was still just markov chains back then, I can't say for sure how accurate the produced text was. It's scary though. Many commenters are only discussion about the business opportunities and path to profitability, but if we step back and stop to think for a while what GPT-3 level tech enables us to do. Robocalls, spam articles and bot generated forum posts are already all too common and while not totally impossible to distinguish, I fear that this level of text generation will only make matters far worse. I'm personally a moderator on a 300k user subreddit which is basically 90% text and very little links, no images or videos, and we are already facing challenges with distinguishing botted marketing campaigns. I fear that in the future it will be even harder to distinguish if you're actually talking with another human being on example a support chat or customer service. The future looks like a Blade Runner esque dystopian landscape of diminishing individuality. I'm doubtful that any attempts at regulating or containing the possible issues would be possible as the barrier for entry has never been lower. Anyone with a decent gaming PC these days can start training their models in mere hours or days. Am I being paranoid? Maybe. Like said, I'm out of the loop and I would definitely love to hear some halming words quenching my fears.

bkanber 6y ago
It'll just be yet another technical advancement that we have to adapt to. And we will. What gives me comfort is that there's still an operator with a motive. It's funny, as I was reading your comment I half expected it to end with "this was generated by GPT-3". And it wouldn't have mattered, because you-the-person still had a sentiment or a message you wanted to communicate, and then you communicated it, whether or not you actually wrote the message. It _would_ be better if support chat understood me better and could communicate with me better. Because again, the motive is understood and aligns with mine. I don't necessarily need a human to do the typing. Typing is a lot of work. In your example, marketing campaigns will continue to be marketing campaigns. I don't worry that the bots will flood your sub more than they already are; if they flood it too much, they destroy any value they hoped to leverage in the first place. The only difference will be that the language in between the spammy links will be more readable. If someone on a, eg, adult chat forum is whispering sweet nothings at you, again I don't care if an AI wrote the prose; someone still decided to whisper sweet nothings at you. They just hired GPT-3 to do the writing instead of doing a sloppy job themselves. I think the real problems start when the AI decides to initiate the action. When an AI, not explicitly instructed by a human operator, decides to launch a marketing campaign. That's the dystopia I'm worried about.
johndough 6y ago
The power that GPT-3 gives to spammers has me worried as well. Anyone with sufficiently many IP addresses can now single-handedly kill any forum by spamming it with text which is at first sight indistinguishable from the average poster. Automatically generated spam could also be used to suppress discussion of political opinions or certain topics by drowning them in a sea of garbage comments, which seems highly problematic. We should come up with a solution to combat this. Here are a few ideas: - To a certain extend, the comment voting mechanism can be used to filter out comments which lead nowhere. If that stops working because comments are too good, that is not a problem. In the words of Randall Munroe: https://xkcd.com/810/ https://xkcd.com/810/ However, voting only works if the number of people voting on a certain comment outnumber the spam comments, so this will fail with too many spam comments. - Another solution would be to verify that every comment author is an actual human being. The GPG web-of-trust could be used for that, but this is untraceable for the average user. There also should be an additional layer of indirection between actual user identities and online identities to preserve privacy, but I am not sure if it is possible to have both privacy and limit a forum user to a single account so they can not circumvent a ban by simply creating new accounts. A trusted third party could solve this, but a distributed solution would of course be preferable. Maybe there is a smart cryptographic solution to this problem? - For the near future, every provider of GPT-3-based services should also provide a service to check whether some given comment has been generated by their model. This can be realized by hashing substrings of all generated output and storing the hashes in a bloom filter. This is not a long term solution since technological advancement will soonishly enable regular people to train similar models. - Training other neural networks to detect automatically generated text is not a solution because the generating networks can simply be trained to not be detected by the detecting networks. This is just a cat and mouse game. Does anyone have a better solution?

bkanber 6y ago

It'll just be yet another technical advancement that we have to adapt to. And we will. What gives me comfort is that there's still an operator with a motive. It's funny, as I was reading your comment I half expected it to end with "this was generated by GPT-3". And it wouldn't have mattered, because you-the-person still had a sentiment or a message you wanted to communicate, and then you communicated it, whether or not you actually wrote the message. It _would_ be better if support chat understood me better and could communicate with me better. Because again, the motive is understood and aligns with mine. I don't necessarily need a human to do the typing. Typing is a lot of work. In your example, marketing campaigns will continue to be marketing campaigns. I don't worry that the bots will flood your sub more than they already are; if they flood it too much, they destroy any value they hoped to leverage in the first place. The only difference will be that the language in between the spammy links will be more readable. If someone on a, eg, adult chat forum is whispering sweet nothings at you, again I don't care if an AI wrote the prose; someone still decided to whisper sweet nothings at you. They just hired GPT-3 to do the writing instead of doing a sloppy job themselves. I think the real problems start when the AI decides to initiate the action. When an AI, not explicitly instructed by a human operator, decides to launch a marketing campaign. That's the dystopia I'm worried about.

johndough 6y ago

The power that GPT-3 gives to spammers has me worried as well. Anyone with sufficiently many IP addresses can now single-handedly kill any forum by spamming it with text which is at first sight indistinguishable from the average poster. Automatically generated spam could also be used to suppress discussion of political opinions or certain topics by drowning them in a sea of garbage comments, which seems highly problematic. We should come up with a solution to combat this. Here are a few ideas: - To a certain extend, the comment voting mechanism can be used to filter out comments which lead nowhere. If that stops working because comments are too good, that is not a problem. In the words of Randall Munroe: https://xkcd.com/810/ https://xkcd.com/810/ However, voting only works if the number of people voting on a certain comment outnumber the spam comments, so this will fail with too many spam comments. - Another solution would be to verify that every comment author is an actual human being. The GPG web-of-trust could be used for that, but this is untraceable for the average user. There also should be an additional layer of indirection between actual user identities and online identities to preserve privacy, but I am not sure if it is possible to have both privacy and limit a forum user to a single account so they can not circumvent a ban by simply creating new accounts. A trusted third party could solve this, but a distributed solution would of course be preferable. Maybe there is a smart cryptographic solution to this problem? - For the near future, every provider of GPT-3-based services should also provide a service to check whether some given comment has been generated by their model. This can be realized by hashing substrings of all generated output and storing the hashes in a bloom filter. This is not a long term solution since technological advancement will soonishly enable regular people to train similar models. - Training other neural networks to detect automatically generated text is not a solution because the generating networks can simply be trained to not be detected by the detecting networks. This is just a cat and mouse game. Does anyone have a better solution?