3 ms·

I was told that GPT-3 was too much power for mere mortals (without a paid subscription!) so what terror is this going to bring upon us?

robbedpeter 5y ago
Transparent and competent automated content moderation, maybe, easily available for anyone to run their own communities by their own standards. Once matured, you can easily envision people sharing policies and templates, or providing moderation as a service, for any sort of social text interaction.
- EamonnMR 5y ago
  I was just thinking about this today. How many (and which?) ML books would one need to read to be able to throw comments into a model like this and have it decide if they're acceptable vs unacceptable? How practical would it be to build a frequently re-training model that could serve as autopilot for a small forum or subreddit moderation? And how much would that serve to reinforce existing filter bubbles and further divide the internet?
  - robbedpeter 5y ago
    0 books. The huggingface hosted gpt-neo 125 model is capable of doing basic question/answer analysis to a sufficient level for most moderation. No finetuning needed if you have it work out yes or no zero shot answers to things like "is this text bullying, excessively negative, political, sexist..." etc, and then iterate over cases and log results. Modify your prompts to handle edge cases and it would probably be capable of replacing 80% of human moderation.
    - EamonnMR 5y ago
      Hmm, I'll still need to scrounge up enough context to understand most of that, but thanks.
      - robbedpeter 5y ago
        https://jalammar.github.io/illustrated-gpt2/ https://jalammar.github.io/illustrated-gpt2/ There are several easily found resources like this, and previous hn threads contain papers, documentation, and guide links spanning the spectrum from beginner to PhD researcher. You can get to a high level of proficiency without any books, but you'll need to learn the jargon for searches. It's a niche enough subject that results don't get too muddied on almost any search engine - good luck!

robbedpeter 5y ago

Transparent and competent automated content moderation, maybe, easily available for anyone to run their own communities by their own standards. Once matured, you can easily envision people sharing policies and templates, or providing moderation as a service, for any sort of social text interaction.

EamonnMR 5y ago
I was just thinking about this today. How many (and which?) ML books would one need to read to be able to throw comments into a model like this and have it decide if they're acceptable vs unacceptable? How practical would it be to build a frequently re-training model that could serve as autopilot for a small forum or subreddit moderation? And how much would that serve to reinforce existing filter bubbles and further divide the internet?
- robbedpeter 5y ago
  0 books. The huggingface hosted gpt-neo 125 model is capable of doing basic question/answer analysis to a sufficient level for most moderation. No finetuning needed if you have it work out yes or no zero shot answers to things like "is this text bullying, excessively negative, political, sexist..." etc, and then iterate over cases and log results. Modify your prompts to handle edge cases and it would probably be capable of replacing 80% of human moderation.
  - EamonnMR 5y ago
    Hmm, I'll still need to scrounge up enough context to understand most of that, but thanks.
    - robbedpeter 5y ago
      https://jalammar.github.io/illustrated-gpt2/ https://jalammar.github.io/illustrated-gpt2/ There are several easily found resources like this, and previous hn threads contain papers, documentation, and guide links spanning the spectrum from beginner to PhD researcher. You can get to a high level of proficiency without any books, but you'll need to learn the jargon for searches. It's a niche enough subject that results don't get too muddied on almost any search engine - good luck!