3 ms·
Maybe I need more coffee, but wouldn't it be easier to not add text from certain parts of the internet, or make scrapping bots with controls to avoid those area
by 1659447091 4y ago
Maybe I need more coffee, but wouldn't it be easier to not add text from certain parts of the internet, or make scrapping bots with controls to avoid those areas? I understand it would still grab some unwanted things, but it seems it would be much easier to deal with. Unless of course, this is basically the entirety of the internet and I just refuse to see it as such.
There is a section that mentioned "An online search for the text reveals that it originated from an online erotica site". If you dont want sexually explicit text, then maybe dont include text from literary erotica sites in the first place? Or is it like advertising where companies keep passing around the same chunk of data and continue adding to it? Still, I would think making or paying for better(smarter) scrapping bots would be cheaper.
- falcolas 4y agoI, at this point in time, don't believe any vetting of the training materials was done, because they needed too much of it to make their ML model work. They didn't check for licenses. They didn't check for content. They didn't check for correctness. At this point, it's cheaper for them to manually alter the output than it is to retrain on a more appropriate data set.