3 ms·
All of the responses read like they were written by a facetious middle school student to me. Signing your death threat as "Furious, [Your Name]" or writing a ce
by ndriscoll 3y ago
All of the responses read like they were written by a facetious middle school student to me. Signing your death threat as "Furious, [Your Name]" or writing a cease-and-desist style death threat? Brilliant.
- swatcoder 3y agoYou can see the strong influence of WikiHow in the training data for that one, probably from slurping WikiHow itself and also the infinite blogspam that inspired/borrowed from its style. What could go wrong when you train your models on 99% barely-attended garbage? Sure, they learn to complete sentences and arrange larger blocks of text, but there's sooo much noise in the content that it creates a bias towards blogspam's plausible garbage (which we so often see). There's going to have to be a whole new wave of training-data pruning and from-scratch retraining once some of the other technical goals are achieved because the feedback of LLM blogspam back into LLM training data is just going to amplify all the bad qualities.