3 ms·
It could also be a result of training data. If every page is weighted equally, you'd expect SEO spam and even autogenerated content to far surpass high quality
by dannyw 6y ago
It could also be a result of training data. If every page is weighted equally, you'd expect SEO spam and even autogenerated content to far surpass high quality content in volume.
I would like to see a GPT model where training data is weighted by credibility / authority (e.g. using Pagerank).
- vladTheInhaler 6y agoMy understanding is that GPT-2 was actually trained on a dataset that was designed to avoid those pitfalls. They followed all the links posted to Reddit that had more than a couple karma, under the theory that the content was at least slightly interesting to some actual humans, as opposed to a giant blob of search keywords or what have you.