5 ms·
A straightforward point that comes to mind regards reusing the output of GPT-n as part of the training input to GPT-n+1: Andrej Karpathy recently tweeted about
by rwnspace 6y ago
A straightforward point that comes to mind regards reusing the output of GPT-n as part of the training input to GPT-n+1: Andrej Karpathy recently tweeted about it in terms of pollution, which has wonderful connotations: https://twitter.com/karpathy/status/1284660899198820352 https://twitter.com/karpathy/status/1284660899198820352
- Don_Patrick 6y agoSo far most of this pollution is effectively tagged through accompanying mentions of "GPT", but filtering that from future training data would mean GPT can never learn about itself.