I think it's already been made clear that the main reason for the "asymptote" is wrong data input. These models attempt to learn from random internet text ... a

I think it's already been made clear that the main reason for the "asymptote" is wrong data input. These models attempt to learn from random internet text ... and this turns out to not be all that accurate. Also, I've observed a model I was training having the same problem as I do myself. If I at any point learn wrong data, which happens of course, then getting that wrong data back out is very hard and requires 10 or 50 times the effort I spent learning the initial data. In fact I strongly suspect I never unlearn bad data, I just additionally learn "if I say X, it's wrong, say Y instead".