This is a good reminder that the output we get from these big transformer language model is a side effect. It's not what they are trained to do per se, it's an

This is a good reminder that the output we get from these big transformer language model is a side effect. It's not what they are trained to do per se, it's an empirical observation that "large language models are few shot learners". There's nothing automatically wrong with that, just that's it's dangerous and irresponsible to put any real trust in this emergent behavior and treat it like you've intentionally build and verified what the model does.