GPT is just a language model. It fills in the blank with a probable continuation. It's kind of amazing how much context the more recent transformer models can r

GPT is just a language model. It fills in the blank with a probable continuation. It's kind of amazing how much context the more recent transformer models can retain. There's definitely something intelligent about it, in the sense that it can parse text and generate consistent sentences that account for context effectively. But, it's not an agent. It doesn't have an objective it's trying to accomplish. The fact that you can kind of chat with it is just a side-effect. In order to prevent GPT from suggesting to people that they should off themselves, it would need to understand that this is inappropriate. Seeing how good recent GPT models are, I think it wouldn't be impossible to train it specifically to understand what could be perceived as unkind, and it might actually do pretty well, you might be surprised, but adding that extra criteria would still fall quite a bit short of having something that behaves like a person with a personality that remains consistent over time and a set of objectives it's trying to accomplish.