Because it isn't RLHFed to do so. It's a good example why alignment sometimes isn't a bad thing. ChatGPT does specifically disallow medical advice and generall

Because it isn't RLHFed to do so. It's a good example why alignment sometimes isn't a bad thing. ChatGPT does specifically disallow medical advice and generally it is RLHFed not to provide it: https://openai.com/policies/usage-policies https://openai.com/policies/usage-policies