Consider what would happen if you did ask it to think about a treatment for cancer and got real results. Clearly you would think it is just summarizing papers

Consider what would happen if you did ask it to think about a treatment for cancer and got real results. Clearly you would think it is just summarizing papers it read. That makes sense, since it is not a cancer researcher. So I'm way more impressed by ChatGPT than that. Even if it correctly data-mined and answered the question, it would not be that impressive. That's right: getting a cure from cancer when you ask is less impressive than what it actually does. Because what it actually does is show the ability to judge novel situations, to invent and act creatively, and to keep abstract notions in its head. That is much more impressive than spitting out a cure for cancer.