4 ms·
Serious question: Why do you think asking it what it is constitutes a good test of its capabilities? Do you think (for example) someone asking you what you are
by seanhunter 3y ago
Serious question: Why do you think asking it what it is constitutes a good test of its capabilities?
Do you think (for example) someone asking you what you are would give them enough ability to judge your abilities? Yes you would get the answer right and the models you are choosing are getting the answer wrong, but you can't extrapolate from that to anything else at all about what this model can or can't do.
- brookst 3y agoMost people, me included, would probably not give a sufficiently sophisticated answer to “what are you” to satisfy a philosopher who’d spent years examining the question and the body of literature attempting to answer it. It’s a little silly to test a small LLM with a question that at least requires knowledge of its own construction that was not included in its training set, and which really requires existential introspection.