4 ms·
Most humans fail this, in my experience. I'm not sure a trick question is going to be the litmus test for AGI.
by qup 3y ago
Most humans fail this, in my experience.
I'm not sure a trick question is going to be the litmus test for AGI.
- mensetmanusman 3y agonot 'super intelligent' humans who are familiar with this old dad joke :)
- jaggederest 3y agoThis is a fun one because it illustrates that you failed the test just like I did. Reread carefully: the question compares "a kilogram of feathers or a kilogram of feathers". I've never heard that one before but my brain pattern matched right over to "a kilogram of feathers or a kilogram of steel" and moved on. I had to read it 3 times before I got the subtlety.
- parineum 3y agoIt's a trick trick question. It's meant to appear as the often used trick question comparing different densities of materials but OP says feathers twice. The trick is that an LLM will add in a dense material to the answer because it's only ever seen the question asked that way.
- digbybk 3y agoI wouldn’t even say it’s failing. Recognizing and correcting obvious errors in speech is a feature not a bug, both in humans and LLMs. Is it more likely that they meant to say feather twice or they mistyped?
- transcriptase 3y agoTrue, but would it not make more sense to ask rather than assume and provide a long winded explanation? And even if it was a typo and I meant to say rocks, the model will often assume a variety of metals and answer with those.
- qup 3y agoHow it proceeds in the case of human error is going to be almost totally removed from its "intelligence," and almost totally related to its training.
- digbybk 3y agoI’m not claiming otherwise here. I’m claiming that it’s not a good test because a human would (almost certainly correctly) also skip past the double feather and answer the likely spirit of the question.
- throwuwu 3y agoA human would probably say “did you mean a kg of feathers and a kg of lead?”