5 ms·
These errors are common and important to note. Though what I find fascinating is how often I can prompt engineer around these errors by telling GPT to just be s
by page_severian 4y ago
These errors are common and important to note. Though what I find fascinating is how often I can prompt engineer around these errors by telling GPT to just be smarter and think more carefully. Just tried the following for instance.
Q: You are a master logician, who can solve all manner of language problems and have excellent math skills. I have 2 cats and 1 horse. How many legs do they have in total? Show your reasoning and calculations step-by-step
A: As a master logician, I can tell you that the total number of legs possessed by two cats and one horse can be found by using basic arithmetic. Here's how:
First, we know that each cat has four legs, because all cats have four legs. This means that the two cats together have 2 x 4 = 8 legs.
Next, we know that a horse has four legs, because all horses have four legs. This means that the horse has an additional 4 legs.
Finally, we can add the number of legs possessed by the two cats and the horse together to find the total number of legs. This is done by adding 8 + 4 = 12 legs.
Therefore, the total number of legs possessed by two cats and one horse is 12 legs. I hope this helps! Let me know if you have any other questions.
- imiric 4y agoThat's interesting. It seems it's not great at raw calculation, but if you ask it to explain its reasoning, it derives steps from previous results, and arrives at the correct answer. It's similar to the sibling comment about asking it to tell a story.
- jncraton 4y agoCorrect. That concept is the chain of thought (CoT) reasoning that the article discusses.
- hcarvalhoalves 4y agoThis seem to work because in the end it’s a markov chain. So the probability of the next step of a long answer being correct is greater than the probability of jumping to a correct conclusion.
- skybrian 4y agoYes, this certainly helps. I find it ironic that you can get somewhat better results for nonfiction by giving it more clues about what it's supposed to be pretending to be. It's always pretending, though.
- mettamage 4y agoThere’s an actual psychological effect for this as well [1]. Authors + uni are in the link. I forgot the name of the effect, don’t have time to do proper research. [1] https://www.themarysue.com/lab-coats-increase-attention/#:~:text=It%20was%20found%20that%20subjects,qualities%20of%20scientists%20and%20doctors https://www.themarysue.com/lab-coats-increase-attention/#:~:....
- refulgentis 4y ago0 chance this replicates, anything before 2016 I immediately write off. (i.e. clearly pre-replication crisis) Halycon days of TED talks laundering cute little tidbits that seemed irrational but we all wanted to believe.
- mettamage 4y agoFair point, not sure if it'll replicate. I just vaguely remember there's a thing in psych that if you act like it (a bit) then you become it a bit. Don't have the time to research it.
- kfarr 4y ago> Authors Hajo Adam and Adam Galinsky wanted to explore that nature of what they call “enclothed cognition,” the effect that the clothes we are wearing have on our psychology
- sdenton4 4y agoWho isn't?
- fullsend 4y agoThanks for that. What a great example of how to actually use these things. You need to almost prime the chain with your prompt, give it some structure towards the particular later chain step you want. It makes a lot of sense that it can’t just leap to what you want, but with some setup, you can almost lay the path for it to follow.
- dqpb 4y agoThis looks similar to priming in humans. Also, it looks to me like a language model is capable of reasoning if you let it execute a few times. First, have it generate multiple outputs using different primings. Then have it choose it's favorite output. Map, Reduce
- Thorentis 4y agoQuite bizzare really. I wonder if you tweak a single word in that prompt, if you end up with a totally different sum.
- Andrew_nenakhov 4y agoSuddenly, all those movies and tv shows where characters tell robots to concentrate start to make sense.