6 ms·
Very impressive Prompt: > there are 3 black blocks on top of an block that we don't know the color of and beneath them there is a blue block. We remove all bl
by msoad 2y ago
Very impressive
Prompt:
> there are 3 black blocks on top of an block that we don't know the color of and beneath them there is a blue block. We remove all blocks and shuffle the blocks with one additional green block, then put them back on top of each other. the yellow block is on top of blue block. What color is the block we don't know the color of? only answer in one word. the color of block we didn't know the color of initially
ChatGPT 4: Green
Claude3 Opus: Green
GPT2-Chatbot: Yellow
- gnyman 2y agoI don't think this is a good test. If I prefix it with "a riddle" then GPT 4 got it right for me "Yellow" I think the "temperature" (randomness) of a LLM makes it so you'd need to run a lot of these to know if it's actually getting it right or just being lucky and selecting the right color randomly
- sebzim4500 2y agollama-3-70B-instruct and GPT-4 both got it right for me
- hnfong 2y agoIn my runs even Llama3-8B-chat gets it right. A dolphin/mistral fine tune also go it right. Deepseek 67B also.
- sumedh 2y agoI just copy pasted your quote ChatGPT GPT 4 said yellow.