4 ms·
Beyond being able to solve Connections, can a LLM generate (good/challenging/solvable) connections? Would be pretty cool to be able to generate a test set.
by KaoruAoiShiho 2y ago
Beyond being able to solve Connections, can a LLM generate (good/challenging/solvable) connections? Would be pretty cool to be able to generate a test set.
- jacobsimon 2y agoI’d be surprised if they’re not using AI or some sort of rule-based generator at this point.
- wiether 2y agoBut creating one must be even funnier than trying to solve it, right?
- KaoruAoiShiho 2y agoTook a bit of prompting and a few awful ones, but ultimately it's not bad. https://chatgpt.com/share/67570ab1-b2c0-8006-b5d2-d3fa7132de76 https://chatgpt.com/share/67570ab1-b2c0-8006-b5d2-d3fa7132de... Going to try to feed this into some other LLMs like qwq and see if they can solve them.
- KaoruAoiShiho 2y agoQwQ gets it wrong, Gemini gets it wrong. o1 gets it right, R1 gets a pretty good not originally intended set of 4... tempted to give it partial credit. 4o gets it wrong. Will update with claude once my usage limits are up lol.