4 ms·
My favorite test is to ask it to invent a magic trick given a set of constraints and props. Because magic is generally published very secretly, surprisingly lit
by Kerrick 1y ago
My favorite test is to ask it to invent a magic trick given a set of constraints and props. Because magic is generally published very secretly, surprisingly little of it is in most training sets. Pretty much just the most common method & gimmick exposures people tend to parrot online, but not the theory or exact routines behind those methods.
The worse an LLM is, the more likely it is to suggest literally impossible actions in the method, like “turn the card over twice to show that it now has three sides. Your spectators can examine the three-sided card.” It can’t tell logic from fantasy, or method from effect.
- jfengel 1y agoI would love to see a trick where you keep turning the same card over with different faces. Bill it as a card trick with just one card. I can think of a half dozen good ways to do it (and you'd need more than one).
- AndyNemmity 1y agoThat's kind of magician thinking though. I'm not sure it's any better than the Elmsley's Dazzle.
- genghisjahn 1y agoThere is a very basic three card trick. You turn each card up one at a time, showing that all three cards are the same(say king spades). End with all three face down in your hand. Then you show all three cards again and all three are a different card(king hearts). At no time are there more or less than 3 cards in your hand.
- AndyNemmity 1y agoI'm a magician. There is a magician AI that is seeded with a ton of tricks that can perform this activity. But it's all context after the fact. There's very little an LLM is going to have about that context as you rightly pointed out.