4 ms·
It is really, really genuinely concerning how many people think there are profound measurable differences between these things. Like yeah tonally I guess there
by ToucanLoucan 6mo ago
It is really, really genuinely concerning how many people think there are profound measurable differences between these things.
Like yeah tonally I guess there are. But with regard to references and information? You’re literally just using three different slot machines and claiming one is hot.
I suppose though I shouldn’t be that surprised then since Vegas and every other casino on Earth has been built on duping people in that exact way.
- aembleton 6mo ago> You’re literally just using three different slot machines and claiming one is hot. It's a fair point. I haven't tested many queries across them all and checked their answers, but if I want to ask one of them a question - right now its Grok just because I trust its answers more.
- ToucanLoucan 6mo agoIt's not a methodology problem, it's a test-ability problem. LLMs are not deterministic. You can ask the same question to the same LLM five times and you'll likely get at least 3 answers. Again. Slot machine.
- Ukv 6mo agoYou can meaningfully test if one slot machine hits the jackpot more often than another, just that the methodology should involve a large number of repeats rather than a few anecdotes. There are some LLM leaderboard sites that do it with blind comparisons.
- 1234letshaveatw 6mo agoIt sounds like you are claiming that all cars are the same, because cars