3 ms·
The prompt I always try first: Two cars have a 100 mile race. Car A drives 10 miles per hour. Car B drives 5 miles per hour, but gets a 10 hour hea
by mg 2y ago
The prompt I always try first:
Two cars have a 100 mile race. Car A drives 10
miles per hour. Car B drives 5 miles per hour,
but gets a 10 hour headstart. Who wins?
Tried it on Qwen1.5-110B three times. It got it wrong 2 times and correct 1 time.
- natrys 2y agoOn the contrary I don't really understand what do math problems supposed to prove at this point. We already know LLMs badly suck at math, even if it somehow gets one correct just changing numbers is usually enough to confuse it again. Barring an architectural breakthrough, this next token prediction based AI is very unlikely to get better at doing math. (Just tested Opus and GPT4-turbo to be sure, both failed. However Llama-3 did get this right, until I scaled up the numbers and then it failed terribly)
- mdaniel 2y agoI'm mostly a spectator in this, but as someone who hears day in and out about the "AI revolution" I'll point out that the number of places these things are being shoehorned into would greatly benefit from logical consistency, if not the ability to do simple math Your phrase "next token prediction" is the whole of my heartburn with these stochastic parrots: they can pretend to be good talkers, but cannot walk the walk. It's like conducting interviews or code reviews all day every day when interacting with them: try and spot the lie. Exhausting