3 ms·

> GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long div

by remus 3y ago

> GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. I don't think this is a great analogy. if your 737 couldn't drive on the ground and your astrophysicist couldn't answer basic maths questions I wouldn't want to fly in that plane or put much faith in the astrophysicists answers to more complex questions. Maybe maths is not a particular strength of LLMs, but asking questions where it is easy to judge the factual accuracy of the responses seems a pretty reasonable test to be running.

brushfoot 3y ago
> asking questions where it is easy to judge the factual accuracy of the responses seems a pretty reasonable test to be running. It isn't reasonable if that isn't what the system was designed to do. It would be a poor test of my general practitioner's competence to ask him calculus questions and conclude he doesn't know what he's talking about because he can't answer them.
- fragmede 3y ago
  Given that a drug's concentration in your bloodstream is absolutely critical for prescribing them, and considering that this is calculated using Calculus, you'd better hope your GP does actually know their Calculus!
xp84 3y ago
My point was that a 737 can be a land vehicle, but very badly, because it's optimized (to an extreme amount) for flying. I fly on 737s all the time, knowing that have terrible stopping distances and cornering. The 0-60 could be decent, but you can't really accelerate all-out to try it, or you'll overshoot and end up going 175 and crashing. The astrophysicist can do long division in his head, but he'll be about as fast and accurate as the next person, because he doesn't practice arithmetic every day. I agree with the commenter somewhere in my thread who said all an LLM should be optimized for is to classify the type of problem and feed it into a purpose-built, deterministic solver that is trained to interpret math as math and not as language, be it ML-based or algorithmic.

by remus 3y ago

brushfoot 3y ago

> asking questions where it is easy to judge the factual accuracy of the responses seems a pretty reasonable test to be running. It isn't reasonable if that isn't what the system was designed to do. It would be a poor test of my general practitioner's competence to ask him calculus questions and conclude he doesn't know what he's talking about because he can't answer them.

fragmede 3y ago
Given that a drug's concentration in your bloodstream is absolutely critical for prescribing them, and considering that this is calculated using Calculus, you'd better hope your GP does actually know their Calculus!

xp84 3y ago

My point was that a 737 can be a land vehicle, but very badly, because it's optimized (to an extreme amount) for flying. I fly on 737s all the time, knowing that have terrible stopping distances and cornering. The 0-60 could be decent, but you can't really accelerate all-out to try it, or you'll overshoot and end up going 175 and crashing. The astrophysicist can do long division in his head, but he'll be about as fast and accurate as the next person, because he doesn't practice arithmetic every day. I agree with the commenter somewhere in my thread who said all an LLM should be optimized for is to classify the type of problem and feed it into a purpose-built, deterministic solver that is trained to interpret math as math and not as language, be it ML-based or algorithmic.