2 ms·
> Or you just need a model that can recognize math, and then pass it to a system that can do math. Wolfram Alpha already does that. But that's because Wolfram
by pdonis 4y ago
> Or you just need a model that can recognize math, and then pass it to a system that can do math.
Wolfram Alpha already does that. But that's because Wolfram Alpha is built as a model whose purpose is "recognize what kind of problem this natural language query requires, then pass it on to the problem engine for that kind of problem", where each problem engine is an actual solution model for that kind of problem, based on actual facts about the world.
ChatGPT, though, is built as a completely different type of model, whose purpose is "find a pattern that this natural language query matches, then generate a greatest probability sequence of natural language words for that pattern based on the training data set". That's a completely different structure.
- zone411 4y agoIt's possible to enhance mathematical abilities of LLMs by enabling them to externally run symbolic mini programs e.g. https://arxiv.org/abs/2211.12588 https://arxiv.org/abs/2211.12588. It's also possible to create fact-grounded retrieval-enhanced language models e.g. https://proceedings.mlr.press/v162/borgeaud22a.html https://proceedings.mlr.press/v162/borgeaud22a.html.
- MerlinsSister 4y agoSure, but that's kind of the point pdonis was making. You have to change the model by introducing symbolic elements. Neural nets alone wont get you there. Personally I think hybridization is the way to go.