3 ms·
The question is whether an LLM can ever be trained to be good at maths. Currently LLMs can string together tokens that roughly equate to words in order to form
by wizofaus 4y ago
The question is whether an LLM can ever be trained to be good at maths. Currently LLMs can string together tokens that roughly equate to words in order to form sentences that carry particular semantic value. But how much change to the underlying technology would be required to give them the ability to string together digits to represent numbers and then numbers and operators in order to represent equations with particular semantics? I'd suggest it's already able to explain what most common mathematical operations "mean", it just doesn't have the wherewithal to perform them (though it can certainly do a good job of faking it for lower-valued integers that are likely to have occurred in its training data).
- rhn_mk1 4y agoTo some limited extent, they are already good at maths: https://ar5iv.labs.arxiv.org/html/2201.02177 https://ar5iv.labs.arxiv.org/html/2201.02177 This paper makes me believe it's less about the lack of ability to understand maths, but the power to learn more of it.
- wizofaus 4y agoIs that paper discussing LLMs though? That some sort of neural network can be trained to perform maths doesn't seem surprising and I would expect eventually it'll be a standard expectation they can do so even without being fed masses of training data (ideally no more than a human needs). It's the path from ChatGPT-4 to such a network I'm curious about.