5 ms·
But I don't want tricks. I want to know that it knows so I don't have to continually guess whether it's right or not.
by nprateem 2y ago
But I don't want tricks. I want to know that it knows so I don't have to continually guess whether it's right or not.
- naasking 2y agoThat's simply not possible. Human understanding is still unreliable, even for geniuses.
- lapitopi 2y agoThat’s why I am asking a computer.
- catapart 2y agoI'm with you. I get that this is akin to asking a human, because we're trying to reason, so we will bring along (assumedly) unavoidable deficiencies of human reasoning. But if I were to ask a human genius this question, ne would grab a calculator and employ it as ne did the rest of ner reasoning. So it seems like we should probably teach LLMs to "use a calculator", rather than try to get them to be more right when doing math 'in their head'.
- naasking 2y agoIndeed, "use a calculator" is "just a trick"!
- deleted 2y ago[deleted]
- throwaway4aday 2y agoSolving that will be a much bigger deal but it's at odds with producing a highly accurate emulation of human thought and language. Language models can serve as tools to understand and experiment with logic formulated as natural language but it isn't their primary purpose. What you're asking is equivalent to creating an auditable trace of everything that goes into making a statement which is pretty much impossible even for the person making a statement. We can get close by limiting ourselves to narrow domains like mathematics but even then someone can come along and question the premises on which we construct such a system. I'm not saying it isn't worth pursuing, it just isn't the standard that we should hold a model to when we ourselves are incapable of it. The goal here is to create a system capable of doing the things that a human can do. If you prefer to have a system that behaves within the confines of a mathematical formalism with well defined rules then build that model instead.
- nprateem 2y agoMy calculator manages
- shepherdjerred 2y agoYour calculator is deterministic. Humans and AI are not.
- nprateem 2y agoExactly
- vidarh 2y agoThe point is don't ask an LLM to do tasks that a calculator can do. Ask if to use the calculator, just like most humans would.
- nprateem 2y agoI know. That was my point.
- chowells 2y agoThe point is that you shouldn't need to ask. If it's actually a general-purpose system it will do it automatically.
- vidarh 2y agoAn LLM in isolation is not a general purpose system, but with ChatGPT at least, most of the time you don't need to ask. In fact, it's increasingly difficult to force it to do "manual" maths, as it's strongly predisposed to do things like write and evaluate Python code instead of doing it "manually". E.g. I verified just now and asked it to multiply two huge numbers, and it immediately spat out Python code, then evaluated it and gave me the result, rather than try to add the numbers itself.
- 2y ago
- krapp 2y agoIt's entirely possible. Don't use LLMs for math. Use the computers we already have that have been capable of doing math accurately for a century. Right tool, right job.
- naasking 2y agoOP said they didn't want tricks from their LLM. Using a calculator, like we do, is technically a trick.
- jhanschoo 2y agoA basic transformer architecture performs only a bounded amount of computation per generated token, so it can never emulate a machine computing sufficiently hard problems.
- EVa5I7bHFq9mnYK 2y agoYes, because it's feed forward. It must have loops to be a Turing machine.