5 ms·

Your incredulity is a bit odd, given that GPT-4 is somewhat numerate, and can compare magnitudes and add/subtract. You’re directionally right I suppose, in tha

Your incredulity is a bit odd, given that GPT-4 is somewhat numerate, and can compare magnitudes and add/subtract. You’re directionally right I suppose, in that LLMs have a structural disadvantage due to the architecture and don’t always get the correct answer. But you seem to be claiming that a LLM could never do maths, which is trivially false. https://chat.openai.com/share/69e4e673-ba78-412a-a8a7-a1b2f81715ee https://chat.openai.com/share/69e4e673-ba78-412a-a8a7-a1b2f8...

blibble 3y ago
can a calculator do maths if it gets 99% of the answers wrong?
- cgriswald 3y ago
  I think there are two separate issues here. The first is whether something can be said to be numerate. Is a working calculator numerate? Would an infinite machine with an infinite lookup table be numerate? Are the rules of math learned by modeling language sufficient to be considered numerate or does it require something more? Whether any LLM is numerate probably depends heavily on the specific criteria an individual uses to define numerate. For some that might include the ability to actually reason mathematically (i.e., to understand mathematical rules and apply them). For others, it might only be to return a reasonable answer. The second is usefulness for purpose. Whether something is numerate is effectively irrelevant for usefulness. We don't care how a tool gets its math answers if the answers are correct. A working calculator is useful. A broken one probably isn't (but could be, if, say, all the answers it returned were too low by one). But we don't really care whether a calculator is numerate by whatever definition when we're trying to get an answer. Whether any LLM is useful for calculations probably depends more on how accurate it is and what you are trying to accomplish.
  - derefr 3y ago
    Literacy is a skill separate from knowing any particular language. Literacy is the meta-ability of understanding that you can read language from, and write language to, a visual storage medium. It's a kind of forced development of a new acceleration area of your brain for efficiently transforming visual-field symbols into subvocalization sequences, and vice-versa. If you learn one spoken language, and then you become literate, and then you learn another spoken language, and the two languages share a script, then you now know how to read and write in two languages. I would expect numeracy to be the same: a numerate agent would be one that understands that amounts and relationships can be modelled by numbers. That numbers are abstract concepts that exist separately from the symbols used to represent numbers. That there are an infinite number of them, but without identity, and without canonical representation (2 "is" 5 - 3). That you therefore must assign properties not to individual numbers, but to the sets of numbers that obey certain rules — and so you must recognize what rules a number obeys when you see it. And so forth. If I teach you to do an "increment" operation, or a "less than" comparison, in Arabic numerals; and then I teach you how to represent numbers in Roman or Chinese numerals; then you should now be able to do an increment operation or a less-than comparison using those numerals. Likewise for e.g. base 10 vs base 2 numbers. Your understanding of numbers should not depend on the symbols themselves, but should instead be an understanding embedded in something more like an abstract, non-quantized visual field, where numbers can be above or below or between other numbers in an abstract visual sense; intervals can overlap other intervals in an abstract visual sense; etc. (I would expect a hypothetical "fully" numerate system to be able to "imagine" any algebraic structure described to it, to see the properties it has, and to use that structure to "do math". I shouldn't have to teach arithmetic to the agent all over again just because it's now e.g. modular arithmetic. It should be able to derive — and perform! — all the operations of "modular arithmetic", just because it 1. knows regular arithmetic, and then 2. hears a description of a modular ring.)
  - blibble 3y ago
    > Whether any LLM is numerate probably depends heavily on the specific criteria an individual uses to define numerate. For some that might include the ability to actually reason mathematically (i.e., to understand mathematical rules and apply them) so, no then if it was in the training set maybe you'll get lucky though
derefr 3y ago
GPT-4 is not a pure LLM. It also accepts image inputs. There's other stuff "going on in there" in a GPT model than just linguistic analysis — and those other "facilities" of the model can potentially serve the needs of doing math better than the language parts can. But presuming that wasn't the critical point you wanted to make: Like I said, a language model can know that "1" "is less than" "2" — and it can also know (if it's either trained with characters as lexemes, or is given access to a pre-parse output to second-chance analyze unknown tokens) that "10" is the same thing as (1 tens). Which then means that it can know that "23" "is less than" "48" because it can do linguistic deductive tricks between the terms (2 tens plus 3 ones) and (4 tens plus 8 ones). But those tricks are tricks. It isn't doing math; it's applying "2" as an adjective to "tens", constructing a verb phrase whose verb is "plus", and then (likely) interpreting your question as a question about analogy. It knows that (2 pineapples) "is less than" (3 pineapples) by analogy — (N of some unit) "is analogous to" N-the-number. But it doesn't know that "tens" is a special unit distinct from "pineapples" in that it changes the meaning of the number-token it's attaching to. To put it another way: a (pure) language model has no way of encoding numbers that allows it to actually do math and get correct results out. It can memorize tables of answers for well-known numbers, and it can try to use language tricks to combine those tables, but it can't perform an algorithm on a number, because no part of its architecture allows the nodes in its model to act as a register to encode an (arbitrarily large) number in such a way that it is actually amenable to numeric operations being performed on that data. A model that is really modelling numbers, should be able to apply any arbitrary algorithm it knows about to those numbers, just like a regular CPU can apply any instruction sequence it reads to its registers. Not just add/sub, or mul/div, but arbitrarily-complex things like e.g. iterated modular exponentiation, should just be a matter of saying "hey LLM, you remember the algorithm for doing MOD-EXP, right? So tell me...." (Note that humans can't do this kind of math purely "in our heads" any more than LLMs can, because we don't have any low-level accelerative infrastructure for modelling and working with numeric data either! We need an external buffer that inherently embeds sequencing/positioning info — like our auditory sensory "loop" memory from [sub]verbally repeating the working data; or our visual sensory persistence-of-vision memory, from writing the data down onto a piece of paper and staring at it as we work.)
- theptip 3y ago

blibble 3y ago

can a calculator do maths if it gets 99% of the answers wrong?

cgriswald 3y ago
I think there are two separate issues here. The first is whether something can be said to be numerate. Is a working calculator numerate? Would an infinite machine with an infinite lookup table be numerate? Are the rules of math learned by modeling language sufficient to be considered numerate or does it require something more? Whether any LLM is numerate probably depends heavily on the specific criteria an individual uses to define numerate. For some that might include the ability to actually reason mathematically (i.e., to understand mathematical rules and apply them). For others, it might only be to return a reasonable answer. The second is usefulness for purpose. Whether something is numerate is effectively irrelevant for usefulness. We don't care how a tool gets its math answers if the answers are correct. A working calculator is useful. A broken one probably isn't (but could be, if, say, all the answers it returned were too low by one). But we don't really care whether a calculator is numerate by whatever definition when we're trying to get an answer. Whether any LLM is useful for calculations probably depends more on how accurate it is and what you are trying to accomplish.
- derefr 3y ago
  Literacy is a skill separate from knowing any particular language. Literacy is the meta-ability of understanding that you can read language from, and write language to, a visual storage medium. It's a kind of forced development of a new acceleration area of your brain for efficiently transforming visual-field symbols into subvocalization sequences, and vice-versa. If you learn one spoken language, and then you become literate, and then you learn another spoken language, and the two languages share a script, then you now know how to read and write in two languages. I would expect numeracy to be the same: a numerate agent would be one that understands that amounts and relationships can be modelled by numbers. That numbers are abstract concepts that exist separately from the symbols used to represent numbers. That there are an infinite number of them, but without identity, and without canonical representation (2 "is" 5 - 3). That you therefore must assign properties not to individual numbers, but to the sets of numbers that obey certain rules — and so you must recognize what rules a number obeys when you see it. And so forth. If I teach you to do an "increment" operation, or a "less than" comparison, in Arabic numerals; and then I teach you how to represent numbers in Roman or Chinese numerals; then you should now be able to do an increment operation or a less-than comparison using those numerals. Likewise for e.g. base 10 vs base 2 numbers. Your understanding of numbers should not depend on the symbols themselves, but should instead be an understanding embedded in something more like an abstract, non-quantized visual field, where numbers can be above or below or between other numbers in an abstract visual sense; intervals can overlap other intervals in an abstract visual sense; etc. (I would expect a hypothetical "fully" numerate system to be able to "imagine" any algebraic structure described to it, to see the properties it has, and to use that structure to "do math". I shouldn't have to teach arithmetic to the agent all over again just because it's now e.g. modular arithmetic. It should be able to derive — and perform! — all the operations of "modular arithmetic", just because it 1. knows regular arithmetic, and then 2. hears a description of a modular ring.)
- blibble 3y ago
  > Whether any LLM is numerate probably depends heavily on the specific criteria an individual uses to define numerate. For some that might include the ability to actually reason mathematically (i.e., to understand mathematical rules and apply them) so, no then if it was in the training set maybe you'll get lucky though

derefr 3y ago

GPT-4 is not a pure LLM. It also accepts image inputs. There's other stuff "going on in there" in a GPT model than just linguistic analysis — and those other "facilities" of the model can potentially serve the needs of doing math better than the language parts can. But presuming that wasn't the critical point you wanted to make: Like I said, a language model can know that "1" "is less than" "2" — and it can also know (if it's either trained with characters as lexemes, or is given access to a pre-parse output to second-chance analyze unknown tokens) that "10" is the same thing as (1 tens). Which then means that it can know that "23" "is less than" "48" because it can do linguistic deductive tricks between the terms (2 tens plus 3 ones) and (4 tens plus 8 ones). But those tricks are tricks. It isn't doing math; it's applying "2" as an adjective to "tens", constructing a verb phrase whose verb is "plus", and then (likely) interpreting your question as a question about analogy. It knows that (2 pineapples) "is less than" (3 pineapples) by analogy — (N of some unit) "is analogous to" N-the-number. But it doesn't know that "tens" is a special unit distinct from "pineapples" in that it changes the meaning of the number-token it's attaching to. To put it another way: a (pure) language model has no way of encoding numbers that allows it to actually do math and get correct results out. It can memorize tables of answers for well-known numbers, and it can try to use language tricks to combine those tables, but it can't perform an algorithm on a number, because no part of its architecture allows the nodes in its model to act as a register to encode an (arbitrarily large) number in such a way that it is actually amenable to numeric operations being performed on that data. A model that is really modelling numbers, should be able to apply any arbitrary algorithm it knows about to those numbers, just like a regular CPU can apply any instruction sequence it reads to its registers. Not just add/sub, or mul/div, but arbitrarily-complex things like e.g. iterated modular exponentiation, should just be a matter of saying "hey LLM, you remember the algorithm for doing MOD-EXP, right? So tell me...." (Note that humans can't do this kind of math purely "in our heads" any more than LLMs can, because we don't have any low-level accelerative infrastructure for modelling and working with numeric data either! We need an external buffer that inherently embeds sequencing/positioning info — like our auditory sensory "loop" memory from [sub]verbally repeating the working data; or our visual sensory persistence-of-vision memory, from writing the data down onto a piece of paper and staring at it as we work.)

theptip 3y ago