7 ms·

...is this supposed to make sense? https://www.dropbox.com/scl/fi/5j0mtwdrqm08dxbh84hh6/Screen-Shot-2023-07-18-at-6.58.56-PM.png?rlkey=7s43dqqvny4n6vjkcx7alzoty

...is this supposed to make sense? https://www.dropbox.com/scl/fi/5j0mtwdrqm08dxbh84hh6/Screen-Shot-2023-07-18-at-6.58.56-PM.png?rlkey=7s43dqqvny4n6vjkcx7alzoty&dl=0 https://www.dropbox.com/scl/fi/5j0mtwdrqm08dxbh84hh6/Screen-...

derefr 3y ago
You're expecting a language model (as opposed to a more general form of ML model) to be numerate? It has no idea that the token "$105,000" has a relationship "is less than" with the token "$140,000". It probably knows that the token "1" has the relationship "is less than" with the token "2" — but that's because it has "1" and "2" as reified concepts, each with many different facts and properties and relationships associated directly with those tokens-as-vertices. "$105,000", meanwhile, is just a lexeme. It maybe knows, due to pre-parsing, that it's "an amount of dollars" — and maybe it even recognizes its order-of-magnitude. It can therefore likely make any statement that takes the token "$105,000" as a meta-syntactic variable standing in for some unknown "amount of dollars." But there's no little numeric model embedded inside the language model that would tell it how many dollars, or be able to compare dollars against dollars.
- sebzim4500 3y ago
  The problem with this kind of simplistic analysis is that if you ask llama which of those numbers is larger it will of course get it right. Clearly there is some understanding of the relative size of numbers, since it can compare numbers that could not possibly have been in the training set.
  - codethief 3y ago
    I think you're both right and, as always, the truth is to be found somewhere in the middle: > Which number is greater, 105,000 or 140,000? > 140,000 is greater than 105,000. > How do you know? > 140,000 is greater than 105,000 because 140,000 is 35,000 more than 105,000. But compare this to: > Which number is greater: 12345 or 112345? > 112345 is greater than 12345. > How do you know? > 112345 has more digits than 12345. > What about 123456 and 134567? > 134567 is greater than 123456. > How do you know? > 134567 has more digits than 123456. > But they have the same number of digits: Six. > 134567 has more significant digits than 123456. > What do you mean by that? > 134567 has more leading zeros than 123456. > But neither has any zero. > 134567 has more digits in total than 123456. > But that's not true. They have the same number of digits. Yet, it is true that 134567 is greater than 123456, as you say. But why? > 134567 is greater than 123456 because it has more digits in total. Another attempt, with smaller context size: > Which number is greater, 123456 or 134567? > 134567 is greater than 123456. > How do you know? > 134567 is greater because 123456 has 6 digits while 134567 has 7 digits. > But 134567 has 6 digits > 134567 has 7 digits.
- theptip 3y ago
  Your incredulity is a bit odd, given that GPT-4 is somewhat numerate, and can compare magnitudes and add/subtract. You’re directionally right I suppose, in that LLMs have a structural disadvantage due to the architecture and don’t always get the correct answer. But you seem to be claiming that a LLM could never do maths, which is trivially false. https://chat.openai.com/share/69e4e673-ba78-412a-a8a7-a1b2f81715ee https://chat.openai.com/share/69e4e673-ba78-412a-a8a7-a1b2f8...
  - blibble 3y ago
    can a calculator do maths if it gets 99% of the answers wrong?
    - cgriswald 3y ago
      I think there are two separate issues here. The first is whether something can be said to be numerate. Is a working calculator numerate? Would an infinite machine with an infinite lookup table be numerate? Are the rules of math learned by modeling language sufficient to be considered numerate or does it require something more? Whether any LLM is numerate probably depends heavily on the specific criteria an individual uses to define numerate. For some that might include the ability to actually reason mathematically (i.e., to understand mathematical rules and apply them). For others, it might only be to return a reasonable answer. The second is usefulness for purpose. Whether something is numerate is effectively irrelevant for usefulness. We don't care how a tool gets its math answers if the answers are correct. A working calculator is useful. A broken one probably isn't (but could be, if, say, all the answers it returned were too low by one). But we don't really care whether a calculator is numerate by whatever definition when we're trying to get an answer. Whether any LLM is useful for calculations probably depends more on how accurate it is and what you are trying to accomplish.
      - derefr 3y ago
        Literacy is a skill separate from knowing any particular language. Literacy is the meta-ability of understanding that you can read language from, and write language to, a visual storage medium. It's a kind of forced development of a new acceleration area of your brain for efficiently transforming visual-field symbols into subvocalization sequences, and vice-versa. If you learn one spoken language, and then you become literate, and then you learn another spoken language, and the two languages share a script, then you now know how to read and write in two languages. I would expect numeracy to be the same: a numerate agent would be one that understands that amounts and relationships can be modelled by numbers. That numbers are abstract concepts that exist separately from the symbols used to represent numbers. That there are an infinite number of them, but without identity, and without canonical representation (2 "is" 5 - 3). That you therefore must assign properties not to individual numbers, but to the sets of numbers that obey certain rules — and so you must recognize what rules a number obeys when you see it. And so forth. If I teach you to do an "increment" operation, or a "less than" comparison, in Arabic numerals; and then I teach you how to represent numbers in Roman or Chinese numerals; then you should now be able to do an increment operation or a less-than comparison using those numerals. Likewise for e.g. base 10 vs base 2 numbers. Your understanding of numbers should not depend on the symbols themselves, but should instead be an understanding embedded in something more like an abstract, non-quantized visual field, where numbers can be above or below or between other numbers in an abstract visual sense; intervals can overlap other intervals in an abstract visual sense; etc. (I would expect a hypothetical "fully" numerate system to be able to "imagine" any algebraic structure described to it, to see the properties it has, and to use that structure to "do math". I shouldn't have to teach arithmetic to the agent all over again just because it's now e.g. modular arithmetic. It should be able to derive — and perform! — all the operations of "modular arithmetic", just because it 1. knows regular arithmetic, and then 2. hears a description of a modular ring.)
- airgapstopgap 3y ago
  > But there's no little numeric model embedded inside the language model that would tell it how many dollars, or be able to compare dollars against dollars This logic applies to any function an LLM may perform, therefore it cannot perform any function, which is absurd. Indeed there functionally are little circuits inside LLMs; its pretraining amounts to formation of those circuits. [1] LLaMA tokenizer parses integers as individual tokens and allows the model to see that "$105,000" is [$][1][0][5][,][0][0][0], which makes it more than capable of learning arithmetic for the general case, and with finetuning even the smallest LLaMA-1 can learn to answer better than GPT-4.[2] It still fails a lot, but so do people, who definitely understand arithmetic on every level from practical to conceptual to algorithmic. The problem is more that it does not have a calculator inside and cannot (by default) use any explicit cache for intermediate values. 1. https://transformer-circuits.pub/2021/framework/index.html https://transformer-circuits.pub/2021/framework/index.html 2. https://arxiv.org/abs/2305.14201 https://arxiv.org/abs/2305.14201
jug 3y ago
LLM's generally don't do math well. GPT-4 has made some strides but I have to wonder if it's using special mechanisms to support it with delegation to special training or modes. I know for a fact Google's Bing has some mechanisms specifically for it too but it's really just crutches that also easily fall apart. These are first and foremost specifically for language... and no, not the "language of math" ;-) If you're interested in doing math on an open model, I'd rather look into integrating LLaMa 2 with Wolfram Alpha. That would be a very nice complement! And there's no reason to see it as admitting defeat. AI and engineering at large is all about using the best tools for the purpose!
- xen2xen1 3y ago
  Google's Bing? I musta missed a big news story..

derefr 3y ago

You're expecting a language model (as opposed to a more general form of ML model) to be numerate? It has no idea that the token "$105,000" has a relationship "is less than" with the token "$140,000". It probably knows that the token "1" has the relationship "is less than" with the token "2" — but that's because it has "1" and "2" as reified concepts, each with many different facts and properties and relationships associated directly with those tokens-as-vertices. "$105,000", meanwhile, is just a lexeme. It maybe knows, due to pre-parsing, that it's "an amount of dollars" — and maybe it even recognizes its order-of-magnitude. It can therefore likely make any statement that takes the token "$105,000" as a meta-syntactic variable standing in for some unknown "amount of dollars." But there's no little numeric model embedded inside the language model that would tell it how many dollars, or be able to compare dollars against dollars.

sebzim4500 3y ago
The problem with this kind of simplistic analysis is that if you ask llama which of those numbers is larger it will of course get it right. Clearly there is some understanding of the relative size of numbers, since it can compare numbers that could not possibly have been in the training set.
- codethief 3y ago
  I think you're both right and, as always, the truth is to be found somewhere in the middle: > Which number is greater, 105,000 or 140,000? > 140,000 is greater than 105,000. > How do you know? > 140,000 is greater than 105,000 because 140,000 is 35,000 more than 105,000. But compare this to: > Which number is greater: 12345 or 112345? > 112345 is greater than 12345. > How do you know? > 112345 has more digits than 12345. > What about 123456 and 134567? > 134567 is greater than 123456. > How do you know? > 134567 has more digits than 123456. > But they have the same number of digits: Six. > 134567 has more significant digits than 123456. > What do you mean by that? > 134567 has more leading zeros than 123456. > But neither has any zero. > 134567 has more digits in total than 123456. > But that's not true. They have the same number of digits. Yet, it is true that 134567 is greater than 123456, as you say. But why? > 134567 is greater than 123456 because it has more digits in total. Another attempt, with smaller context size: > Which number is greater, 123456 or 134567? > 134567 is greater than 123456. > How do you know? > 134567 is greater because 123456 has 6 digits while 134567 has 7 digits. > But 134567 has 6 digits > 134567 has 7 digits.
theptip 3y ago
Your incredulity is a bit odd, given that GPT-4 is somewhat numerate, and can compare magnitudes and add/subtract. You’re directionally right I suppose, in that LLMs have a structural disadvantage due to the architecture and don’t always get the correct answer. But you seem to be claiming that a LLM could never do maths, which is trivially false. https://chat.openai.com/share/69e4e673-ba78-412a-a8a7-a1b2f81715ee https://chat.openai.com/share/69e4e673-ba78-412a-a8a7-a1b2f8...
- blibble 3y ago
  can a calculator do maths if it gets 99% of the answers wrong?
  - cgriswald 3y ago
    I think there are two separate issues here. The first is whether something can be said to be numerate. Is a working calculator numerate? Would an infinite machine with an infinite lookup table be numerate? Are the rules of math learned by modeling language sufficient to be considered numerate or does it require something more? Whether any LLM is numerate probably depends heavily on the specific criteria an individual uses to define numerate. For some that might include the ability to actually reason mathematically (i.e., to understand mathematical rules and apply them). For others, it might only be to return a reasonable answer. The second is usefulness for purpose. Whether something is numerate is effectively irrelevant for usefulness. We don't care how a tool gets its math answers if the answers are correct. A working calculator is useful. A broken one probably isn't (but could be, if, say, all the answers it returned were too low by one). But we don't really care whether a calculator is numerate by whatever definition when we're trying to get an answer. Whether any LLM is useful for calculations probably depends more on how accurate it is and what you are trying to accomplish.
    - derefr 3y ago
      Literacy is a skill separate from knowing any particular language. Literacy is the meta-ability of understanding that you can read language from, and write language to, a visual storage medium. It's a kind of forced development of a new acceleration area of your brain for efficiently transforming visual-field symbols into subvocalization sequences, and vice-versa. If you learn one spoken language, and then you become literate, and then you learn another spoken language, and the two languages share a script, then you now know how to read and write in two languages. I would expect numeracy to be the same: a numerate agent would be one that understands that amounts and relationships can be modelled by numbers. That numbers are abstract concepts that exist separately from the symbols used to represent numbers. That there are an infinite number of them, but without identity, and without canonical representation (2 "is" 5 - 3). That you therefore must assign properties not to individual numbers, but to the sets of numbers that obey certain rules — and so you must recognize what rules a number obeys when you see it. And so forth. If I teach you to do an "increment" operation, or a "less than" comparison, in Arabic numerals; and then I teach you how to represent numbers in Roman or Chinese numerals; then you should now be able to do an increment operation or a less-than comparison using those numerals. Likewise for e.g. base 10 vs base 2 numbers. Your understanding of numbers should not depend on the symbols themselves, but should instead be an understanding embedded in something more like an abstract, non-quantized visual field, where numbers can be above or below or between other numbers in an abstract visual sense; intervals can overlap other intervals in an abstract visual sense; etc. (I would expect a hypothetical "fully" numerate system to be able to "imagine" any algebraic structure described to it, to see the properties it has, and to use that structure to "do math". I shouldn't have to teach arithmetic to the agent all over again just because it's now e.g. modular arithmetic. It should be able to derive — and perform! — all the operations of "modular arithmetic", just because it 1. knows regular arithmetic, and then 2. hears a description of a modular ring.)
airgapstopgap 3y ago
> But there's no little numeric model embedded inside the language model that would tell it how many dollars, or be able to compare dollars against dollars This logic applies to any function an LLM may perform, therefore it cannot perform any function, which is absurd. Indeed there functionally are little circuits inside LLMs; its pretraining amounts to formation of those circuits. [1] LLaMA tokenizer parses integers as individual tokens and allows the model to see that "$105,000" is [$][1][0][5][,][0][0][0], which makes it more than capable of learning arithmetic for the general case, and with finetuning even the smallest LLaMA-1 can learn to answer better than GPT-4.[2] It still fails a lot, but so do people, who definitely understand arithmetic on every level from practical to conceptual to algorithmic. The problem is more that it does not have a calculator inside and cannot (by default) use any explicit cache for intermediate values. 1. https://transformer-circuits.pub/2021/framework/index.html https://transformer-circuits.pub/2021/framework/index.html 2. https://arxiv.org/abs/2305.14201 https://arxiv.org/abs/2305.14201

jug 3y ago

LLM's generally don't do math well. GPT-4 has made some strides but I have to wonder if it's using special mechanisms to support it with delegation to special training or modes. I know for a fact Google's Bing has some mechanisms specifically for it too but it's really just crutches that also easily fall apart. These are first and foremost specifically for language... and no, not the "language of math" ;-) If you're interested in doing math on an open model, I'd rather look into integrating LLaMa 2 with Wolfram Alpha. That would be a very nice complement! And there's no reason to see it as admitting defeat. AI and engineering at large is all about using the best tools for the purpose!

xen2xen1 3y ago
Google's Bing? I musta missed a big news story..