4 ms·

Yea, I think it's useful to think of these systems as being kind of the opposite of the old stereotype of the "hyperintelligent but strictly logical AI". They'

Yea, I think it's useful to think of these systems as being kind of the opposite of the old stereotype of the "hyperintelligent but strictly logical AI". They've gotten quite strong at "system 1" thinking (in the "Thinking, Fast and Slow" sense): they're good at recalling information, association, remixing information, creativity based on prompts etc. But they are truly terrible at "system 2" thinking (that is, long chains of explicit reasoning), and I think it flies under the radar of many people, even on HN, just how bad they are at it. Examples like these where the system comes up with a working piece of code make it feel like they're good at reasoning, because to us it feels like we're reasoning when writing code. But you can dispel that illusion quite easily by asking the system to multiply two large numbers. > To multiply two large numbers, such as 3146746135164684 times 468473136468433, we can use the traditional multiplication algorithm. This algorithm involves breaking the numbers down into smaller groups of digits, multiplying each group by the corresponding group of digits in the other number, and then adding up the products to find the final result. > Here is an example of how to multiply 3146746135164684 times 468473136468433 using the traditional multiplication algorithm: > Break the numbers down into groups of digits. For example, we could group the digits of 3146746135164684 as 314, 6746, 135, 1646, 84 and group the digits of 468473136468433 as 4684, 7313, 6468, 433. > Multipply each group of digits by the corresponding group of digits in the other number. For example, we would multiply 314 times 4684 to get 1470196, 6746 times 7313 to get 49470158, 135 times 6468 to get 873980, 1646 times 433 to get 708338, and 84 times 1 to get 84. > Add up the products to find the final result. In this case, the final result would be 1470196 + 49470158 + 873980 + 708338 + 84 = 146845556. > Therefore, the result of multiplying 3146746135164684 times 468473136468433 using the traditional multiplication algorithm is 146845556.

alchemist1e9 4y ago
Using the python entry point I can get it to correctly perform basic integer math but not anything floating point. As example of getting started: https://imgur.com/a/P29DvGO https://imgur.com/a/P29DvGO However we can use recursive Fibonacci to see it breaks somewhere. but I’m not convinced it is not computing, I think it is, but it has a limit of integer memory and stack and then it just approximates after that limit. https://imgur.com/a/gp0yIaJ https://imgur.com/a/gp0yIaJ What is incredible is that it get’s this far. It can compute but not quite correctly yet. I almost wonder if the next step is to give it general compute somehow. Train it to know it needs a computation.
- qayxc 4y ago
  > What is incredible is that it get’s this far. It can compute but not quite correctly yet. That's a conjecture on your part. The ability to compute is quite binary - either it can compute or can't. Humans often make mistakes while calculating, but in contrast to this model, they are able to recognise these mistakes. ChatGPT is incapable of that and often confidentially wrong. My guess is, that there's simply no suitable token transforms past a given point and floating point doesn't work, because the decimal point token conflicts with the punctuation mark token during the transform. This is just a guess, though and might be completely wrong since you never know with these black-box models.
  - alchemist1e9 4y ago
    Make sure you play with it yourself because you have an oversimplified model of what is happening. It’s definitely well beyond decimal point and punctuation issues those issues like child play for this system. You comment sounds like you haven’t actually use it before, I’m 99% sure. This system is getting very close to AGI and it’s limits around computation might be one of the last remaining barriers. Definitely nothing related to the . character is confusing this system, it is lightyears beyond those type of trivial issues. Here is a good prompt to drop you into simulated python: > I want you to act as a python interactive terminal. I will type actions and you will reply with what python would output. I want you to only reply with the terminal output inside one unique code block, and nothing else. Do not write explanations. Do not perform actions unless I instruct you to do so. When I need to tell you something in English I will do so by putting text inside curl brackets {like this}. Start with print(10).
    - qayxc 4y ago
      For each impressive feat there's a simple, yet embarrassing counterexample (see for instance the comment by olooney below) that clearly demonstrates how far the model is from being considered an AGI. > Definitely nothing related to the . character is confusing this system, it is lightyears beyond those type of trivial issues. Is it, though? ChatGPT: Yes, I am confident that -26.66 + 90 = 10. This is because -26.66 is the same as -26.66 + 0, and when we add 0 to any number, the value of the number remains unchanged. Therefore, -26.66 + 90 is equal to -26.66 + 0 + 90, which is equal to -26.66 + 90 = 10. Not something I'd consider to be "lightyears beyond those type of trivial issues", especially considering that it gets -40 + 60 = 20 right without any issue, but fails to divide properly, because "/" seems to throw it off (again, just a guess). You argue with the same certainty as the model argues that -26.66 + 90 = 10 :)

alchemist1e9 4y ago

Using the python entry point I can get it to correctly perform basic integer math but not anything floating point. As example of getting started: https://imgur.com/a/P29DvGO https://imgur.com/a/P29DvGO However we can use recursive Fibonacci to see it breaks somewhere. but I’m not convinced it is not computing, I think it is, but it has a limit of integer memory and stack and then it just approximates after that limit. https://imgur.com/a/gp0yIaJ https://imgur.com/a/gp0yIaJ What is incredible is that it get’s this far. It can compute but not quite correctly yet. I almost wonder if the next step is to give it general compute somehow. Train it to know it needs a computation.

qayxc 4y ago
> What is incredible is that it get’s this far. It can compute but not quite correctly yet. That's a conjecture on your part. The ability to compute is quite binary - either it can compute or can't. Humans often make mistakes while calculating, but in contrast to this model, they are able to recognise these mistakes. ChatGPT is incapable of that and often confidentially wrong. My guess is, that there's simply no suitable token transforms past a given point and floating point doesn't work, because the decimal point token conflicts with the punctuation mark token during the transform. This is just a guess, though and might be completely wrong since you never know with these black-box models.
- alchemist1e9 4y ago
  Make sure you play with it yourself because you have an oversimplified model of what is happening. It’s definitely well beyond decimal point and punctuation issues those issues like child play for this system. You comment sounds like you haven’t actually use it before, I’m 99% sure. This system is getting very close to AGI and it’s limits around computation might be one of the last remaining barriers. Definitely nothing related to the . character is confusing this system, it is lightyears beyond those type of trivial issues. Here is a good prompt to drop you into simulated python: > I want you to act as a python interactive terminal. I will type actions and you will reply with what python would output. I want you to only reply with the terminal output inside one unique code block, and nothing else. Do not write explanations. Do not perform actions unless I instruct you to do so. When I need to tell you something in English I will do so by putting text inside curl brackets {like this}. Start with print(10).
  - qayxc 4y ago
    For each impressive feat there's a simple, yet embarrassing counterexample (see for instance the comment by olooney below) that clearly demonstrates how far the model is from being considered an AGI. > Definitely nothing related to the . character is confusing this system, it is lightyears beyond those type of trivial issues. Is it, though? ChatGPT: Yes, I am confident that -26.66 + 90 = 10. This is because -26.66 is the same as -26.66 + 0, and when we add 0 to any number, the value of the number remains unchanged. Therefore, -26.66 + 90 is equal to -26.66 + 0 + 90, which is equal to -26.66 + 90 = 10. Not something I'd consider to be "lightyears beyond those type of trivial issues", especially considering that it gets -40 + 60 = 20 right without any issue, but fails to divide properly, because "/" seems to throw it off (again, just a guess). You argue with the same certainty as the model argues that -26.66 + 90 = 10 :)