6 ms·

It's well documented that these models can solve variations of questions that are not found anywhere in their training set, and even entirely novel problems inv

It's well documented that these models can solve variations of questions that are not found anywhere in their training set, and even entirely novel problems invented by prompters. Not with 100% success, but they can do it far with a rate far better than chance, so the idea that they're pulling responses from their training data is simply not correct.

331c8c71 4y ago
Well, my experience was that ChatGPT produced utter nonsense including literally "1/19+1/19=2/19=1" when asked a simple probability question.
- shaunsingh0207 4y ago
  IQ test, the sat, and exact mathematics are very different fields though. ChatGPT etc. are very good at "emulating" by sheer force and size logic, exact math doesn't work with "emulation"
- nayroclade 4y ago
  Transcript of a conversation I just had with ChatGPT: Prompt: I'm talking with a user named "331c8c71" on Hacker News. According to statistics I found by Googling, the number of Hacker News readers living in the USA is 36%. Assuming this is correct, what is the probability that 331c8c71 does not reside in the USA? Please explain your reasoning. Response: If we assume that the statistic you found is accurate, then we know that 36% of Hacker News readers live in the USA. Therefore, the probability that any given Hacker News reader does not live in the USA is 100% - 36% = 64%. However, we do not know if 331c8c71 is a typical Hacker News reader. They could be part of a demographic that is more or less likely to live in the USA. Without more information, we cannot say for certain what the probability is that 331c8c71 does not reside in the USA. So, the answer is 64% assuming that 331c8c71 is a typical Hacker News reader, but it may not be an accurate estimate depending on additional information about the user. --- Like I said, these models are far from 100% accurate, and there is lots they get wrong, but they clearly are capable of some kind of reasoning that goes beyond simple text substitution of training data.
  - kenjackson 4y ago
    The fact that just a few years ago even parsing the question would be an achievement. I feel like people are just glossing over how incredible this answer is.
    - magicalhippo 4y ago
      Also it doesn't seem like such a leap to couple these language models with dedicated computation systems and similar. Think training a model to feed prompts to Wolfram Alpha to actually compute the results, then reporting back.
    - Workaccount2 4y ago
      I feel like there is some contingent of people who are really bent on downplaying the achievements of AI as of late. Its objectively insane, yet somehow every discussion is still sprinkled with some form of "It told me 8x8=60 so I closed the window and never used it again"
      - 331c8c71 4y ago
        I am impressed with LLMs but I think their inability to produce an honest "I don't know" instead of hallucinating is an issue.
        zuminator 4y ago
        True, but the problem is that they never know. They don't interact with the real world and have no way of verifying their training data's accuracy. Perhaps they could assign a confidence level to their response? But then, if they assign a response a high confidence level and produce an incorrect answer, it will compound their failure. Not only were they wrong, but they vouched for their wrong answer.
      - d0mine 4y ago
        Google search was incredibly valuable immediately even if most links could have been rubbish. I can't say the same with the current LLMs It is an incredible achievement that LLMs produce human-like output (e.g., wouldn't know if a gpt bot answers me unless we are discussing a topic where precision/accuracy are important) but they hallucinate (they are confident BS-generators). The hype is that LLMs can solve any problem and replace humans (jobs). It is not so. It may depend on what you do but I find it is easier/faster to do the work myself then to spot and fix [a possibly subtle] error in AI output. Though some of the specific things will improve in time and you can find tasks where AI is useful even today. I don't see how the models can improve for general tasks (AGI) without being existential threat to humans (not just jobs).
bootsmann 4y ago
Ok maybe I should rephrase but think of it like this: Would your IQ test score be accurate if you had a week beforehand to train solving IQ tests? You don't get exactly the same test in the end, similar with SAT, but the constraints we put on these tests (they have to be comparable) produce patterns in the questions you can train for. This is the same logic why people can train to improve their SAT scores, if they were a measure of true innate intelligence their training would have no impact on their score.
- siva7 4y ago
  I mean, let's be fair. That's also how many humans learn and do IQ tests.
  - prng2021 4y ago
    Yes but the critical difference is that AI couldn’t pass these tests if they weren’t trained on a very similar set of questions and answers. Not every person takes a SAT prep class to improve their test score. There are lots of people who are truly above average in terms of intelligence and can score very high on the first try.
    - naasking 4y ago
      > Yes but the critical difference is that AI couldn’t pass these tests if they weren’t trained on a very similar set of questions and answers. First, I don't think that's strictly true. Obviously they wouldn't do as well but they still do better than chance. Second, there's evidence that this is a big part of the Flynn effect, which means humans are susceptible to a similar phenomenon.
      - prng2021 4y ago
        “ Obviously they wouldn't do as well but they still do better than chance” is a huge understatement. If a model wasn’t trained with any SAT questions and answers but was trained with the same verbal and mathematical knowledge a high school student would have, the AI would do extremely poorly in an actual test. In contrast, the vast majority of human test takers would score leagues above picking answers by chance. Your original reply insinuated that the AI is learning very similar to how humans do and that’s just not true. Yes, humans do pattern matching based on prior experiences/knowledge like AI does when you train a model, but human intelligence goes way beyond that.

331c8c71 4y ago

Well, my experience was that ChatGPT produced utter nonsense including literally "1/19+1/19=2/19=1" when asked a simple probability question.

shaunsingh0207 4y ago
IQ test, the sat, and exact mathematics are very different fields though. ChatGPT etc. are very good at "emulating" by sheer force and size logic, exact math doesn't work with "emulation"
nayroclade 4y ago
Transcript of a conversation I just had with ChatGPT: Prompt: I'm talking with a user named "331c8c71" on Hacker News. According to statistics I found by Googling, the number of Hacker News readers living in the USA is 36%. Assuming this is correct, what is the probability that 331c8c71 does not reside in the USA? Please explain your reasoning. Response: If we assume that the statistic you found is accurate, then we know that 36% of Hacker News readers live in the USA. Therefore, the probability that any given Hacker News reader does not live in the USA is 100% - 36% = 64%. However, we do not know if 331c8c71 is a typical Hacker News reader. They could be part of a demographic that is more or less likely to live in the USA. Without more information, we cannot say for certain what the probability is that 331c8c71 does not reside in the USA. So, the answer is 64% assuming that 331c8c71 is a typical Hacker News reader, but it may not be an accurate estimate depending on additional information about the user. --- Like I said, these models are far from 100% accurate, and there is lots they get wrong, but they clearly are capable of some kind of reasoning that goes beyond simple text substitution of training data.
- kenjackson 4y ago
  The fact that just a few years ago even parsing the question would be an achievement. I feel like people are just glossing over how incredible this answer is.
  - magicalhippo 4y ago
    Also it doesn't seem like such a leap to couple these language models with dedicated computation systems and similar. Think training a model to feed prompts to Wolfram Alpha to actually compute the results, then reporting back.
  - Workaccount2 4y ago
    I feel like there is some contingent of people who are really bent on downplaying the achievements of AI as of late. Its objectively insane, yet somehow every discussion is still sprinkled with some form of "It told me 8x8=60 so I closed the window and never used it again"
    - 331c8c71 4y ago
      I am impressed with LLMs but I think their inability to produce an honest "I don't know" instead of hallucinating is an issue.
      - zuminator 4y ago
        True, but the problem is that they never know. They don't interact with the real world and have no way of verifying their training data's accuracy. Perhaps they could assign a confidence level to their response? But then, if they assign a response a high confidence level and produce an incorrect answer, it will compound their failure. Not only were they wrong, but they vouched for their wrong answer.
    - d0mine 4y ago
      Google search was incredibly valuable immediately even if most links could have been rubbish. I can't say the same with the current LLMs It is an incredible achievement that LLMs produce human-like output (e.g., wouldn't know if a gpt bot answers me unless we are discussing a topic where precision/accuracy are important) but they hallucinate (they are confident BS-generators). The hype is that LLMs can solve any problem and replace humans (jobs). It is not so. It may depend on what you do but I find it is easier/faster to do the work myself then to spot and fix [a possibly subtle] error in AI output. Though some of the specific things will improve in time and you can find tasks where AI is useful even today. I don't see how the models can improve for general tasks (AGI) without being existential threat to humans (not just jobs).

bootsmann 4y ago

Ok maybe I should rephrase but think of it like this: Would your IQ test score be accurate if you had a week beforehand to train solving IQ tests? You don't get exactly the same test in the end, similar with SAT, but the constraints we put on these tests (they have to be comparable) produce patterns in the questions you can train for. This is the same logic why people can train to improve their SAT scores, if they were a measure of true innate intelligence their training would have no impact on their score.

siva7 4y ago
I mean, let's be fair. That's also how many humans learn and do IQ tests.
- prng2021 4y ago
  Yes but the critical difference is that AI couldn’t pass these tests if they weren’t trained on a very similar set of questions and answers. Not every person takes a SAT prep class to improve their test score. There are lots of people who are truly above average in terms of intelligence and can score very high on the first try.
  - naasking 4y ago
    > Yes but the critical difference is that AI couldn’t pass these tests if they weren’t trained on a very similar set of questions and answers. First, I don't think that's strictly true. Obviously they wouldn't do as well but they still do better than chance. Second, there's evidence that this is a big part of the Flynn effect, which means humans are susceptible to a similar phenomenon.
    - prng2021 4y ago
      “ Obviously they wouldn't do as well but they still do better than chance” is a huge understatement. If a model wasn’t trained with any SAT questions and answers but was trained with the same verbal and mathematical knowledge a high school student would have, the AI would do extremely poorly in an actual test. In contrast, the vast majority of human test takers would score leagues above picking answers by chance. Your original reply insinuated that the AI is learning very similar to how humans do and that’s just not true. Yes, humans do pattern matching based on prior experiences/knowledge like AI does when you train a model, but human intelligence goes way beyond that.