3 ms·

> To the best of my knowledge—and I double-checked—this exam has never before been posted on the public Internet, and could not have appeared in GPT-4’s trainin

> To the best of my knowledge—and I double-checked—this exam has never before been posted on the public Internet, and could not have appeared in GPT-4’s training data. Sure, but you can Google the answers to most of the questions. Personally I've accepted that GPT does learn and apply concepts present in its training data, and all of this would be. Learning is part of intelligence but not the whole thing. (I thought this was why, decades ago, most of the research in this general area rebranded itself from the ambitious goal of artificial intelligence to the more humble but accessible goal of machine learning.) The interesting question now is how much better it can get, in difficulty it can tackle, reliability, and quality of explanations. How long before AI can answer questions whose answers are not widely known, for which templates do not already exist? Personally, I believe this will not be some trivial matter of simply scaling up more. The waters of intelligence are much deeper than that. It's hard to believe given the rapid one two punch of GPT 3.5 and 4, but we are about to stall. If I'm wrong, mark this comment and make fun of me in five years. Wrong or right, it's going to be interesting!

munchler 4y ago
> you can Google the answers to most of the questions 1) Students can Google the answers also, but neither students nor GPT-4 are allowed to Google the answers during the test, so it remains a fair comparison. 2) Many of the questions require calculations, which are far less Googleable.
- civilized 4y ago
  1) It's not possible to fairly compare human intelligence with something that can memorize gigabytes of text and hold it in non-volatile memory. 2) Months ago, in my earliest interactions with ChatGPT, I asked it to solve math problems. It gave me back stuff with LaTeX formatting. Obviously it had, if not these exact problems, similar templates in its training set. Recently it was shown that GPT is completely incapable of solving Codeforces problems that appeared after it was trained. Whatever is going on here is interesting but less impressive than it looks.
  - munchler 4y ago
    1) Do you really think ChatGPT works by memorizing quantum mechanics textbooks? There are only 355 billion parameters in GPT-3.5, which is several orders of magnitude less than the 600 trillion synapses in the human brain. 2) Your conclusion is unfounded. ChatGPT speaks many languages, including Latex.
    - civilized 4y ago
      Beautiful theory, meet ugly fact: https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks https://aisnakeoil.substack.com/p/gpt-4-and-professional-ben...
  - Sharlin 4y ago
    > It gave me back stuff with LaTeX formatting. So? It can write LaTeX just like it can write Java or Python or Rust.
    - civilized 4y ago
      It put LaTeX in there unbidden because it pattern matched to LaTeX source code in its training set. Code containing solved math problems similar to those it was being asked.
sebzim4500 4y ago
I used to agree with you. The paper that made me unsure was "Transformers learn in-context by gradient descent" [1]. Basically, the model learns weights that let it run gradient descent at inference time in order to do in context learning. If transformers can learn this, then I think they can learn almost anything given enough compute/parameters/data. Of course, even if this is true then it's possible that there simply isn't enough high quality data, or that the amount of compute required is beyond our current hardware. [1] https://arxiv.org/abs/2212.07677 https://arxiv.org/abs/2212.07677
- civilized 4y ago
  Too theoretical to be pivotal. Years ago we had results about how all sorts of learning was reducible to kernel SVM and that drove a big fad for kernel SVM research at NeurIPS and other hot conferences. Nobody cares about SVM anymore. So we don't have a good enough grasp on the theory of AI to predict what is going to be a breakthrough. Only rigorous empirical results convince. Second, the idea of meta-learning gradient descent, or that some learning algorithms can do it, isn't new or specific to Transformer architectures.
  - sebzim4500 4y ago
    I'm not suggesting that you can use meta-learning to discover gradient descent, that would not be surprising. My claim is that GPT-4 (probably even GPT-2) is almost certainly already using gradient descent internally inside it's own activations. How else can it learn a random small neural network given enough samples in the prompt?
red75prime 4y ago
> but we are about to stall Maybe if it's a deliberate decision by the researchers. A path to more capabilities is rather clear (recurrence and explicit memory).
- civilized 4y ago
  No, they won't make any real attempt to slow down. No, they won't do more than make incremental advances. To the extent that large benchmark gains occur, they will be shown to have little practical value.
  - red75prime 4y ago
    If it'll come to pass, we'll probably have a groundbreaking discovery that the brain violates the physical Church-Turing thesis on our hands. That would be quite unexpected.

munchler 4y ago

> you can Google the answers to most of the questions 1) Students can Google the answers also, but neither students nor GPT-4 are allowed to Google the answers during the test, so it remains a fair comparison. 2) Many of the questions require calculations, which are far less Googleable.

civilized 4y ago
1) It's not possible to fairly compare human intelligence with something that can memorize gigabytes of text and hold it in non-volatile memory. 2) Months ago, in my earliest interactions with ChatGPT, I asked it to solve math problems. It gave me back stuff with LaTeX formatting. Obviously it had, if not these exact problems, similar templates in its training set. Recently it was shown that GPT is completely incapable of solving Codeforces problems that appeared after it was trained. Whatever is going on here is interesting but less impressive than it looks.
- munchler 4y ago
  1) Do you really think ChatGPT works by memorizing quantum mechanics textbooks? There are only 355 billion parameters in GPT-3.5, which is several orders of magnitude less than the 600 trillion synapses in the human brain. 2) Your conclusion is unfounded. ChatGPT speaks many languages, including Latex.
  - civilized 4y ago
    Beautiful theory, meet ugly fact: https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks https://aisnakeoil.substack.com/p/gpt-4-and-professional-ben...
- Sharlin 4y ago
  > It gave me back stuff with LaTeX formatting. So? It can write LaTeX just like it can write Java or Python or Rust.
  - civilized 4y ago
    It put LaTeX in there unbidden because it pattern matched to LaTeX source code in its training set. Code containing solved math problems similar to those it was being asked.

sebzim4500 4y ago

I used to agree with you. The paper that made me unsure was "Transformers learn in-context by gradient descent" [1]. Basically, the model learns weights that let it run gradient descent at inference time in order to do in context learning. If transformers can learn this, then I think they can learn almost anything given enough compute/parameters/data. Of course, even if this is true then it's possible that there simply isn't enough high quality data, or that the amount of compute required is beyond our current hardware. [1] https://arxiv.org/abs/2212.07677 https://arxiv.org/abs/2212.07677

civilized 4y ago
Too theoretical to be pivotal. Years ago we had results about how all sorts of learning was reducible to kernel SVM and that drove a big fad for kernel SVM research at NeurIPS and other hot conferences. Nobody cares about SVM anymore. So we don't have a good enough grasp on the theory of AI to predict what is going to be a breakthrough. Only rigorous empirical results convince. Second, the idea of meta-learning gradient descent, or that some learning algorithms can do it, isn't new or specific to Transformer architectures.
- sebzim4500 4y ago
  I'm not suggesting that you can use meta-learning to discover gradient descent, that would not be surprising. My claim is that GPT-4 (probably even GPT-2) is almost certainly already using gradient descent internally inside it's own activations. How else can it learn a random small neural network given enough samples in the prompt?

red75prime 4y ago

> but we are about to stall Maybe if it's a deliberate decision by the researchers. A path to more capabilities is rather clear (recurrence and explicit memory).

civilized 4y ago
No, they won't make any real attempt to slow down. No, they won't do more than make incremental advances. To the extent that large benchmark gains occur, they will be shown to have little practical value.
- red75prime 4y ago
  If it'll come to pass, we'll probably have a groundbreaking discovery that the brain violates the physical Church-Turing thesis on our hands. That would be quite unexpected.