3 ms·

Here is Jason Wei's response "Common arguments regarding emergent abilities": https://www.jasonwei.net/blog/common-arguments-regarding-emergent-abilities https:

by lhl 3y ago

Here is Jason Wei's response "Common arguments regarding emergent abilities": https://www.jasonwei.net/blog/common-arguments-regarding-emergent-abilities https://www.jasonwei.net/blog/common-arguments-regarding-eme... Wei was the lead author of the original "Emergent Abilities" paper: Wei, Jason, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, et al. “Emergent Abilities of Large Language Models.” arXiv, October 26, 2022. https://doi.org/10.48550/arXiv.2206.07682 https://doi.org/10.48550/arXiv.2206.07682.

ljlolel 3y ago
This response is very poor and ignores many very well-developed arguments in the Stanford paper (such as incorrect NLP regex, or that exact answer nonlinearity can still be measured more finely with larger number of exact answer test questions)
- jiocrag 3y ago
  It’s literally addressed in his first bullet point… “ Response: While there is evidence that some tasks that appear emergent under exact match have smoothly improving performance under another metric, I don’t think this rebuts the significance of emergence, since metrics like exact match are what we ultimately want to optimize for many tasks.”
  - ljlolel 3y ago
    You haven’t read the paper so you don’t understand what I just wrote and why Wei is not responding to the paper. It’s not a different metric
    - jiocrag 3y ago
      I did read the paper and Wei is responding to it, and others. What do you mean "It's not a different metric?"
jmull 3y ago
> we … defined emergent abilities as “abilities that are not present in small language models but are present in large language models.” This is a poor definition because it doesn’t match what is generally meant by emergent behavior and abilities, which leads to people talking past each other. For example, it can easily be true that LLMs have emergent behaviors by this definition and not be greater than the sum of their parts. A better definition would be exhibiting abilities or behaviors not in the training data.
- chaxor 3y ago
  It seems a positive control is missing in these studies. Perhaps humans (not a perfect solution, but unfortunately the best 'learning system' available) could be used as a control?
- stephendause 3y ago
  Hmm; I would think a better definition would include a description of the behavior as being greater than the sum of its parts. I can imagine abilities and behaviors being in the training data but not exhibited by a smaller LLM but being present in a bigger one. My favorite example is the ability to multiply large number consistently and accurately (e.g. 23540932 x 3274, which ChatGPT-4 just failed to do correctly for me when I asked it). A better LLM would learn how to carry the process out step-by-step.
muskmusk 3y ago
> I don’t think this rebuts the significance of emergence, since metrics like exact match are what we ultimately want to optimize for many tasks. Consider asking ChatGPT what 15 + 23 is—you want the answer to be 38, and nothing else. Maybe 37 is closer to 38 than -2.591, but assigning some partial credit to that answer seems unhelpful for testing ability to do that task, and how to assign it would be arbitrary. Not sure I can agree with this. Let's for the sake of the argument say that normal calculators don't exist. I can choose to run my calculation thought ChatGPT or do it myself. It is true that I would vastly prefer the answer to always be correct, but it's not true that there is never value in bounding the error. Put another way: would you rather want a calculator that is at most 5% off 100% of the time. Or a calculator that is 100% correct 95% of the time, but may output garbage the remaining 5%? Upon further reflection: it seems mighty ambitious to expect a statistical model to be always correct. If always correct was possible you probably didn't need a statistical model to begin with. Given that the model will fall from time to time it seems especially useful that you can bound the error somehow. +1 for smooth metrics i guess. I guess all of this is slightly off a tangent as Wei seems to be just arguing that "there might be things LLM's can do or will learn to do in the future that we didn't train them for. And we won't necessarily be able to predict it from capabilities of smaller models". I agree with this, just not the way he arrives at the conclusion.
- carlitossway 3y ago
  And in turn, I cannot agree with you. I can do math myself, thank you very much. If I take the effort to pull out a math calculating machine, the calculation is important enough that I want 99.99% certainty that the answer is right. Under the premise that the answer is wrong, I want 99.999% certainty that the failure is related to data capture and not the calculating mechanism itself. Within the context of LLMs, the goal should be to make them recognize that the problem at hand is not well suited for its own set of capabilities, and to defer to an external system. In the case of ChatGPT, any numeric answer to "15+23" (including and specially "38") is wrong. The correct answer is "sorry, as a Large Language Model I can retrieve information that is encoded in English and I am not specially suited to solve math problems". A LLM-empowered Alexa may be proactive, run a Calculator program for you and report back "According to bc program, the answer is 38. Here's the full log for you to check if this is what you wanted" This is not so much AGI as deliberate programmatic patching, not unlike the disclaimers when you ask ChatGPT about controversial topics.

by lhl 3y ago

ljlolel 3y ago

This response is very poor and ignores many very well-developed arguments in the Stanford paper (such as incorrect NLP regex, or that exact answer nonlinearity can still be measured more finely with larger number of exact answer test questions)

jiocrag 3y ago
It’s literally addressed in his first bullet point… “ Response: While there is evidence that some tasks that appear emergent under exact match have smoothly improving performance under another metric, I don’t think this rebuts the significance of emergence, since metrics like exact match are what we ultimately want to optimize for many tasks.”
- ljlolel 3y ago
  You haven’t read the paper so you don’t understand what I just wrote and why Wei is not responding to the paper. It’s not a different metric
  - jiocrag 3y ago
    I did read the paper and Wei is responding to it, and others. What do you mean "It's not a different metric?"

jmull 3y ago

> we … defined emergent abilities as “abilities that are not present in small language models but are present in large language models.” This is a poor definition because it doesn’t match what is generally meant by emergent behavior and abilities, which leads to people talking past each other. For example, it can easily be true that LLMs have emergent behaviors by this definition and not be greater than the sum of their parts. A better definition would be exhibiting abilities or behaviors not in the training data.

chaxor 3y ago
It seems a positive control is missing in these studies. Perhaps humans (not a perfect solution, but unfortunately the best 'learning system' available) could be used as a control?
stephendause 3y ago
Hmm; I would think a better definition would include a description of the behavior as being greater than the sum of its parts. I can imagine abilities and behaviors being in the training data but not exhibited by a smaller LLM but being present in a bigger one. My favorite example is the ability to multiply large number consistently and accurately (e.g. 23540932 x 3274, which ChatGPT-4 just failed to do correctly for me when I asked it). A better LLM would learn how to carry the process out step-by-step.

muskmusk 3y ago

> I don’t think this rebuts the significance of emergence, since metrics like exact match are what we ultimately want to optimize for many tasks. Consider asking ChatGPT what 15 + 23 is—you want the answer to be 38, and nothing else. Maybe 37 is closer to 38 than -2.591, but assigning some partial credit to that answer seems unhelpful for testing ability to do that task, and how to assign it would be arbitrary. Not sure I can agree with this. Let's for the sake of the argument say that normal calculators don't exist. I can choose to run my calculation thought ChatGPT or do it myself. It is true that I would vastly prefer the answer to always be correct, but it's not true that there is never value in bounding the error. Put another way: would you rather want a calculator that is at most 5% off 100% of the time. Or a calculator that is 100% correct 95% of the time, but may output garbage the remaining 5%? Upon further reflection: it seems mighty ambitious to expect a statistical model to be always correct. If always correct was possible you probably didn't need a statistical model to begin with. Given that the model will fall from time to time it seems especially useful that you can bound the error somehow. +1 for smooth metrics i guess. I guess all of this is slightly off a tangent as Wei seems to be just arguing that "there might be things LLM's can do or will learn to do in the future that we didn't train them for. And we won't necessarily be able to predict it from capabilities of smaller models". I agree with this, just not the way he arrives at the conclusion.

carlitossway 3y ago
And in turn, I cannot agree with you. I can do math myself, thank you very much. If I take the effort to pull out a math calculating machine, the calculation is important enough that I want 99.99% certainty that the answer is right. Under the premise that the answer is wrong, I want 99.999% certainty that the failure is related to data capture and not the calculating mechanism itself. Within the context of LLMs, the goal should be to make them recognize that the problem at hand is not well suited for its own set of capabilities, and to defer to an external system. In the case of ChatGPT, any numeric answer to "15+23" (including and specially "38") is wrong. The correct answer is "sorry, as a Large Language Model I can retrieve information that is encoded in English and I am not specially suited to solve math problems". A LLM-empowered Alexa may be proactive, run a Calculator program for you and report back "According to bc program, the answer is 38. Here's the full log for you to check if this is what you wanted" This is not so much AGI as deliberate programmatic patching, not unlike the disclaimers when you ask ChatGPT about controversial topics.