3 ms·

> Emergence is something that is intuitively noticed by human observers. The takeaway from this paper is actually that this says more about the human observers

> Emergence is something that is intuitively noticed by human observers. The takeaway from this paper is actually that this says more about the human observers than the LLMs themselves. Humans observe "emergence" using some loose metrics themselves. Whether we formalise the metrics in a quantitative way or we stay in the realm of humnan intuition is not important; we still use some criteria to analyse things, in the latter case these are more clear and amenable to critique, in the former case they are obscure and can evade critique by moving goalposts and ground easily. In the end of the day, either we talk about quantification or just qualitative observations, it is the same phenomenon we observe with the same qualities. The problem is that human intuition uses a lot of discontinuous metrics; we judge if an LLM passes or fails what we ask for it, but it is harder to judge the underlying tokenisation process as itself. For this reason, and considering the findings of this paper, the observations and claims of emergence in LLMs carry less value now, imo.

p-e-w 4y ago
Calling human intuition "just another metric" ignores the fact that human intuition performs spectacularly well at many high-level tasks. Whatever "insight" and "understanding" actually mean, there is no denying that they are immensely useful, which is why we want AIs that can replicate them. When trying to understand complex systems that don't yield to quantitative analysis in an obvious way, the starting point should be to assume that the intuitive evaluation is (roughly) correct, until proven otherwise. Trying to cast this intuition into a simple metric and then using that metric (or other simple metrics) to demonstrate that the intuition is wrong is circular reasoning.
- kelseyfrog 4y ago
  > human intuition performs spectacularly well at many high-level tasks. It's also the number one most common cognitive bias. Humans are especially prone to reification - the confusion that the construction of a measure equates to an objective reality. Humans are often launder subjectivity through the creation of metrics without 1) knowing they've done so, and 2) become emotional when accused of having done so.

p-e-w 4y ago

Calling human intuition "just another metric" ignores the fact that human intuition performs spectacularly well at many high-level tasks. Whatever "insight" and "understanding" actually mean, there is no denying that they are immensely useful, which is why we want AIs that can replicate them. When trying to understand complex systems that don't yield to quantitative analysis in an obvious way, the starting point should be to assume that the intuitive evaluation is (roughly) correct, until proven otherwise. Trying to cast this intuition into a simple metric and then using that metric (or other simple metrics) to demonstrate that the intuition is wrong is circular reasoning.

kelseyfrog 4y ago
> human intuition performs spectacularly well at many high-level tasks. It's also the number one most common cognitive bias. Humans are especially prone to reification - the confusion that the construction of a measure equates to an objective reality. Humans are often launder subjectivity through the creation of metrics without 1) knowing they've done so, and 2) become emotional when accused of having done so.