5 ms·
> They find that with a different measuring stick, emergence vanishes. Isn’t that the case for most/all emergent behavior? If you change the scale and watch in
by tambourine_man 3y ago
> They find that with a different measuring stick, emergence vanishes.
Isn’t that the case for most/all emergent behavior? If you change the scale and watch individual water molecules, you’d see them snapping into a crystal structure one by one instead of a sudden emergent block of ice.
- godelski 3y agoNot quite. The problem is that the definition is especially poorly defined in ML. I elaborate more here[0]. You are describing emergence, but not what was claimed when LLMs were said to have emergent abilities (the distinction is explained in the article fwiw) [0] https://news.ycombinator.com/item?id=39812315 https://news.ycombinator.com/item?id=39812315
- n2d4 3y agoBut knowing the molecule structures at 50C and 75C tells you very little about the freezing point. Different example: If you measure the number of cases of a given virus, it will either spread across the entire globe (R0 > 1, eg. COVID-19) or fail to spread widely (R0 < 1, eg. Ebola). Even though it's not completely binary, it's emergent behaviour because it looks binary. But if you were to measure R0 directly, you'd see a gradual increase, and could predict future variants/vaccine efficiencies/etc much more easily. "Emergent" refers to eg. sigmoids, while "gradual" refers to eg. linear or logarithmic functions.
- throwaway290 3y agoNobody mistakes ice for something sudden emergent though, it's obvious to naked eye that it gradually appears
- maxcoder4 3y agoI don't know what exactly "emergent" means in this context, but doesn't ice appear pretty rapidly? It appears between -1 and 1 degrees, and when cooling down from 100 to 75 to 50 to 25 water you would learn nothing about the freezing point.
- throwaway290 3y agoEvery volume of water IRL will disagree about the rapidity though, go below 0 and first it's water, then it has some ice, then more ice, and only after a while eventually it's all ice (and even then maybe it has some water underneath, who knows). I can't see how it is not common sense. But with LLMs the common sense was until now "they can't then they suddenly can"
- RandomLensman 3y agoThe water usually isn't below 0 in those cases.
- throwaway290 3y agoMy thermometer says it's -5, yet water is liquid. To be more precise, covered with thin ice. And covered with (thicker) ice it shall stay all the way to -20 probably. Whatever technical correctness you throw doesn't matter to common sense logic. And that is even if there is a technical possibility to lower the temperature of every water molecule to -0.0001 simultaneously and demonstrate that it instantly freezes the entire volume in an instant. It'd be irrelevant since none of us live in that lab.
- RandomLensman 3y agoYour water is -5C? Is it salt water?
- throwaway290 3y agoThanks for another example!
- RandomLensman 3y agoFor what - that different liquids have different freezing points? Not sure anyone is doubting that.