4 ms·

It's always easier to predict the future after it's arrived. There may be some emergent features that represent phase changes that are truly difficult, even in

It's always easier to predict the future after it's arrived. There may be some emergent features that represent phase changes that are truly difficult, even in retrospect, to predict, but I doubt these are common (does anyone even have a single convincing LLM example?). I think the more normal case is that higher level capabilities are dependent on multiple lower level ones in ways that may be hard to predict. These smooth improvements in the constituent building blocks may often be there, but you'd have to know where to look (i.e. what the critical building blocks are). In order to predict emergent capabilities you'd need to have identified ahead of time what the necessary building blocks are, and maybe done some simulation of what level each of those need would need to be at, to support the anticipated/hoped-for behavior. Of course it's not just about scaling model size and/or data, but also about type and quality of data, where there may be abrupt changes between model versions. It's going to be very difficult ahead of time to analyze what new patterns/manipulations (i.e. building-block capabilities) the model will be able to learn from an updated training set, and therefore what emergent abilities can be built out of those new blocks. I wonder how often the opposite happens, and model designers have been able to successfully identify that "in order to do X, we need capabilities A, B and C, and in order to get capabilities A, B and C, we need new datasets P & Q" ? In a hypothetical case like that they'd have been able to measure progress towards capability X.

Jensson 3y ago
> I wonder how often the opposite happens, and model designers have been able to successfully identity that "in order to do X, we need capabilities A, B and C, and in order to get capabilities A, B and C, we need new datasets P & Q" ? In a hypothetical case like that they'd have been able to measure progress towards capability X. As far as I know before transformers that just didn't happen at all since models couldn't contain that many separate skills without them interfering with each other. This whole thing with models having many high quality capabilities is still pretty new. But yeah, I think we need something like that and all the best LLMs today probably already do it, but none have said what they do so that is just speculation.
- HarHarVeryFunny 3y ago
  I'm sure that there are plenty of cases where model designers are at least attempting to pursue new capabilities in a targeted approach (but how often to this degree of complexity?), while at the same time realizing that new models+datasets will also have unanticipated new capabilities. Before LLMs (whether transformer-based or not) most NNs were built to perform single tasks - a single objective, so having multiple higher-level capabilities was essentially out of the question. Of course LLMs nominally only have a single objective too, predict next word, but really they are targeting language. In the GOFAI era of rule-based symbolic AI there were also some systems/approaches that had multiple skills (e.g. expert systems like CYC, or cognitive architectures like SOAR), so maybe there are forgotten lessons there on decomposability of skills.

Jensson 3y ago

> I wonder how often the opposite happens, and model designers have been able to successfully identity that "in order to do X, we need capabilities A, B and C, and in order to get capabilities A, B and C, we need new datasets P & Q" ? In a hypothetical case like that they'd have been able to measure progress towards capability X. As far as I know before transformers that just didn't happen at all since models couldn't contain that many separate skills without them interfering with each other. This whole thing with models having many high quality capabilities is still pretty new. But yeah, I think we need something like that and all the best LLMs today probably already do it, but none have said what they do so that is just speculation.

HarHarVeryFunny 3y ago
I'm sure that there are plenty of cases where model designers are at least attempting to pursue new capabilities in a targeted approach (but how often to this degree of complexity?), while at the same time realizing that new models+datasets will also have unanticipated new capabilities. Before LLMs (whether transformer-based or not) most NNs were built to perform single tasks - a single objective, so having multiple higher-level capabilities was essentially out of the question. Of course LLMs nominally only have a single objective too, predict next word, but really they are targeting language. In the GOFAI era of rule-based symbolic AI there were also some systems/approaches that had multiple skills (e.g. expert systems like CYC, or cognitive architectures like SOAR), so maybe there are forgotten lessons there on decomposability of skills.