4 ms·

> There are a few ways to look at this from the standpoint of information that is available to the decision-maker. One is that human knowledge is valuable for g

by gwern 1y ago

> There are a few ways to look at this from the standpoint of information that is available to the decision-maker. One is that human knowledge is valuable for guiding developing the model, but once you have a statistical model, it’s a better aggregator of the information. This is echoed by research on judgmental bootstrapping (https://gwern.net/doc/statistics/decision/1974-dawes.pdf https://gwern.net/doc/statistics/decision/1974-dawes.pdf), where a statistical model trained on a human expert’s past judgments will tend to outperform that expert. By the way, note that this applies to LLMs too. One of the biggest pons asinorums that people get hung up on is the idea that "it just imitates the data, therefore, it can never be better than the average datapoint (or at least, best datapoint); how could it possibly be better?" Well, we know from a long history that this is not that hard: humans make random errors all the time, and even a linear model with a few parameters or a little flowchart can outperform them. So it shouldn't be surprising or a mystery if some much more complicated AI system could too.

AIPedant 1y ago
> One of the biggest pons asinorums that people get hung up on is the idea that "it just imitates the data, therefore, it can never be better than the average datapoint (or at least, best datapoint); how could it possibly be better?" Hmm - the phrasing that perhaps holds more water is that LLMs just imitate the data, which means that novel ideas / code tends to be smashed against the force of averaging when fed into an LLM. E.g. NotebookLM summaries/podcasts are good infotainment but they tend to flatten unconventional paragraphs into platitudes or common wisdom. Obviously this is very subjective and hard to benchmark.
- airstrike 1y ago
  > Obviously this is very subjective and hard to benchmark. I agree, but it also feels very obvious once you've been exposed to it enough times. The internet is filled of written or spoken AI slop that can generally be spotted with ease by trained eyes and ears.
  - jon_richards 1y ago
    The problem making a bear-proof trash can is that there's significant overlap between the smartest bears and the dumbest tourists.
roenxi 1y ago
> and even a linear model with a few parameters Using a simple average of past performance to predict future performance is also a technique that is often disturbingly effective vs. standard practice. I suppose technically that is a linear model, but really deserves its own class.
- AstralStorm 1y ago
  Up to a point where the prediction runs afoul of the time horizon and changing unmodelled circumstances. They do not have sufficient explicit risk or variance management. Makes them highly fragile. There are more robust versions of the estimators... Still have a problem. Remember 2008? That market ran on these easy models.
- gwern 1y ago
  Yes, exponential smoothing in forecasting is another classic example of the robustness of simple models. You can throw all your fancy ARIMAs and Box-Cox transforms at a time-series, and much of the time, it is hard to distinguish from a simple moving average. Specifically, the Makridakis M forecasting competitions (https://en.wikipedia.org/wiki/Makridakis_Competitions https://en.wikipedia.org/wiki/Makridakis_Competitions) have shown for a long time that beating the baselines is shockingly difficult. In fact, classic machine learning only really started to convincingly win with the second-to-last, M5: https://www.sciencedirect.com/science/article/pii/S0169207021001874 https://www.sciencedirect.com/science/article/pii/S016920702... ; and neural methods only just sort of began working with the latest one, M6: https://www.sciencedirect.com/science/article/pii/S0169207024001079 https://www.sciencedirect.com/science/article/pii/S016920702... . (Possibly with M7 we'll see scaled-up meta-learning Transformers finally start beating the Bayesian or decision-tree forecasters. But I don't know if or when a M7 might be held.)
senkora 1y ago
> pons asinorums This is a new one for me, so, in the spirit of the article, I will "act in the world to acquire more information as needed". > An obstacle which will defeat a beginner or foolish person. [from 17th c.] > From New Latin pons asinorum, from Latin pōns (“bridge”) + genitive plural of asinus (“donkey”). Literally, “bridge of donkeys”. https://en.wiktionary.org/wiki/pons_asinorum https://en.wiktionary.org/wiki/pons_asinorum

by gwern 1y ago

AIPedant 1y ago

> One of the biggest pons asinorums that people get hung up on is the idea that "it just imitates the data, therefore, it can never be better than the average datapoint (or at least, best datapoint); how could it possibly be better?" Hmm - the phrasing that perhaps holds more water is that LLMs just imitate the data, which means that novel ideas / code tends to be smashed against the force of averaging when fed into an LLM. E.g. NotebookLM summaries/podcasts are good infotainment but they tend to flatten unconventional paragraphs into platitudes or common wisdom. Obviously this is very subjective and hard to benchmark.

airstrike 1y ago
> Obviously this is very subjective and hard to benchmark. I agree, but it also feels very obvious once you've been exposed to it enough times. The internet is filled of written or spoken AI slop that can generally be spotted with ease by trained eyes and ears.
- jon_richards 1y ago
  The problem making a bear-proof trash can is that there's significant overlap between the smartest bears and the dumbest tourists.

roenxi 1y ago

> and even a linear model with a few parameters Using a simple average of past performance to predict future performance is also a technique that is often disturbingly effective vs. standard practice. I suppose technically that is a linear model, but really deserves its own class.

AstralStorm 1y ago
Up to a point where the prediction runs afoul of the time horizon and changing unmodelled circumstances. They do not have sufficient explicit risk or variance management. Makes them highly fragile. There are more robust versions of the estimators... Still have a problem. Remember 2008? That market ran on these easy models.
gwern 1y ago
Yes, exponential smoothing in forecasting is another classic example of the robustness of simple models. You can throw all your fancy ARIMAs and Box-Cox transforms at a time-series, and much of the time, it is hard to distinguish from a simple moving average. Specifically, the Makridakis M forecasting competitions (https://en.wikipedia.org/wiki/Makridakis_Competitions https://en.wikipedia.org/wiki/Makridakis_Competitions) have shown for a long time that beating the baselines is shockingly difficult. In fact, classic machine learning only really started to convincingly win with the second-to-last, M5: https://www.sciencedirect.com/science/article/pii/S0169207021001874 https://www.sciencedirect.com/science/article/pii/S016920702... ; and neural methods only just sort of began working with the latest one, M6: https://www.sciencedirect.com/science/article/pii/S0169207024001079 https://www.sciencedirect.com/science/article/pii/S016920702... . (Possibly with M7 we'll see scaled-up meta-learning Transformers finally start beating the Bayesian or decision-tree forecasters. But I don't know if or when a M7 might be held.)

senkora 1y ago

> pons asinorums This is a new one for me, so, in the spirit of the article, I will "act in the world to acquire more information as needed". > An obstacle which will defeat a beginner or foolish person. [from 17th c.] > From New Latin pons asinorum, from Latin pōns (“bridge”) + genitive plural of asinus (“donkey”). Literally, “bridge of donkeys”. https://en.wiktionary.org/wiki/pons_asinorum https://en.wiktionary.org/wiki/pons_asinorum