3 ms·
Current AI models are approximation functions with huge number of parameters. These approximation functions are reasonably good at interpolation, meh at extrapo
by lsh123 3y ago
Current AI models are approximation functions with huge number of parameters. These approximation functions are reasonably good at interpolation, meh at extrapolation, and have nothing to do with generalization.
- famouswaffles 3y agoLearning in High Dimension Always Amounts to Extrapolation https://arxiv.org/abs/2110.09485 https://arxiv.org/abs/2110.09485 "Supercharged Interpolation" is not a real thing. They generalize fine when the data incentivizes that.
- lsh123 3y agoYou always can extrapolate. E.g. linear approximation for x^2 by 2 points will extrapolate reasonably well around these 2 points but will be bad with x -> +/- infinity. Similarly, there are examples where GPT invented legal cases when asked to create a legal brief.
- samus 3y agoI asked GPT 3.5 how to access the Unihan character database[0] using the ICU4J library[1]. GPT 3.5 suggested to use the `com.ibm.icu.util.Unihan` class. It actually doesn't exist, but after pointing that out to it, GPT showed me how to parse the raw database files. [0]: https://www.unicode.org/charts/unihan.html https://www.unicode.org/charts/unihan.html [1]: https://unicode-org.github.io/icu/userguide/icu4j/ https://unicode-org.github.io/icu/userguide/icu4j/