4 ms·
It's curve fitting at its core. It just so happens that when you fit a function with enough parameters, you can start using much more abstract and higher-level
by hnuser123456 3y ago
It's curve fitting at its core. It just so happens that when you fit a function with enough parameters, you can start using much more abstract and higher-level methods to more quickly describe other outcomes and applications of this curve fitting. Really simple algebra just with a lot of variables. It's not a black box at all and it's disingenuous when people call it that. It outputs what it does because it's multiplying A and B and adding C to the input you gave it.
- jprete 3y agoThat's not a useful description, it's the equivalent of explaining the entire software stack as a pile of logic gates.
- jimbokun 3y ago"It's all just 0s and 1s."
- ogogmad 3y agoThis is a misunderstanding. You're right that you can always find some weights that approximate any reasonable (that is, continuous) mathematical function. The problem is, you have to find those weights in a small amount of computation time, and you need to estimate them from only finitely many samples. In general, ***no neural network architecture can solve this problem efficiently in every single case***. See "No Free Lunch theorem". The fact that current architectures work well in practice, despite the NFL theorem - for seemingly difficult problems - is surprising, and demands an explanation. Appendix: By "current architecture", I mean Transformers plus Stochastic Gradient Descent.