3 ms·
In the case of AI, the more parameters, the better! In Physics is the opposite.
by maitola 2y ago
In the case of AI, the more parameters, the better! In Physics is the opposite.
- elijahbenizzy 2y agoOne of the hardest parts of training models is avoiding overfitting, so "more parameters are better" should be more like "more parameters are better given you're using those parameters in the right way, which can get hard and complicated". Also LLMs just straight up do overfit, which makes them function as a database, but a really bad one. So while more parameters might just be better, that feels like a cop-out to the real problem. TBD what scaling issues we hit in the future.
- will1am 2y agoA dichotomy between these fields