3 ms·
There are hard limits to what a "brain" of a given size can do. Expecting ensembles of 7B models to match what properly trained 180B model can do at specialised
by logicchains 3y ago
There are hard limits to what a "brain" of a given size can do. Expecting ensembles of 7B models to match what properly trained 180B model can do at specialised tasks is like expecting a thousand monkeys trained on a specialised task to be able to match human performance.
- samatman 3y agoIf this were true, then GPT4 would be better at chess than Stockfish. Since it's not, it isn't.
- Terr_ 3y agoWhat if there was a large inefficient model just good enough that it could generate small specialized ones and orchestrate them? Now that I've said it, I wonder if there's any evidence human brains work that way.
- taneq 3y agoBy that logic, humans should be better at chess than Stockfish, but they aren’t.