4 ms·
> Was the evolutionary convergence of those weights not itself training? No, inductive biases are not training. I'm saying that better models(ie: better induc
by kelseyfrog 3y ago
> Was the evolutionary convergence of those weights not itself training?
No, inductive biases are not training.
I'm saying that better models(ie: better inductive biases) or non-language data is needed to advance LLMs and somehow we've arrived at "evolution is training." I'm not sure how that's relevant to the point.
- famouswaffles 3y agoEvolutionary training is not just inductive bias. They're not comparable at all lol. And the more inductive bias we've shoved into models, the worse they've performed. Transformers have a lot less bias than either RNNs or CNNs and are better for it. Same story with what preceded both.
- kelseyfrog 3y agoThis is a good time to stop and ask, what point do you think I'm making?
- stevenhuang 3y agoPretty much all of your assertions in this thread and https://news.ycombinator.com/item?id=37797108 https://news.ycombinator.com/item?id=37797108 in particular are what we're disagreeing with.
- kelseyfrog 3y agoRight, I'm checking that you understand what point I'm making by asking you to reflect it.