7 ms·
It's kind of amazing how there seems to be a wall where sizing up the model starts to diminish in terms of intelligence gains. I guess that's why we can still
by swalsh 2y ago
It's kind of amazing how there seems to be a wall where sizing up the model starts to diminish in terms of intelligence gains. I guess that's why we can still compete with whales even though their brains are like twice as big as ours.
- vessenes 2y agoThere is a line of thinking that the required tokens shoved through training also needs to go up, perhaps super-linearly to the model size. And of course, there's a line of thinking that there are diminishing returns in all things, including these models. Both could be true!