2 ms·
That'd be too restrictive, but to get improvement in a particular domain you definitely need to train specifically for it. Just cramming more random internet te
by stymaar 1mo ago
That'd be too restrictive, but to get improvement in a particular domain you definitely need to train specifically for it. Just cramming more random internet text in a bigger model has stopped being a effective way of scaling since at least mid 2023.
- maleldil 1mo agoNo, you don't. There have been technological advances beyond just "more random Internet text", and they lead to more powerful models with more refined emergent behaviours.