4 ms·
That seems like it would mean you're restricted to a dataset from before 2022ish or you have to use data that's significantly more expensive to clean. That see
by Kamq 3y ago
That seems like it would mean you're restricted to a dataset from before 2022ish or you have to use data that's significantly more expensive to clean.
That seems like it might make the training process untenable.
- coffeebeqn 3y agoIs it really true though? If they scrape HN today I don’t think their sampling would be that “polluted” compared to 2022. If you were to teach a LLM to play chess today - wouldn’t you use fully AI generated data because that’s by far the best data ?
- johntb86 3y agoIsn't all of recorded human history up until 2022 enough to train a pretty smart ai? It'll miss out on future trends, but some targeted training on trusted news sources should be enough. Eventually languages will change enough that the old training data will seem dated, but that's going to take a while to happen.
- Kamq 3y agoYeah, but basically all of that comes from post 2000. Information production seems to be on an exponential curve, so it won't be that long before you're training on a minority of all data.