5 ms·
And what do you think epochs in machine learning are? Or why more modern training efforts (i.e. for LLMs) are focussing hard on deduplicating scraped data?
by sigmoid10 3y ago
And what do you think epochs in machine learning are? Or why more modern training efforts (i.e. for LLMs) are focussing hard on deduplicating scraped data?
- llm_trw 3y agoWhy don't you tell me instead of asking questions that you surely know the answer for?
- sigmoid10 3y agoIt was rhetorical. But in case you actually don't know: what you described (i.e. multi sampling) has been common practice in ML for ages. Only now the latest models are getting so big that people are actually trying hard to move away from this idea because it would take a human lifetime in wall clock time to train a cutting edge LLM on similar datastreams.