2 ms·
30,000 words wouldn't be enough to train this from scratch - you'd ideally train from hundreds of millions of words at least. 30,000 words would be enough to f
by londons_explore 4y ago
30,000 words wouldn't be enough to train this from scratch - you'd ideally train from hundreds of millions of words at least.
30,000 words would be enough to finetune an existing model. If you did that, then the model would output text similar to the finetuning data. For example, if you finetuned it on shakespeare, then you might be able to use the model to make a new play, in shakespeare's style.
- ProjectArcturis 4y agoIf you finetuned it on the text of Shakespeare's plays, how would it link that text to the string "Shakespeare"?
- londons_explore 4y agoIt still has the knowledge from the main training on data from across the whole internet, so would still know the word Shakespeare... But you're right - the model finetuned on shakespeare would be good at writing a new play in the style of shakespeare, but would be bad at giving a critique of shakespeare's works.