4 ms·
I wonder if the makers of AI may have some index of learning material to perhaps prove they did not infringe? Or they just throw everything they get their hands
by xyzal 3y ago
I wonder if the makers of AI may have some index of learning material to perhaps prove they did not infringe? Or they just throw everything they get their hands on to the LLM?
- qayxc 3y agoIt's the latter. The current approach is akin to throwing the entire internet at the foundational model, train it for an epoch or two, and that's that. Afterwards, the finetuning with curated training material, and techniques like RLHF (Reinforcement Learning from Human Feedback) takes place. The sources of the training material are... questionable, to say the least. There's a reason the training dataset for GPT 3.5 and 4 remains undisclosed.