3 ms·
More accurately: Modern chat-oriented LLMs are not simply statistical models trained on web scale datasets. Instead, they are the result of a two-stage process
by hapali 1y ago
More accurately:
Modern chat-oriented LLMs are not simply statistical models trained on web scale datasets. Instead, they are the result of a two-stage process: first, large-scale pretraining on internet data, and then extensive fine-tuning through human feedback. Much of what makes these models feel responsive, safe, or emotionally intelligent is the outcome of thousands of hours of human annotation, often performed by outsourced data labelers around the world. The emotional skill and nuance attributed to these systems is, in large part, a reflection of the preferences and judgments of these human annotators, not merely the accumulation of web text.
So, when you interact with an advanced LLM, you’re not just engaging with a statistical model, nor are you simply seeing the unfiltered internet regurgitated back to you. Rather, you’re interacting with a system whose responses have been shaped and constrained by large-scale human feedback—sometimes from workers in places like Kenya—generalized through a neural network to handle any topic you bring up.
- tim333 1y agoSounds a bit like humans. Much data modified by "don't hit your sister" etc.
- fragmede 1y ago> and then extensive fine-tuning through human feedback how extensive is the work involved to take a model that's willing to talk about Tianamen square into one that isn't? What's involved with editing Llama to tell me how to make cocaine/bombs/etc? It's not so extensive so as to require an army of subcontractors to provide large scale human feedback.