4 ms·
Funny to think that the distillation of 16 years of Facebook posts is now considered "intelligence."
by SoftTalker 2y ago
Funny to think that the distillation of 16 years of Facebook posts is now considered "intelligence."
- glitchc 2y agoThis made me chuckle.
- ithkuil 2y agoLol-ed myself but OTOH if you want a model to learn how people actually speak you cannot expect to get that by reading curated scientific documents.
- SmellTheGlove 2y agoI wonder how much safety work they have to do specifically because of this. I’d imagine their model might have a fairly paranoid, slightly racist bias if not. Particularly as younger demographics shifted away from FB in the last decade.
- kevin_thibedeau 2y agoWith suitable sentiment analysis, you can train known bigoted models and then run unknown input through them to see if it appeals to the model.
- fire_lake 2y agoConsidering the standard datasets contain 4chan posts it’s probably a marginal improvement.
- PaulHoule 2y agoActually if you want to train a model that can recognize bad things you want to have those bad things in the language model training data, otherwise it won’t see the characteristics of those things and it will later struggle to recognize them in later training stages.
- btown 2y agoPresence may be necessary, but researcher-driven weighting of different sources of content can still introduce bias. For instance, [0] suggests (sources are unclear) that OpenAI boosted by 5x the weight of their WebText2 dataset, which consists of sites linked to by upvoted Reddit comments. Reddit, in this sense, with all the biases of its various communities, was artificially elevated in importance. (Per [1], there were well-thought-through reasons for this around previous failures due to overreliance on Common Crawl, but it's nonetheless a choice that was made by humans to go in this direction.) [0] https://gregoreite.com/drilling-down-details-on-the-ai-training-datasets/#AI_Training_Datasets_TEXT_contents https://gregoreite.com/drilling-down-details-on-the-ai-train... [1] https://insightcivic.s3.us-east-1.amazonaws.com/language-models.pdf https://insightcivic.s3.us-east-1.amazonaws.com/language-mod...
- dotancohen 2y agoThat would be supervised learning: they tell the model that these are undesirable::racist values. Or just fed into the RAG::blacklist dataset.
- changoplatanero 2y agoI haven't worked at facebook for a number of years but I'm imagining what they trained on here was public instagram images and not the random text that people write on facebook. The text in facebook posts is likely to be low value but the images are a data gold mine.
- segasaturn 2y ago[flagged]
- squigz 2y agoIt's... really not. Despite what some people seem to think, most of the 3 billion+ Facebook users are normal people who aren't just posting nonsense or memes.
- solardev 2y agoTheir algorithm sure does a good job at filtering those normal people out...
- squigz 2y agoMaybe that says more about you and how you engage with it than it does the platform though.
- solardev 2y agoDoes it? I deleted my Facebook back in 2016 or so when it got so bad. Before that, I used extensions to make it show posts from friends in chronological order instead of whatever it was optimizing for by default (engagement/controversy, I think?) It's just such a toxic, manipulated environment... I don't use social media anymore, just text some friends, and am much happier.
- doublepg23 2y agoYou don’t consider HN social media?
- solardev 2y agoI guess it's a gray area? It's more like a forum to me, of the pre-Facebook sort, more like Slashdot than reddit. And there's extremely strong moderation here that does the opposite of Facebook: It optimizes against controversy and vitriol rather than encouraging it. We end up with a bunch of nerds mostly talking shop and sometimes complaining about the job market, but that's still far less ragebaitey than most social media.
- emporas 2y agoIf it is high entropy, what's the significance of the source of information's age? It could as well be 2 years old and no one would care.
- derefr 2y agoWhat you get from the self-supervised training of a base model is more like "language fluency plus a web of crystallized-knowledge relationships." But also, ML model training is a bit like the stock market: the noise/stupidity in individual examples points in a bunch of random directions, and so ends up cancelling out; while the signal all points in the same direction, and so ends up captured in the distilled model. (You might call this the "Anna Karenina principle of Information Distillation": all right answers are the same, while each wrong answer is wrong in a different way.)
- veidelis 2y agoIt's still wrong if the majority is wrong, isn't it? Like propaganda on major channels which praise the same thing, is consumed by a major part of the population which then assume that it's the right thing and continue the misinformation, which then propagates to the AI model.
- derefr 2y agoYes; but in the case of a "common misconception" like this, there's always also a nontrivial minority who do know the "right answer"[1] — and so enough examples of that occur in the training data to enable the model to embed the knowledge-web of "right ideas" (as a niche activation), alongside the "wrong idea" (in its default-mode network). The "initial fine-tuning of a 'raw' base model to produce a 'generalized pre-trained' base model" process, is commonly talked about in terms of "alignment" — making the model ethical, making it not swear at you, making it refuse to engage with certain content, etc. And really, that part is all optional, with there existing "non-aligned" or "orthogonalized" models that don't have these steps performed on them or have had them reversed, but which are still useful. But a large part of this initial fine-tuning process, consists of debiasing the model's default activation, moving it away from making associations with "common misconceptions" and toward making associations with "right answers." And this process is crucial to a model being able to reason intelligently — as these common misconceptions aren't coherent in a chain of reasoning they appear in, and so lead to the chain of reasoning falling apart / being non-productive. This is, in large part, the "secret sauce" that makes a model of a given size "more intelligent" than another model of the same size. Every base model that anyone actually cares about or uses — "aligned" or not — has had some process of de-biasing like this applied to it; or, at least, has short-cutted this process by training on a training dataset generated by or filtered by a model that has already had this de-biasing applied to it, such that the derived training dataset doesn't contain the "common misconceptions" in the first place. And when OpenAI and Meta brag about using RLHF, a large part of what they mean, is crowdsourcing recognition of long-tail "common misconceptions" at scale, to allow a much more thorough version of this de-biasing process[2]. --- [1] Of course, if nobody in the training data ever demonstrates the "right answer" knowledge/associations, then the model will never learn that knowledge/associations. But then, given that these training datasets usually represent decent samples of the population, a "right answer" being missing entirely would likely mean that nobody on Earth knows the "right answer" — so we humans wouldn't be able to recognize the model was wrong. The stock market can be wrong too, for the same reason. [2] Which, perhaps surprisingly, can add up to more than the sum of its parts. The more of this human-labelled "common misconception"-response RLHF data you have, the more you can derive patterns from this bias data. You can distil out negative examples that you can use to prompt a model to filter the training dataset; but more interestingly, you can distil positive examples of the sort of structured chains of reasoning that work best to inherently avoid triggering the bias. Where, if you can overlap the activation-space hyperspheres of many such inversion-of-bias examples, then you get, essentially, the hypersphere within the model's activation space that contains its instrumental rationality. You can then just bias the model toward living in that part of activation space as much as possible — and this shoots its apparent reasoning capacity way up.
- VoodooJuJu 2y agoI don't know why that's funny, since that is the very essence of the ideal of democracy: the distillation or averaging of the sentiment of the masses yields truth. Some contributions are outliers in one way or another, and it's these outliers that live in many people's heads, but these extremes are ironed out by averaging against the masses.
- TheOtherHobbes 2y agoMaybe the plan is to replace all FB users with bots but keep charging advertisers anyway. "But how can anyone tell?"
- jprd 2y agoI keep waiting for one of their models to just start the answer with "FWD: FWD FWD: FWD: BLAH BLAH FWD ON"
- vunderba 2y agoAmusing but a large part of the training data other LLM models is social media platforms. I mean hop on chat GPT right now and ask it to come up with any kind of novel joke. I can almost guarantee it will either be some kind of word play or other equally low hanging kind of pun that you would find on Reddit fifty replies deep. Where's the Mensa member only social media dating platform for a properly erudite and snobbishly arrogant LLM to train on?
- mig_ 2y agoGoogle scholar ?
- deleted 2y ago[deleted]
- pjs_ 2y agoThere's a fun retrospective PoV which is that Silicon Valley has been through a sequence of consecutive hype waves, each of which has been viewed with derision as mega-scale fraud or brainrot: - Video games (epic waste of time and moral debasement) gave us GPUs and NVIDIA - Social media (rage bait-fueled argument machine and cat pic repository) gave us a huge corpus of text on which to train language models. Yes scientific papers and Wikipedia are necessary but probably not sufficient? - Crypto (fraud, giant waste of resources) gave us a generation of young people who were comfortable building ludicrously oversized GPU clusters, and to some extent funded NVIDIA and TSMC R&D all of which led to where we are with AI, which is that it is making (in my humble opinion) impressive dents in the problem of solving intelligence -- cue HN commenters telling me that AI is a giant fraud and waste of resources also :)