4 ms·
Very well done animation. In no way does that change my mind about the eggregious continuing use of words like "learn" for what happens inside the software syst
by ggm 3y ago
Very well done animation. In no way does that change my mind about the eggregious continuing use of words like "learn" for what happens inside the software system.
To me, the english-language choices in how LLM and Transformers are described as working is part and parcel of the "sell" of steps-to-AGI because of normalising use of words which imply intentionality, introspection (as humans know it), thinking, cognition.
So I both love and loathe this animation. It re-inforces people's naieve tendency to "its alive" when in fact, its doing a really good job of explaining how weighted sums and statistics inform the parse, and the consequences of modulation of the products of the parse.
The programmers were very smart people. They've done well and have codified a huge body of knowledge which helps systems with producing outputs from input strings, in line with our requirements. Sounds exciting doesn't it!
- ben_w 3y ago> and have codified a huge body of knowledge which helps systems with producing outputs from input strings That is specifically what the programmers have not done.
- famouswaffles 3y agoYeah. Maybe it's because we use the word "train" and that conjures up a mental model of some artisan distilling his process to an an apprentice ?
- ggm 3y agoThis artisan hasn't been trained in how to make steel. It's been trained in how to talk about steel making plausibly.
- famouswaffles 3y agoMy man you're completely off base here. No one is programming anything into neural networks and we have no idea how models like GPT-4 work at a high level. The whole point of neural networks is that we don't know shit and we don't know shit about how to distill the mechanics of some very real, very important processes so we figured out a way to make the machine figure this out on its own.
- ggm 3y agoYou're saying that the billions of lines of text have not informed a statistical model of relationships of words in grammer? You think the "body of knowledge" has to be in the meaning of the words? No. It's in the weighted maths of the likelihood of each element of the phrase. That statistical model is knowledge. Some semantic intent is there too, to disambiguate who flies like an arrow or flies like a banana.
- ben_w 3y ago> That statistical model is knowledge Glad you agree. None of the free parameters of these statistical models were selected by programmers.
- ggm 3y agoHmm. Is adjusting weights metaprogramming? It's based on observations of the outputs of the system. It's certainly not directed programming intent in the normal sense. It shares one property of traditional programming: garbage in garbage out. The fruits of their labour indirectly is a model. It's proving valuable. You want to ascribe the value to .. the bit nobody claims to understand? I'm sticking to the labour theory of value here, either the robot makers imbued value, or we have to redefine the value.
- deleted 3y ago[deleted]
- xcv123 3y agoHumans are not adjusting the weights. That is done automatically by the algorithm. The humans only provide training input.
- ggm 3y agoSo when people told the staff the models were racist you think they just "asked it to be less racist" I think they did a little more than just adjust the input data, they also modulated the logic behind the Weighting methods. No?
- xcv123 3y agoSo the term "Machine Learning" is egregious misuse of language? These models are not explicitly programmed. They actually learn a model from data automatically. The term is technically correct.
- ggm 3y agoThe "learning" part has always caused me grief. Always. Unfortunately it's beyond recovery. But "hallucination" and other new coined terms.. they definitely have problems.
- xcv123 3y agoI don't understand how that would cause grief. Seems totally irrational to me.
- ggm 3y agoThese words are applied to living people. Applied to machines they create states of belief outside the cognoscenti what is going on. It's as if we called programming "magic" and programmers "wizards" and then kids ask "is Harry potter real" and somebody says "yes" It's not hallucinating. It's an analogy. But analogies are not definitional. We're going to wind up in conversations about AGI and use of imprecise analogistic language will alter how people discuss this problem. Sorry you think I'm irratinal. I think you're being obtuse and not considering real world consequences of the ontologies used outside of the field.
- nercury 3y agoDon't "run" programs, don't let them "spawn" threads, don't allow CPU to "follow" instructions, don't "read" or "write" files, should I keep going? All these words used to mean humans doing things, we have been anthropomorphising the machines since the beginning. You are fighting a loosing battle here, better to just calm down and relax.
- 3y ago