3 ms·
> and have codified a huge body of knowledge which helps systems with producing outputs from input strings That is specifically what the programmers have not d
by ben_w 3y ago
> and have codified a huge body of knowledge which helps systems with producing outputs from input strings
That is specifically what the programmers have not done.
- famouswaffles 3y agoYeah. Maybe it's because we use the word "train" and that conjures up a mental model of some artisan distilling his process to an an apprentice ?
- ggm 3y agoThis artisan hasn't been trained in how to make steel. It's been trained in how to talk about steel making plausibly.
- famouswaffles 3y agoMy man you're completely off base here. No one is programming anything into neural networks and we have no idea how models like GPT-4 work at a high level. The whole point of neural networks is that we don't know shit and we don't know shit about how to distill the mechanics of some very real, very important processes so we figured out a way to make the machine figure this out on its own.
- ggm 3y agoYou're saying that the billions of lines of text have not informed a statistical model of relationships of words in grammer? You think the "body of knowledge" has to be in the meaning of the words? No. It's in the weighted maths of the likelihood of each element of the phrase. That statistical model is knowledge. Some semantic intent is there too, to disambiguate who flies like an arrow or flies like a banana.
- ben_w 3y ago> That statistical model is knowledge Glad you agree. None of the free parameters of these statistical models were selected by programmers.
- ggm 3y agoHmm. Is adjusting weights metaprogramming? It's based on observations of the outputs of the system. It's certainly not directed programming intent in the normal sense. It shares one property of traditional programming: garbage in garbage out. The fruits of their labour indirectly is a model. It's proving valuable. You want to ascribe the value to .. the bit nobody claims to understand? I'm sticking to the labour theory of value here, either the robot makers imbued value, or we have to redefine the value.
- deleted 3y ago[deleted]
- xcv123 3y agoHumans are not adjusting the weights. That is done automatically by the algorithm. The humans only provide training input.
- ggm 3y agoSo when people told the staff the models were racist you think they just "asked it to be less racist" I think they did a little more than just adjust the input data, they also modulated the logic behind the Weighting methods. No?
- rottc0dd 3y agoHow do babies learn languages? I know they understand the world and the meaning. But, process of handling the language itself, is not statistical at all? I am sure if the machine is not alive, but all live things has statistical inference part of their learning.