3 ms·
> I attribute the 'magic' to large scale statistical knowledge assimilation by the models Can the magic of the human brain not also be attributed to "large sca
by zarzavat 4y ago
> I attribute the 'magic' to large scale statistical knowledge assimilation by the models
Can the magic of the human brain not also be attributed to "large scale statistical knowledge assimilation" as well, aka learning?
> GPT-3 is known to fail in many circumstances which would otherwise be commonplace logic. (I remember seeing how addition of two small numbers yielded results - but larger numbers gave garbled output; more likely that GPT3 had seen similar training data.)
This is a bug, they did not encode digits properly. They should have encoded each digit as a separate token but instead they encoded them together. Later models fixed this.
The human brain is full of bugs too, e.g. optical illusions. https://en.m.wikipedia.org/wiki/Optical_illusion https://en.m.wikipedia.org/wiki/Optical_illusion
> It's a fad
No, it's objectively not a fad. The PaLM paper shows that Google's model exceeds average human performance on >50% of language tasks. The set of things that make us us is vanishing at an alarming rate. Eventually it will be empty, or close to it.
Do I think Google's models are sentient? No, they lack several necessary ingredients of sentience such as a self and long-term memory. However we are clearly on the road to sentient AI and it pays to have that discussion now.
- wlorenz65 4y ago> Google's models are sentient? No, they lack several necessary ingredients of sentience such as a self They haven't been finetuned on their identity long enough because finetuning is expensive and Google lacks money ;-)
- achenet 4y ago> The set of things that make us us is vanishing at an alarming rate. Eventually it will be empty, or close to it. Computers can speak, but can they love? Can they care? Can they dance?
- mjburgess 4y ago>"large scale statistical knowledge assimilation" as well, aka learning No, experimentation is an act on the world to set its state and then measure it. That's what learning involves. These machines do not act on the world, they just capture correlations. In this sense, machines are maximally schizophrenic. They answer "yes" to "is there a cat on the matt?" not because there is one, but because "yes" was what they heard most often. Producing models of correlations in half-baked measures of human activity has nothing to do with learning. And everything to do with a magic light box that fools dumb apes.
- heavyset_go 4y agoWell said.
- TheOtherHobbes 4y agoI don't see how this is different to a computer which exceeds average human performance on math tasks - which they all do, from an 8-bit micro upwards. Being able to do arithmetic at some insane factor faster than humans isn't evidence of sentience. It's evidence of a narrow-purpose symbol processor which works very quickly. Working with more complex symbols - statistical representations of "language" - doesn't change that. The set of things that makes us us is not primarily intellectual, and it's a fallacy to assume it is. The core bedrock of human experience is built from individual motivation, complex social awareness and relationship building, emotional expression and empathy, awareness of body language and gesture, instinct, and ultimately from embodied sensation. It's not about chess or go. Or language. And it's not obviously statistical.
- Touche 4y agoAre animals not sentient? They lack some qualities you are ascribing.
- jobs_throwaway 4y ago> Google's model exceeds average human performance on >50% of language tasks I guess I'm just not interested, or worried, in a model that can beat the average human performance. That's an astoundingly low bar. Let me know when it can outperform experts in meaningful language tasks.
- jstanley 4y agoThis is a great example of the "moving goalposts" of AI. 100 years ago, a comment like yours would have sounded absurd. There was a time when we didn't have any models that could beat any human's performance in any task.