2 ms·
Do we really understand? What's the mechanism for that? My thinking is that we humans have some ability to do things by rote until we have a good statistical u
by sdenton4 6y ago
Do we really understand? What's the mechanism for that?
My thinking is that we humans have some ability to do things by rote until we have a good statistical understanding, at which point we use the statistical understanding mostly, and fall back to rote where we have to be careful. But, mostly, we're stats machines too.
- tsimionescu 6y agoWe obviously have a working model of the world that we draw facts from, that is partly built-in (trained by evolutionary processes) and partly refined with personal learning. By contrast, GPT-3 only has a knowledge of text. To the extent you could say that it thinks, it thinks in terms of textual symbols that it has seen before - that is its world. It is theoretically possible that if we found a way to encode experiences of the world of the world in a dataset and used the same training mechanisms and models that we used for GPT-3, we would get an AI with a decent model of the world. But it is magical thinking to imagine that we can get there by training a model on raw text.