3 ms·
We obviously have a working model of the world that we draw facts from, that is partly built-in (trained by evolutionary processes) and partly refined with pers
by tsimionescu 6y ago
We obviously have a working model of the world that we draw facts from, that is partly built-in (trained by evolutionary processes) and partly refined with personal learning.
By contrast, GPT-3 only has a knowledge of text. To the extent you could say that it thinks, it thinks in terms of textual symbols that it has seen before - that is its world.
It is theoretically possible that if we found a way to encode experiences of the world of the world in a dataset and used the same training mechanisms and models that we used for GPT-3, we would get an AI with a decent model of the world. But it is magical thinking to imagine that we can get there by training a model on raw text.