4 ms·

I think thinking in terms of affordances and goals already makes a lot of assumptions about how a potential AGI would work, but imo the kind of "radical emergen

I think thinking in terms of affordances and goals already makes a lot of assumptions about how a potential AGI would work, but imo the kind of "radical emergence" talked about in the article is totally within the realm of possible applications of modern machine learning. If you think about how a language model works, it's not totally clear that it should work at all. Humans use sentences to communicate pre-existing ideas - ie. we already have the object, subject and action in our heads prior to forming the sentence. In order to generate realistic sentences, an LLM needs some ability to plan ahead, so that when generating the logits for the first token, it needs some idea of how the sentence will end. But if you think about it, nothing about next-token prediction should necessarily lead to this planning ability. It can easily get into a feedback loop of predict the same token over and over (and small LMs do in fact do this) We also didn't tell it explicitly that sentences should have objects, subjects and actions, these behaviors are purely emergent from the task of next token prediction. In a similar vein, I think artificial agents would not need explicit goals or affordances, just a sufficiently complex environment that a large model would not overfit on.

davidmanheim 4y ago
Yeah, and the idea that these programs can't have affordances is silly reliance on a definition that ignores what can happen. It clearly proves too much - humans have a limited action space - they can only move muscles. So they can't truly explore a larger action space, so they cannot be general intelligences. But more specifically, if something is within an LLM's action space, whether or not you call it an affordance doesn't change whether it gets explored and used. And perhaps you'd argue that their action space is limited because they are in a box. But hook an LLM up to the internet to allow it to query and retrieve data, and suddenly the action space is essentially infinite. So the limitation isn't the model, it's how the model was hobbled by being denied access to the world.

davidmanheim 4y ago

Yeah, and the idea that these programs can't have affordances is silly reliance on a definition that ignores what can happen. It clearly proves too much - humans have a limited action space - they can only move muscles. So they can't truly explore a larger action space, so they cannot be general intelligences. But more specifically, if something is within an LLM's action space, whether or not you call it an affordance doesn't change whether it gets explored and used. And perhaps you'd argue that their action space is limited because they are in a box. But hook an LLM up to the internet to allow it to query and retrieve data, and suddenly the action space is essentially infinite. So the limitation isn't the model, it's how the model was hobbled by being denied access to the world.