4 ms·
This is a great point. But I'd say that capable entities have a habit of turning themselves into agents. A great example is totalitarian governments. Even if
by duvenaud 3y ago
This is a great point. But I'd say that capable entities have a habit of turning themselves into agents. A great example is totalitarian governments. Even if every single citizen hates the regime, they're still forced to support it.
You could similarly ask: Why would we ever build a government or institution that cared more about its own self-preservation than its original mission? The answer is: Natural selection favors the self-interested, even if they don't have genes.
- enord 3y agoNow agency is an end unto itself I wholeheartedly agree. I feel though, that any worry about the agency of supercapable computer systems is premature until we see even the tiniest— and I mean really anything at all— sign of their agency. Heck, even agency _in theory_ would suffice, and yet: nada.
- duvenaud 3y agoI'm confused. You agree that we're surround by naturally-arising, self-organizing agents, both biological and institutional. People are constantly experimenting with agentic AIs of all kinds. There are tons of theoretical characterizations of agency and how it's a stable equilibrium. I'm not sure what you're hoping for if none of these are reasons to even worry.
- enord 3y agoNone we have made from unliving things no. «agentic» is a 5$ word of ill construction. The literature is littered with the corpses of failed definitions of «agency», «planning», «goal dorected behavior». ‘Twas the death of the Expert System AI (now it’s just constraint solvers). It will be the death of attention/transformer AI before long, I wonder what banality we will bestow upon it.
- duvenaud 3y agoOkay, well it seems funny to both claim that there is no good definition of agency, but also be sure that we've never demonstrated any hint of it in theory or practice. Planners, optimizers, and RL agents seem agentic to some degree to me, even if our current ones aren't very competent.
- enord 3y agoYou know, i thought of the right response just now, browsing my flamewars two weeks later. «Some degree» of agency is not even near sufficient identification of agency to synthesize it ex nihilo. There is no Axiom of Choice in real life, proof of existence is not proof of construction.