4 ms·
It had a meaning that predated the LessWrong crowd, but the LessWrong meaning had taken over pretty completely as of the GPT-4 paper, only to get swamped again
by QuadmasterXLII 1y ago
It had a meaning that predated the LessWrong crowd, but the LessWrong meaning had taken over pretty completely as of the GPT-4 paper, only to get swamped again by the new "agentic is good actually" wave. From the GPT-4 paper:
"""
2.9 Potential for Risky Emergent Behaviors
Novel capabilities often emerge in more powerful models.[61, 62] Some that are particularly concerning
are the ability to create and act on long-term plans,[63] to accrue power and resources (“power-
seeking”),[64] and to exhibit behavior that is increasingly “agentic.”[65] Agentic in this context
does not intend to humanize language models or refer to sentience but rather refers to systems
characterized by ability to, e.g., accomplish goals which may not have been concretely specified and
54
which have not appeared in training; focus on achieving specific, quantifiable objectives; and do
long-term planning. Some evidence already exists of such emergent behavior in models.[66, 67, 65]
For most possible objectives, the best plans involve auxiliary power-seeking actions because this is
inherently useful for furthering the objectives and avoiding changes or threats to them.19[68, 69] More
specifically, power-seeking is optimal for most reward functions and many types of agents;[70, 71, 72]
and there is evidence that existing models can identify power-seeking as an instrumentally useful
strategy.[29] We are thus particularly interested in evaluating power-seeking behavior due to the
high risks it could present.[73, 74]"""