4 ms·

I was confused why you would say LLM's have agency, but I see there's something like dozens of potential definitions of agency, much like there's probably dozen

I was confused why you would say LLM's have agency, but I see there's something like dozens of potential definitions of agency, much like there's probably dozens of potential definitions of AGI.

barfbagginus 3y ago
Here's the most simple definition of agency: Agency is the ability to independently make and execute decisions in an environment. This typically involves three components: 1. Perception: The ability to observe the environment 2. Cognition: the ability to process perceptions and arrive at decisions 3. Action: the ability to act on the environment LLMs can process many kinds of perceptions and make decisions about them - often as well as a human. That means that giving agency to an LLM is often just a matter of giving it an environment it can perceive and act on. There are of course different levels of agency. A General Intelligence is a higher level agent that can handle many different environments and actions. LLMs have two main challenges here. There are environments they can't connect to. And many environments require complex reasoning that LLMs can't do on their own. There are two low hanging approaches here; 1. Adaptors. These are perception and action adaptors that can connect LLMs to many environments. Work here includes multimodal llms, LLM driven robots, single LLM agents, and communities of LLM agents 2. Accelerators: These help LLMs reason about complex environments like math, code, optimal planning, and knowledge bases. Work here includes tool generating LLMs (Toolformer, LLM+P), RAG engines, LLM-driven mixtures of experts etc. Conclusion: Agency is easy Overall LLMs have the cognitive horsepower to be agents in many environments. All they need is efficient and general adaptors and tools. We can modify LLMs to better work with these components, like multimodal LLMs or mixtures of experts. Or we can plug and play different adapters and tools to increase the level of agency, like tool use and RAG. Future generations of AI will likely integrate these features in much more effective ways, achieving higher levels of agency. But even as it stands, LLMs have very real agent capabilities, needing only tools and adaptors.
- staticman2 3y ago
  I'd say that's a pretty useless definition of agency, it's not clear why a thermostat wouldn't meet that low bar, and with that overly generous definition we now we need a new term "super agency" or whatever to talk about what other people mean by agency.
  - barfbagginus 3y ago
    What is your ideal definition of agency? And would you say that definition is helpful for people trying to increase the agency level of AIs? What makes it so useful?
    - staticman2 3y ago
      I doubt arguing about whether LLM are "AGI" or have "agency" is useful to developers at all. But I also don't understand what the usefulness is of claiming architectures that are entirely passive- only speaking when spoken to, have "agency." If you want a definition, let's start with a neural network that runs ongoingly, does some sort of goal driven, useful behavior, and doesn't merely respond to prompts. Here's a different more sober take on current models: https://twitter.com/ylecun/status/1718263303485501784?s=19 https://twitter.com/ylecun/status/1718263303485501784?s=19

I was confused why you would say LLM's have agency, but I see there's something like dozens of potential definitions of agency, much like there's probably dozens of potential definitions of AGI.

barfbagginus 3y ago

Here's the most simple definition of agency: Agency is the ability to independently make and execute decisions in an environment. This typically involves three components: 1. Perception: The ability to observe the environment 2. Cognition: the ability to process perceptions and arrive at decisions 3. Action: the ability to act on the environment LLMs can process many kinds of perceptions and make decisions about them - often as well as a human. That means that giving agency to an LLM is often just a matter of giving it an environment it can perceive and act on. There are of course different levels of agency. A General Intelligence is a higher level agent that can handle many different environments and actions. LLMs have two main challenges here. There are environments they can't connect to. And many environments require complex reasoning that LLMs can't do on their own. There are two low hanging approaches here; 1. Adaptors. These are perception and action adaptors that can connect LLMs to many environments. Work here includes multimodal llms, LLM driven robots, single LLM agents, and communities of LLM agents 2. Accelerators: These help LLMs reason about complex environments like math, code, optimal planning, and knowledge bases. Work here includes tool generating LLMs (Toolformer, LLM+P), RAG engines, LLM-driven mixtures of experts etc. Conclusion: Agency is easy Overall LLMs have the cognitive horsepower to be agents in many environments. All they need is efficient and general adaptors and tools. We can modify LLMs to better work with these components, like multimodal LLMs or mixtures of experts. Or we can plug and play different adapters and tools to increase the level of agency, like tool use and RAG. Future generations of AI will likely integrate these features in much more effective ways, achieving higher levels of agency. But even as it stands, LLMs have very real agent capabilities, needing only tools and adaptors.

staticman2 3y ago
I'd say that's a pretty useless definition of agency, it's not clear why a thermostat wouldn't meet that low bar, and with that overly generous definition we now we need a new term "super agency" or whatever to talk about what other people mean by agency.
- barfbagginus 3y ago
  What is your ideal definition of agency? And would you say that definition is helpful for people trying to increase the agency level of AIs? What makes it so useful?
  - staticman2 3y ago
    I doubt arguing about whether LLM are "AGI" or have "agency" is useful to developers at all. But I also don't understand what the usefulness is of claiming architectures that are entirely passive- only speaking when spoken to, have "agency." If you want a definition, let's start with a neural network that runs ongoingly, does some sort of goal driven, useful behavior, and doesn't merely respond to prompts. Here's a different more sober take on current models: https://twitter.com/ylecun/status/1718263303485501784?s=19 https://twitter.com/ylecun/status/1718263303485501784?s=19