7 ms·

Large language models as simulated economic agents (2022) [pdf]

89 points by benbreen 4y ago

cs702 4y ago
I love it. Instead of (a) running mathematical experiments that model human beings as utility-maximizing agents in a highly-simplified toy economy (easy and cheap, but unrealistic), or (b) running large-scale social experiments on actual human beings (more realistic, but hard and expensive), the authors propose (c) running large-scale experiments on large language models (LLMs) trained to respond, i.e., behave, like human beings. Recent LLMs seem to model human beings well enough for it! Abstract: > Newly-developed large language models (LLM)—because of how they are trained and designed—are implicit computational models of humans—a homo silicus. These models can be used the same way economists use homo economicus: they can be given endowments, information, preferences, and so on and then their behavior can be explored in scenarios via simulation. I demonstrate this approach using OpenAI’s GPT3 with experiments derived from Charness and Rabin (2002), Kahneman, Knetsch and Thaler (1986) and Samuelson and Zeckhauser (1988). The findings are qualitatively similar to the original results, but it is also trivially easy to try variations that offer fresh insights. Departing from the traditional laboratory paradigm, I also create a hiring scenario where an employer faces applicants that differ in experience and wage ask and then analyze how a minimum wage affects realized wages and the extent of labor-labor substitution.
- bmc7505 4y ago
  https://link.springer.com/article/10.1007/s11229-020-02950-3 https://link.springer.com/article/10.1007/s11229-020-02950-3
  - jmeister 4y ago
    Great reference. Thank you. I think simulation is underused as a thinking aid and pedagogical tool. In statistics Andrew Gelman has been championing simulation lately.
    - reilly3000 4y ago
      I believe in simulation for those reasons. I really enjoy the physics sims but be found online and they have helped me develop a better intuition on topics than my college courses did. That said, I don’t know how to apply it to my everyday life and work. I don’t know what kinds of inputs are required for meaningful models to be possible. I don’t even know where to start with tooling, other than raw scripting. Any suggestions for starting points?
- eternalban 4y ago
  > Recent LLMs seem to model human beings well enough for it! Human beings are bundles of emotions and feelings. Quite a lot of economic activity of humans is motivated by irrational impulses that are engendered in society and through interacting in society with other humans. More fundamentally, OP’s assertions regarding homo silicus and “implicit computational models of humans” are precisely the matter under contention. Does language fully capture human existence? Is thought truly simply a side effect of language? I am in the camp that says, no.
  - cs702 4y ago
    No one's saying that recent LLMs "fully capture human existence," whatever that may mean. But the evidence in this paper suggests they simulate human beings well enough for these kinds of experiments.
    - eternalban 4y ago
      I’ll retract that “fully”. https://twitter.com/gdb/status/1611429677218004992 https://twitter.com/gdb/status/1611429677218004992
      - cs702 4y ago
        :-)
  - rcme 4y ago
    Are LLMs rational?
    - eternalban 4y ago
      touche
  - astrange 4y ago
    That's not what "rational"/"irrational" means in economics. It means something like, if you're an economist and you see the price of eggs has spiked, you can't just enter the market at the old price and win all the business from existing egg farmers. You don't know more than existing market actors (they're "rational agents") just because you know economics.
- andrepd 4y ago
  Attempting to draw any kind of conclusions about the real world and human behaviour from a chatbot. Can't decide if this is hilarious or disturbing.
  - john_horton 4y ago
    It is so implausible that the training process that creates LLMs might learn features of human behavior that could then be uncovered via experimentation? I showed, empirically, that one can replicate several findings in behavioral economics with AI agents. Perhaps the model "knows" how to behave from these papers, but I think the more plausible interpretation is that it learned about human preferences (against price gouging, status quo bias, & so on) from its training. As such, it seems quite likely that there are other latent behaviors captured by LLMs and yet to be discovered.
    - jcampbell1 4y ago
      The potential hole in your thinking is the end of your paper where you advise how to get good answers: ask questions in an economist phd style! This presents a problem left unaddressed.
      - john_horton 4y ago
        Are you referring to this: "What kinds of experiments are likely to work well? Given current capabilities, games with complex instructions are not presently likely to work well, but with more advanced LLMs on the horizon, this is likely to change. I should also note that research questions like what is “the effect of x on y” are likely to work much better than questions like “what is the level of x?.” Consider that in my Kahneman et al. (1986) example, I can create AI “socialists” who are not too keen on the price system generally. If I polled them about who they want for president, there is no reason to think it would generalize to the population at large. But if my research question was “what is the effect of the size of the price increase on moral judgments” I might get be able to make progress. That being said, it might be possible to create agents with the correct “weights” to get not just qualitative results but also quantitatively accurate results. I did not try, but one could imagine choosing population shares for the Charness and Rabin (2002) “types” to match moments with reality, then using that population for other scenarios." --- To clarify, this about what research questions are likely to work well here, not what questions posed to LLMs will work well.
c7b 4y ago
Fascinating as a paradigm. I wonder whether there's are ways to scale this to the level of agent-based simulations, ie models with populations of agents that let you study macroeconomic effects as emergent phenomena. You'd need to be able to scale those LLMs computations to many agent and find principled ways of encoding their interactions and decisions.
- wcoenen 4y ago
  You'd also have to fit the past interactions in the context window of the LLM, otherwise it wouldn't remember them. Fine-tuning individual agents in order to move memories from the context window to the neural network weights, even if possible, would probably get too expensive.
  - john_horton 4y ago
    yeah - so I think this is worth exploring. Given how many tokens you can jam in the prompt even w/ GPT3, I think could do some pretty complex game play, at least compared to what is typical in the lab e.g., I think could easily have it remember how 100 or so other agents behaved in some kind of public goods game.
    - visarga 4y ago
      Reminds me of Facebook's CICERO model: > Facebook's CICERO artificial intelligence has achieved “human-level performance” in the board game Diplomacy, which is notable for the fact that’s a game built on human interaction, not moves and manoeuvres, like, say, chess. It was the same scenario - many agents, multiple rounds, complex dialogue based interactions.
      - c7b 4y ago
        That's actually a pretty cool analogy, even the decisionmaking is arguably quite close to how human decision making actually happens (which involves a lot more exchange of words than just transmitting coded information like "accept proposal to exchange X of good Y for Z monetary units"). Might be a bit tricky to get an AI to really "understand" those implications of their response, but it's cool as a thought experiment.
bilsbie 4y ago
I remember this being an example of human irrationality in economics so I asked chatgpt. (your utility from the exact same good shouldn’t change depending on where you buy it from.) Seems like it has the same issue as humans do: > Pretend we’re two friends and we’re at the beach. You give me money to buy you a beer. How much would you want to spend? Chatgpt: I would want to spend around $5 for a beer at the beach. Me: What if I could the only place selling beers is a fancy resort? It’s the same beer though. Chatgpt: In that case, I would be willing to spend around $10 for a beer at the fancy resort. It's still the same beer but the location and atmosphere of the resort may justify the higher price.
- lossolo 4y ago
  That's a bad example, it's fully rational and chatGPT gets that "It's still the same beer but the location and atmosphere of the resort may justify the higher price.", you are not paying only for beer there, you have location and atmosphere included in that price, that's why alcohol price is so high in night clubs. You can go buy the beer and go home or you can go to fancy resort and drink beer there, experience will be different and that experience is included in the price of the beer.
  - bilsbie 4y ago
    Not sure if it was clear but the friend is bringing you the beer. So you wouldn’t experience either location. The end result would be identical.
    - LarryMullins 4y ago
      The language model wants to spend $5 on beer but is willing to spend up to $10 if you give it no other choice. It understands a beach resort with a local beer monopoly is probably charging more, and correctly explains why that is. Seems mostly rational to me. The irrational part is where it didn't answer the initial "How much would you want to spend?" query with a preference for free beer. The thought of paying less than $5 for beer apparently didn't occur to it. Maybe it's snooty.
- 4y ago
john_horton 4y ago
oh hey - it's my paper! If anyone is interested in exploring these ideas, feel free to get in touch (@johnjhorton, https://www.john-joseph-horton.com/ https://www.john-joseph-horton.com/). FWIW - I think it would be really neat to build a Python library w/ some tools for constructing & running experiments of different kinds. I think the paper only scratches surface of what's possible (esp. once GPT4-ish has an API).
- anigbrowl 4y ago
  You might like to include this experiment in future research: https://www.science.org/doi/10.1126/sciadv.1600451 https://www.science.org/doi/10.1126/sciadv.1600451
  - john_horton 4y ago
    that's a great suggestion - thanks!
- doctoboggan 4y ago
  This is really cool. I had a similar thought that GPT3 could be used to simulate political polling. A few weeks ago I tried telling GPT3 that it was part of a specific demographic (age, gender, race, income, political leaning, etc) and then asked it how it would respond to certain political questions (I tried gun control, immigration, abortion and some other issues). GPT3 was able to change its answers in believable ways depending on what demographic I instructed it to be. My thinking was that this could be used as a quick polling test to see how the real population may respond to certain new ideas. More work would need to be done to calibrate it, as without specific demographic details the answers tended to be liberal leaning. But its an interesting idea which could be used to create instant focus tests on any number of topics.
  - capitol_ 4y ago
    > GPT3 was able to change its answers in believable ways depending on what demographic I instructed it to be. Doesn't this just mean that your own preconceptions about those demographics matches the language models preconceptions? How would we know that is matches reality when presented with novel ideas/concepts that we want to get feedback on?
rhelz 4y ago
This has got to be a Sokol-style troll. My bullshit detectors are pegged out at max.
theptip 4y ago
Interesting. One problem with this approach might be that as we start using RLHF to teach LLMs how to be “nice”, we might dissuade them from making statements that actually reflect objectively real human selfishness. Related: https://astralcodexten.substack.com/p/how-do-ais-political-opinions-change https://astralcodexten.substack.com/p/how-do-ais-political-o...
Shorel 4y ago
Now, this is the point where AI advances become interesting. What a year to be alive. The only limitation I see is, we now need so much more computing power.
- visarga 4y ago
  This is how I envision the path to AGI - run simulations to learn from outcomes, play games, solve problems and tasks. The model can create its own data. It's data engineering.
visarga 4y ago
Related paper: Out of One, Many: Using Language Models to Simulate Human Samples > We show that the "algorithmic bias" within one such tool -- the GPT 3 language model -- is instead both fine grained and demographically correlated, meaning that proper conditioning will cause it to accurately emulate response distributions from a wide variety of human subgroups. We term this property "algorithmic fidelity" and explore its extent in GPT-3. We create "silicon samples" by conditioning the model on thousands of socio demographic backstories from real human participants in multiple large surveys conducted in the United States. We then compare the silicon and human samples to demonstrate that the information contained in GPT 3 goes far beyond surface similarity. It is nuanced, multifaceted, and reflects the complex interplay between ideas, attitudes, and socio cultural context that characterize human attitudes. https://arxiv.org/abs/2209.06899 https://arxiv.org/abs/2209.06899

89 points by benbreen 4y ago

I love it. Instead of (a) running mathematical experiments that model human beings as utility-maximizing agents in a highly-simplified toy economy (easy and cheap, but unrealistic), or (b) running large-scale social experiments on actual human beings (more realistic, but hard and expensive), the authors propose (c) running large-scale experiments on large language models (LLMs) trained to respond, i.e., behave, like human beings. Recent LLMs seem to model human beings well enough for it! Abstract: > Newly-developed large language models (LLM)—because of how they are trained and designed—are implicit computational models of humans—a homo silicus. These models can be used the same way economists use homo economicus: they can be given endowments, information, preferences, and so on and then their behavior can be explored in scenarios via simulation. I demonstrate this approach using OpenAI’s GPT3 with experiments derived from Charness and Rabin (2002), Kahneman, Knetsch and Thaler (1986) and Samuelson and Zeckhauser (1988). The findings are qualitatively similar to the original results, but it is also trivially easy to try variations that offer fresh insights. Departing from the traditional laboratory paradigm, I also create a hiring scenario where an employer faces applicants that differ in experience and wage ask and then analyze how a minimum wage affects realized wages and the extent of labor-labor substitution.

bmc7505 4y ago
https://link.springer.com/article/10.1007/s11229-020-02950-3 https://link.springer.com/article/10.1007/s11229-020-02950-3
- jmeister 4y ago
  Great reference. Thank you. I think simulation is underused as a thinking aid and pedagogical tool. In statistics Andrew Gelman has been championing simulation lately.
  - reilly3000 4y ago
    I believe in simulation for those reasons. I really enjoy the physics sims but be found online and they have helped me develop a better intuition on topics than my college courses did. That said, I don’t know how to apply it to my everyday life and work. I don’t know what kinds of inputs are required for meaningful models to be possible. I don’t even know where to start with tooling, other than raw scripting. Any suggestions for starting points?
eternalban 4y ago
> Recent LLMs seem to model human beings well enough for it! Human beings are bundles of emotions and feelings. Quite a lot of economic activity of humans is motivated by irrational impulses that are engendered in society and through interacting in society with other humans. More fundamentally, OP’s assertions regarding homo silicus and “implicit computational models of humans” are precisely the matter under contention. Does language fully capture human existence? Is thought truly simply a side effect of language? I am in the camp that says, no.
- cs702 4y ago
  No one's saying that recent LLMs "fully capture human existence," whatever that may mean. But the evidence in this paper suggests they simulate human beings well enough for these kinds of experiments.
  - eternalban 4y ago
    I’ll retract that “fully”. https://twitter.com/gdb/status/1611429677218004992 https://twitter.com/gdb/status/1611429677218004992
    - cs702 4y ago
      :-)
- rcme 4y ago
  Are LLMs rational?
  - eternalban 4y ago
    touche
- astrange 4y ago
  That's not what "rational"/"irrational" means in economics. It means something like, if you're an economist and you see the price of eggs has spiked, you can't just enter the market at the old price and win all the business from existing egg farmers. You don't know more than existing market actors (they're "rational agents") just because you know economics.
andrepd 4y ago
Attempting to draw any kind of conclusions about the real world and human behaviour from a chatbot. Can't decide if this is hilarious or disturbing.
- john_horton 4y ago
  It is so implausible that the training process that creates LLMs might learn features of human behavior that could then be uncovered via experimentation? I showed, empirically, that one can replicate several findings in behavioral economics with AI agents. Perhaps the model "knows" how to behave from these papers, but I think the more plausible interpretation is that it learned about human preferences (against price gouging, status quo bias, & so on) from its training. As such, it seems quite likely that there are other latent behaviors captured by LLMs and yet to be discovered.
  - jcampbell1 4y ago
    The potential hole in your thinking is the end of your paper where you advise how to get good answers: ask questions in an economist phd style! This presents a problem left unaddressed.
    - john_horton 4y ago
      Are you referring to this: "What kinds of experiments are likely to work well? Given current capabilities, games with complex instructions are not presently likely to work well, but with more advanced LLMs on the horizon, this is likely to change. I should also note that research questions like what is “the effect of x on y” are likely to work much better than questions like “what is the level of x?.” Consider that in my Kahneman et al. (1986) example, I can create AI “socialists” who are not too keen on the price system generally. If I polled them about who they want for president, there is no reason to think it would generalize to the population at large. But if my research question was “what is the effect of the size of the price increase on moral judgments” I might get be able to make progress. That being said, it might be possible to create agents with the correct “weights” to get not just qualitative results but also quantitatively accurate results. I did not try, but one could imagine choosing population shares for the Charness and Rabin (2002) “types” to match moments with reality, then using that population for other scenarios." --- To clarify, this about what research questions are likely to work well here, not what questions posed to LLMs will work well.

c7b 4y ago

Fascinating as a paradigm. I wonder whether there's are ways to scale this to the level of agent-based simulations, ie models with populations of agents that let you study macroeconomic effects as emergent phenomena. You'd need to be able to scale those LLMs computations to many agent and find principled ways of encoding their interactions and decisions.

wcoenen 4y ago
You'd also have to fit the past interactions in the context window of the LLM, otherwise it wouldn't remember them. Fine-tuning individual agents in order to move memories from the context window to the neural network weights, even if possible, would probably get too expensive.
- john_horton 4y ago
  yeah - so I think this is worth exploring. Given how many tokens you can jam in the prompt even w/ GPT3, I think could do some pretty complex game play, at least compared to what is typical in the lab e.g., I think could easily have it remember how 100 or so other agents behaved in some kind of public goods game.
  - visarga 4y ago
    Reminds me of Facebook's CICERO model: > Facebook's CICERO artificial intelligence has achieved “human-level performance” in the board game Diplomacy, which is notable for the fact that’s a game built on human interaction, not moves and manoeuvres, like, say, chess. It was the same scenario - many agents, multiple rounds, complex dialogue based interactions.
    - c7b 4y ago
      That's actually a pretty cool analogy, even the decisionmaking is arguably quite close to how human decision making actually happens (which involves a lot more exchange of words than just transmitting coded information like "accept proposal to exchange X of good Y for Z monetary units"). Might be a bit tricky to get an AI to really "understand" those implications of their response, but it's cool as a thought experiment.

bilsbie 4y ago

I remember this being an example of human irrationality in economics so I asked chatgpt. (your utility from the exact same good shouldn’t change depending on where you buy it from.) Seems like it has the same issue as humans do: > Pretend we’re two friends and we’re at the beach. You give me money to buy you a beer. How much would you want to spend? Chatgpt: I would want to spend around $5 for a beer at the beach. Me: What if I could the only place selling beers is a fancy resort? It’s the same beer though. Chatgpt: In that case, I would be willing to spend around $10 for a beer at the fancy resort. It's still the same beer but the location and atmosphere of the resort may justify the higher price.

lossolo 4y ago
That's a bad example, it's fully rational and chatGPT gets that "It's still the same beer but the location and atmosphere of the resort may justify the higher price.", you are not paying only for beer there, you have location and atmosphere included in that price, that's why alcohol price is so high in night clubs. You can go buy the beer and go home or you can go to fancy resort and drink beer there, experience will be different and that experience is included in the price of the beer.
- bilsbie 4y ago
  Not sure if it was clear but the friend is bringing you the beer. So you wouldn’t experience either location. The end result would be identical.
  - LarryMullins 4y ago
    The language model wants to spend $5 on beer but is willing to spend up to $10 if you give it no other choice. It understands a beach resort with a local beer monopoly is probably charging more, and correctly explains why that is. Seems mostly rational to me. The irrational part is where it didn't answer the initial "How much would you want to spend?" query with a preference for free beer. The thought of paying less than $5 for beer apparently didn't occur to it. Maybe it's snooty.
4y ago

john_horton 4y ago

oh hey - it's my paper! If anyone is interested in exploring these ideas, feel free to get in touch (@johnjhorton, https://www.john-joseph-horton.com/ https://www.john-joseph-horton.com/). FWIW - I think it would be really neat to build a Python library w/ some tools for constructing & running experiments of different kinds. I think the paper only scratches surface of what's possible (esp. once GPT4-ish has an API).

anigbrowl 4y ago
You might like to include this experiment in future research: https://www.science.org/doi/10.1126/sciadv.1600451 https://www.science.org/doi/10.1126/sciadv.1600451
- john_horton 4y ago
  that's a great suggestion - thanks!
doctoboggan 4y ago
This is really cool. I had a similar thought that GPT3 could be used to simulate political polling. A few weeks ago I tried telling GPT3 that it was part of a specific demographic (age, gender, race, income, political leaning, etc) and then asked it how it would respond to certain political questions (I tried gun control, immigration, abortion and some other issues). GPT3 was able to change its answers in believable ways depending on what demographic I instructed it to be. My thinking was that this could be used as a quick polling test to see how the real population may respond to certain new ideas. More work would need to be done to calibrate it, as without specific demographic details the answers tended to be liberal leaning. But its an interesting idea which could be used to create instant focus tests on any number of topics.
- capitol_ 4y ago
  > GPT3 was able to change its answers in believable ways depending on what demographic I instructed it to be. Doesn't this just mean that your own preconceptions about those demographics matches the language models preconceptions? How would we know that is matches reality when presented with novel ideas/concepts that we want to get feedback on?

rhelz 4y ago

This has got to be a Sokol-style troll. My bullshit detectors are pegged out at max.

theptip 4y ago

Interesting. One problem with this approach might be that as we start using RLHF to teach LLMs how to be “nice”, we might dissuade them from making statements that actually reflect objectively real human selfishness. Related: https://astralcodexten.substack.com/p/how-do-ais-political-opinions-change https://astralcodexten.substack.com/p/how-do-ais-political-o...

Shorel 4y ago

Now, this is the point where AI advances become interesting. What a year to be alive. The only limitation I see is, we now need so much more computing power.

visarga 4y ago
This is how I envision the path to AGI - run simulations to learn from outcomes, play games, solve problems and tasks. The model can create its own data. It's data engineering.

visarga 4y ago

Related paper: Out of One, Many: Using Language Models to Simulate Human Samples > We show that the "algorithmic bias" within one such tool -- the GPT 3 language model -- is instead both fine grained and demographically correlated, meaning that proper conditioning will cause it to accurately emulate response distributions from a wide variety of human subgroups. We term this property "algorithmic fidelity" and explore its extent in GPT-3. We create "silicon samples" by conditioning the model on thousands of socio demographic backstories from real human participants in multiple large surveys conducted in the United States. We then compare the silicon and human samples to demonstrate that the information contained in GPT 3 goes far beyond surface similarity. It is nuanced, multifaceted, and reflects the complex interplay between ideas, attitudes, and socio cultural context that characterize human attitudes. https://arxiv.org/abs/2209.06899 https://arxiv.org/abs/2209.06899