6 ms·

> In the simulation, four AI workers are each controlled by their own reinforcement-learning models. They interact with a two-dimensional world, gathering wood

> In the simulation, four AI workers are each controlled by their own reinforcement-learning models. They interact with a two-dimensional world, gathering wood and stone and either trading these resources with others or using them to build houses, which earns them money. The workers have different levels of skill, which leads them to specialize. Lower-skilled workers learn they do better if they gather resources, and higher-skilled ones learn they do better if they buy resources to build houses. So basically they're tackling one of the hardest macroeconomic problems with a simple and completely abstract microeconomic model which sounds less like cutting edge AI and more like a fusion of iterated prisoner dilemma models and the Edgeworth box. > the AI’s policy cobbled together aspects of both, applying the highest tax rates to rich and poor Sounds like they've overfitted to the model's assumptions so much their agents are choosing to not be poor primarily to reduce their tax bill...

yowlingcat 6y ago
How did I know that this would be the result from the headline? It's just like the old saying. Garbage in, garbage out.
- rrobukef 6y ago
  An important distinction: publishable garbage! Luckily, there is now a baseline, the next person can only publish improvements. Maybe one day we reach Hari Seldon levels of AI.
Cthulhu_ 6y ago
Yeah, it sounds more like they're training an AI for resource management games (like idk, Dwarf Fortress and co?)
Serious_Cheese 6y ago
The headline is misleading (click-bait). Reading the headline, I imagined they have simulated the entire economy of the world and have used reinforcement learning to find optimal policies, but they only simulated 4 workers and are making generalization to the entire economy.
im3w1l 6y ago
An issue I see here is that while punishing the poor is clever way of reducing poverty, a monetary punishment risks trapping them in a negative spiral. Beating them up (but not so much they can't work) or even killing them might work better. I hope people using AI for public policy are careful what they wish for.
- thedudeabides5 6y ago
  Agree with these comments. Like maybe the "AI" found some "Hand of God" move for the economy.... Like many solutions that AIs come up with—such as some of AlphaZero’s game-winning moves—the result appears counterintuitive and not something that a human might have devised. ...or maybe their model doesn't include the fact that when you raise taxes on the poor too high, in eras of tremendous inequality, you get bread riots, political violence, and eventually, revolutions. Not to say this kind of modeling isn't potentially useful! Just modern monetary policy already uses algorithms etc (Taylor rule optimization etc), so the question is how can ML/AI supplement those humans with better tools, rather than thinking a computer is going to be able to deal with the complexity of the real world. Put another way, what does the machine do when interest rates hit 0?
  - neffy 6y ago
    The so called A.I. was not working with anything that resembled a real economy. Standard economic models also have this problem, they also have other problems, but this particular example is just doubling down and then some. What is particularly unfortunate is that there are people out there trying to create real agent based models based on actual simulations of the Economy that are far more advanced than this (realistic markets, accurate simulations of the Banking System) but are not getting any of this kind of exposure. Salesforce obviously is backing its own product, but the MIT Technology Review should really know better.
- jbay808 6y ago
  If the AI workers can't get sick, die, starve, suffer reduced productivity from chronic health issues, or care for children, then I don't see what lessons they can really expect to learn from this model. If their poor AI workers can just keep working hard gathering wood and stone forever, being fully productive without ever eating any food so they can afford to pay their taxes, then it's disconnected from reality beyond anything I'd even consider a "toy model".
monadic2 6y ago
TBH I'd trust a describable model much more than "cutting edge AI"
- notahacker 6y ago
  Yep. Looking into their model in detail, whilst it has all the simplifications of a typical economic model designed for tractability and algebraic solutions (two good economy, four agents, tax officials have perfect information) , it also adds plenty of unusual assumptions like agents moving round a grid to find resources. Which as others point out might tell us a lot about how AI can optimise strategy games (more precisely that AI optimises for values in a strategy game better than simply using the Saez formula or US federal tax brackets), but very little about the relationship between debt, taxes and investment.
- tharne 6y ago
  I would trust a monkey throwing darts at the stock page over "cutting edge AI".
clavalle 6y ago
Whereas, in the real world, the poor might not get the chance to not be poor because destitution would cause their death. I'd be interested in playing with such a simulation where the agents must buy or make food to eat and must have shelter or suffer. And, also, that the other agents know this and can play their own hand accordingly. I think this would change where the redistribution scheme would land quite a bit if the idea is to maximize value creation.
Gimpei 6y ago
And yet, as simple as this is, four agents is more than you see in most "complex" DSGE models... This looks to me like they're applying deep RL to agent-based modeling. It does seem pretty silly now, but it could be promising. You don't have to restrict these agents to any neoclassical assumptions and there isn't any reason you couldn't introduce firms as well. The only limit is computational power. Macroeconomics tends to look down on agent-based modeling because it doesn't provide neat, closed form solutions. But I've never bought that the results from DSGE models are anything more than products of all their simplifying assumptions. After all, they seem to be the very definition of overfitedness with absolutely no predictive power. At least they're trying something new here.
- QuesnayJr 6y ago
  DSGE models are designed be solved on your laptop in 0.0001 seconds. There are more complicated models, like Krussell-Smith, but it's not clear that they help explain macroeconomics better. Agent-based modelling isn't more popular because they assume that the agents are very, very stupid. Also, not only do they not produce closed-form solutions, they don't tend to lead to clear statements at all. Also also, the leading agent-based modeler, Cars Hommes, is infinitely better published than most economists, so he's not suffering too bad.
Jabbles 6y ago
It reminds me of the covid models, particularly the infamous Imperial model. https://en.wikipedia.org/wiki/CovidSim https://en.wikipedia.org/wiki/CovidSim The question is whether it's useful to have a model at all, or whether it's better to just reason about it from first principles. If a basic simulation gave you a tool to explore emergent properties, perhaps some theories could be generalised to apply to real economies? Or maybe not, I don't know.
- whatshisface 6y ago
  Models are supposed to be based on first-principles reasoning. A "model" that's not designed to capture some aspect of reality is called a computer game...
- SiempreViernes 6y ago
  The Imperial model gives reproducible results, so most of the noise was professional programmers being salty about the fact that ugly code can still be accurate code. The fact that it predicted things like 80% of the population becoming infected was simply unrelated to code quality.
  - benmller313 6y ago
    Regardless of code quality, are these models really useful if, as you say, they are "accurate", but still make predictions wildly out of step with reality?
    - whatshisface 6y ago
      The real imperial story is that there were two wrong parties: - The scientists who were not including enough detail in their math. - The drive-by developers who had no idea what did or didn't make scientific simulations work. Together they produced both an inaccurate estimate, and a github page full of uninformed speculation about what the problem might have been. ;)
      - SiempreViernes 6y ago

yowlingcat 6y ago

How did I know that this would be the result from the headline? It's just like the old saying. Garbage in, garbage out.

rrobukef 6y ago
An important distinction: publishable garbage! Luckily, there is now a baseline, the next person can only publish improvements. Maybe one day we reach Hari Seldon levels of AI.

Cthulhu_ 6y ago

Yeah, it sounds more like they're training an AI for resource management games (like idk, Dwarf Fortress and co?)

Serious_Cheese 6y ago

The headline is misleading (click-bait). Reading the headline, I imagined they have simulated the entire economy of the world and have used reinforcement learning to find optimal policies, but they only simulated 4 workers and are making generalization to the entire economy.

im3w1l 6y ago

An issue I see here is that while punishing the poor is clever way of reducing poverty, a monetary punishment risks trapping them in a negative spiral. Beating them up (but not so much they can't work) or even killing them might work better. I hope people using AI for public policy are careful what they wish for.

thedudeabides5 6y ago
Agree with these comments. Like maybe the "AI" found some "Hand of God" move for the economy.... Like many solutions that AIs come up with—such as some of AlphaZero’s game-winning moves—the result appears counterintuitive and not something that a human might have devised. ...or maybe their model doesn't include the fact that when you raise taxes on the poor too high, in eras of tremendous inequality, you get bread riots, political violence, and eventually, revolutions. Not to say this kind of modeling isn't potentially useful! Just modern monetary policy already uses algorithms etc (Taylor rule optimization etc), so the question is how can ML/AI supplement those humans with better tools, rather than thinking a computer is going to be able to deal with the complexity of the real world. Put another way, what does the machine do when interest rates hit 0?
- neffy 6y ago
  The so called A.I. was not working with anything that resembled a real economy. Standard economic models also have this problem, they also have other problems, but this particular example is just doubling down and then some. What is particularly unfortunate is that there are people out there trying to create real agent based models based on actual simulations of the Economy that are far more advanced than this (realistic markets, accurate simulations of the Banking System) but are not getting any of this kind of exposure. Salesforce obviously is backing its own product, but the MIT Technology Review should really know better.
jbay808 6y ago
If the AI workers can't get sick, die, starve, suffer reduced productivity from chronic health issues, or care for children, then I don't see what lessons they can really expect to learn from this model. If their poor AI workers can just keep working hard gathering wood and stone forever, being fully productive without ever eating any food so they can afford to pay their taxes, then it's disconnected from reality beyond anything I'd even consider a "toy model".

monadic2 6y ago

TBH I'd trust a describable model much more than "cutting edge AI"

notahacker 6y ago
Yep. Looking into their model in detail, whilst it has all the simplifications of a typical economic model designed for tractability and algebraic solutions (two good economy, four agents, tax officials have perfect information) , it also adds plenty of unusual assumptions like agents moving round a grid to find resources. Which as others point out might tell us a lot about how AI can optimise strategy games (more precisely that AI optimises for values in a strategy game better than simply using the Saez formula or US federal tax brackets), but very little about the relationship between debt, taxes and investment.
tharne 6y ago
I would trust a monkey throwing darts at the stock page over "cutting edge AI".

clavalle 6y ago

Whereas, in the real world, the poor might not get the chance to not be poor because destitution would cause their death. I'd be interested in playing with such a simulation where the agents must buy or make food to eat and must have shelter or suffer. And, also, that the other agents know this and can play their own hand accordingly. I think this would change where the redistribution scheme would land quite a bit if the idea is to maximize value creation.

Gimpei 6y ago

And yet, as simple as this is, four agents is more than you see in most "complex" DSGE models... This looks to me like they're applying deep RL to agent-based modeling. It does seem pretty silly now, but it could be promising. You don't have to restrict these agents to any neoclassical assumptions and there isn't any reason you couldn't introduce firms as well. The only limit is computational power. Macroeconomics tends to look down on agent-based modeling because it doesn't provide neat, closed form solutions. But I've never bought that the results from DSGE models are anything more than products of all their simplifying assumptions. After all, they seem to be the very definition of overfitedness with absolutely no predictive power. At least they're trying something new here.

QuesnayJr 6y ago
DSGE models are designed be solved on your laptop in 0.0001 seconds. There are more complicated models, like Krussell-Smith, but it's not clear that they help explain macroeconomics better. Agent-based modelling isn't more popular because they assume that the agents are very, very stupid. Also, not only do they not produce closed-form solutions, they don't tend to lead to clear statements at all. Also also, the leading agent-based modeler, Cars Hommes, is infinitely better published than most economists, so he's not suffering too bad.

Jabbles 6y ago

It reminds me of the covid models, particularly the infamous Imperial model. https://en.wikipedia.org/wiki/CovidSim https://en.wikipedia.org/wiki/CovidSim The question is whether it's useful to have a model at all, or whether it's better to just reason about it from first principles. If a basic simulation gave you a tool to explore emergent properties, perhaps some theories could be generalised to apply to real economies? Or maybe not, I don't know.

whatshisface 6y ago
Models are supposed to be based on first-principles reasoning. A "model" that's not designed to capture some aspect of reality is called a computer game...
SiempreViernes 6y ago
The Imperial model gives reproducible results, so most of the noise was professional programmers being salty about the fact that ugly code can still be accurate code. The fact that it predicted things like 80% of the population becoming infected was simply unrelated to code quality.
- benmller313 6y ago
  Regardless of code quality, are these models really useful if, as you say, they are "accurate", but still make predictions wildly out of step with reality?
  - whatshisface 6y ago
    The real imperial story is that there were two wrong parties: - The scientists who were not including enough detail in their math. - The drive-by developers who had no idea what did or didn't make scientific simulations work. Together they produced both an inaccurate estimate, and a github page full of uninformed speculation about what the problem might have been. ;)
    - SiempreViernes 6y ago