4 ms·

I'm looking for predictive economic models out in the open. I want to see the underlying assumptions and experiment. I'm not even picky about the kind of model:

by xpe 2y ago

I'm looking for predictive economic models out in the open. I want to see the underlying assumptions and experiment. I'm not even picky about the kind of model: it could be a relatively simple linear regression model, a systems dynamics model, a complex simulation, or anything in-between. I just want models to a focal point of discussion rather than some poorly-explained textual interpretation. I know... I'm asking for too much... this is usually proprietary stuff, so people are reluctant to share. But I want to live in a world where sensible people can compare models of what might happen. We don't have to necessarily agree on the assumptions or results. But we could at least be open about why we predict various outcomes. That would be better foundation that how we do things now, which is really embarrassing for a world with our capabilities. Eventually, I want to live in a world where we have sensible discussions about our predictions and about which outcomes are more and less favorable. P.S. If you are building a product or startup doing this, let me know. Or let everyone know. I built a product in this general area many years ago, but I didn't find product-market fit. Anyhow, I just want modeling tools like this to exist; I don't care who makes them. P.P.S. I'm very intentionally not bringing in my political philosophy nor level of personal concern here. To be open, I am personally affected, as are many of my friends, but I want to find some kind of common ground that doesn't make this about my particular take on the overall situation.

trescenzi 2y ago
I mean there’s some fed ones. Two are - GDPNow if you want shorter term: https://www.atlantafed.org/cqer/research/gdpnow https://www.atlantafed.org/cqer/research/gdpnow - DSGE if you want longer term: https://www.newyorkfed.org/research/policy/dsge#/overview https://www.newyorkfed.org/research/policy/dsge#/overview
- whatever1 2y ago
  Eia also publishes their models.
- xyst 2y ago
  GDP is such a deeply flawed metric of measuring human welfare. It’s an aggregate measure and doesn’t tell the distribution of wealth amongst nation. Also, it’s a metric that was sort of useful in the 1950s and during wartime when this country manufactured goods. Now it’s all unclear in this modern economy. It doesn’t take into proper account of services, quality of care, quality of life. It doesn’t even take into account _losses_ that may occur. Such as a natural disaster. Sure, in a post disaster GDP goes up because region is rebuilding. But the losses due to disaster are not figured into GDP. It’s a broken metric. Stop using this as an indicator of economic health and human welfare. edit; never heard of DSGE but yet again looks like yet another flawed “model” that assumes a level of “equilibrium” and seems to have been birthed out of the pseudoscience known as neoclassical (and orthodox) economics.
  - xpe 2y ago
    What you say has a lot of truth. And yet, these kinds of models are still useful, provided a broader range of models is also available and socialized. This touches on a key point: just because we can predict something doesn’t mean we should optimize for it. Still, being aware of consequences is useful: perhaps we want to satisfice under various constraints.
  - xpe 2y ago
    I don’t disagree with this criticism, but it also illustrates a common pattern where modelers tend to attack each other’s work. This criticism is somewhat inevitable because all models are flawed in some way, and reasonable people can disagree on how to best apply them. But a bigger problem is the lack of a variety of models to choose from and compare. My suggestion? If we want to encourage more models to exist in the world, we should use a mixture of (1) criticizing models (like above); (2) pointing to existing alternative models; (3) building new models; (4) writing up specific comparisons of models; (5) showing how to make models; (6) writing up successful case studies. And lots more.
mglz 2y ago
> I just want models to a focal point of discussion rather than some poorly-explained textual interpretation. Do we have any economic models which managed to predict any non-obvious, relevant things?
carlgreene 2y ago
I think my concern here would be people weighting their ideas/opinions far too much on mathematical models that may, or may not represent the complex economic reality. We already see this with people arguing with near 100% confidence of X happening because of Y and Z…in a vacuum. I want to see something similar to what you’re asking, don’t get me wrong. But the underbelly of that is a broken clock being right twice a day and far too much weight and decision making being put into some model that doesn’t actually reflect reality
- xpe 2y ago
  The solution to a bad model is more models :). I'm joking but serious. "All models are wrong, but some are useful."
  - dwattttt 2y ago
    Although also responsive, the solution to incorrectly applying models _isn't_ to get more models, it's to understand the mistakes being made in applying them.
    - xpe 2y ago
      True. I'll add a few points: - Statistically, more models tends to be better (in the same sense that more information is better) under the assumption that people can tell better from worse. - I like how you talk about _applying_ models instead of _using_ models. It is an easy one word change that (to me at least) emphasizes the choice points of: (a) selecting the model; (b) selecting parameters; (c) interpretation. - All models (by definition) are unrealistic in some way. If a model operates at a useful level of abstraction, inaccuracies at lower levels may be acceptable. In fact, such inaccuracies may be the key to making the model tractable and efficient.
mindcrime 2y ago
I don't have any insights on specific models that are "out there", but I'll share one tidbit you might find of interest if you're not already familiar with it. Douglas Hubbard's "Applied Information Economics"[1] is a methodology that could probably be applied to doing what you're asking for. Is it the only such methodology? Almost certainly not. Is it the best? I really don't know, as I think the scope of what you're looking at goes a little beyond what Doug originally intended for AIE. But it could potentially still be applied there, I think. [1]: https://hubbardresearch.com/about/applied-information-economics/ https://hubbardresearch.com/about/applied-information-econom...
- xpe 2y ago
  Thanks. From the linked page I only see a marketing page that sells the general ideas. It looks like the common pattern where a consultancy sells their expertise, perhaps with a differentiating advantage that their analysis is backed by a quantitative model that has been refined over time. I don’t know how they work, but they probably custom build or tailor their in-house model and provide the results and interpretation to clients. Maybe they even allow clients to play around with the model directly, but I wouldn’t bet on it. I’m guessing they gate access to it via consulting fees. One wouldn’t want a client to “misinterpret” the model, would we? :P Furthermore, the idea of open sourcing their base model isn’t probably something that they would consider; it is likely perceived as incompatible with their business model.
  - mindcrime 2y ago
    There's no "model" there, just a methodology. Hubbard does consulting and sells books, that's how he monetizes this as far as I know. His book How to Measure Anything[1] lays out the core ideas in sufficient detail for somebody to implement the basic idea. A used copy could be acquired for < $10 if one were interested in learning more. [1]: https://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/1118539273 https://www.amazon.com/How-Measure-Anything-Intangibles-Busi...
xyst 2y ago
> predictive economic models Not possible with this neoclassical economy we live in. It’s an ongoing grift that has been ongoing for _decades_ which has allowed for irrational thinking to prevail, for wealth and success to be focused into a select group, and for multibillion dollar corporations to be bailed out when their own decisions drive the company to the ground. Privatize the profits, socialize the losses.
- xpe 2y ago
  I get your point of view. But I'm intentionally trying to focus on prediction. There are ways to track ownership of entities and relationships. There are ways to quantify how contracts are awarded, the results, and the impacts. These are complex models, but they can give insight into the dynamics.
xpe 2y ago
A follow up idea: Don’t forget we modelers are still in a phase that might be characterized as “shifting mindsets”. The capability to model is getting steadily better, but I don’t think the practice of modeling is anywhere close to maturity nor broad diffusion. I was hopeful ~5-10 years ago with the rise of data science, but I don’t know how much this growth corresponded with decision makers using the models both broadly and in intellectually honest ways. There are areas where the results seem clear; e.g. advertising and product-specific decision points, such as recommendation engines. Beyond these short-term ML applications, there are other domains where modeling is essential: weather forecasting, politics, markets, economics. But this is just the tip of the iceberg. Here is a metric I’m interested in: what percentage of decisions are informed by quantitative models? Aside: Before you complain that e.g. “some decisions aren’t impactful enough to warrant a model-backed approach”, let me say that I sort of agree, but… There are indeed lightweight quantitative models (or at least processes) that can track / predict when using a heavier model is worth it! I don’t have systematic data on this; it is just a hunch, inspired by what I have heard and seen. Why? Effective modeling provides differential advantages. -intra-organizational diffusion: technically savvy people will rise into management. - inter-organizational dispersion: techniques used by winners will spread across organizations and industries.

by xpe 2y ago

trescenzi 2y ago

I mean there’s some fed ones. Two are - GDPNow if you want shorter term: https://www.atlantafed.org/cqer/research/gdpnow https://www.atlantafed.org/cqer/research/gdpnow - DSGE if you want longer term: https://www.newyorkfed.org/research/policy/dsge#/overview https://www.newyorkfed.org/research/policy/dsge#/overview

whatever1 2y ago
Eia also publishes their models.
xyst 2y ago
GDP is such a deeply flawed metric of measuring human welfare. It’s an aggregate measure and doesn’t tell the distribution of wealth amongst nation. Also, it’s a metric that was sort of useful in the 1950s and during wartime when this country manufactured goods. Now it’s all unclear in this modern economy. It doesn’t take into proper account of services, quality of care, quality of life. It doesn’t even take into account _losses_ that may occur. Such as a natural disaster. Sure, in a post disaster GDP goes up because region is rebuilding. But the losses due to disaster are not figured into GDP. It’s a broken metric. Stop using this as an indicator of economic health and human welfare. edit; never heard of DSGE but yet again looks like yet another flawed “model” that assumes a level of “equilibrium” and seems to have been birthed out of the pseudoscience known as neoclassical (and orthodox) economics.
- xpe 2y ago
  What you say has a lot of truth. And yet, these kinds of models are still useful, provided a broader range of models is also available and socialized. This touches on a key point: just because we can predict something doesn’t mean we should optimize for it. Still, being aware of consequences is useful: perhaps we want to satisfice under various constraints.
- xpe 2y ago
  I don’t disagree with this criticism, but it also illustrates a common pattern where modelers tend to attack each other’s work. This criticism is somewhat inevitable because all models are flawed in some way, and reasonable people can disagree on how to best apply them. But a bigger problem is the lack of a variety of models to choose from and compare. My suggestion? If we want to encourage more models to exist in the world, we should use a mixture of (1) criticizing models (like above); (2) pointing to existing alternative models; (3) building new models; (4) writing up specific comparisons of models; (5) showing how to make models; (6) writing up successful case studies. And lots more.

mglz 2y ago

> I just want models to a focal point of discussion rather than some poorly-explained textual interpretation. Do we have any economic models which managed to predict any non-obvious, relevant things?

carlgreene 2y ago

I think my concern here would be people weighting their ideas/opinions far too much on mathematical models that may, or may not represent the complex economic reality. We already see this with people arguing with near 100% confidence of X happening because of Y and Z…in a vacuum. I want to see something similar to what you’re asking, don’t get me wrong. But the underbelly of that is a broken clock being right twice a day and far too much weight and decision making being put into some model that doesn’t actually reflect reality

xpe 2y ago
The solution to a bad model is more models :). I'm joking but serious. "All models are wrong, but some are useful."
- dwattttt 2y ago
  Although also responsive, the solution to incorrectly applying models _isn't_ to get more models, it's to understand the mistakes being made in applying them.
  - xpe 2y ago
    True. I'll add a few points: - Statistically, more models tends to be better (in the same sense that more information is better) under the assumption that people can tell better from worse. - I like how you talk about _applying_ models instead of _using_ models. It is an easy one word change that (to me at least) emphasizes the choice points of: (a) selecting the model; (b) selecting parameters; (c) interpretation. - All models (by definition) are unrealistic in some way. If a model operates at a useful level of abstraction, inaccuracies at lower levels may be acceptable. In fact, such inaccuracies may be the key to making the model tractable and efficient.

mindcrime 2y ago

I don't have any insights on specific models that are "out there", but I'll share one tidbit you might find of interest if you're not already familiar with it. Douglas Hubbard's "Applied Information Economics"[1] is a methodology that could probably be applied to doing what you're asking for. Is it the only such methodology? Almost certainly not. Is it the best? I really don't know, as I think the scope of what you're looking at goes a little beyond what Doug originally intended for AIE. But it could potentially still be applied there, I think. [1]: https://hubbardresearch.com/about/applied-information-economics/ https://hubbardresearch.com/about/applied-information-econom...

xpe 2y ago
Thanks. From the linked page I only see a marketing page that sells the general ideas. It looks like the common pattern where a consultancy sells their expertise, perhaps with a differentiating advantage that their analysis is backed by a quantitative model that has been refined over time. I don’t know how they work, but they probably custom build or tailor their in-house model and provide the results and interpretation to clients. Maybe they even allow clients to play around with the model directly, but I wouldn’t bet on it. I’m guessing they gate access to it via consulting fees. One wouldn’t want a client to “misinterpret” the model, would we? :P Furthermore, the idea of open sourcing their base model isn’t probably something that they would consider; it is likely perceived as incompatible with their business model.
- mindcrime 2y ago
  There's no "model" there, just a methodology. Hubbard does consulting and sells books, that's how he monetizes this as far as I know. His book How to Measure Anything[1] lays out the core ideas in sufficient detail for somebody to implement the basic idea. A used copy could be acquired for < $10 if one were interested in learning more. [1]: https://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/1118539273 https://www.amazon.com/How-Measure-Anything-Intangibles-Busi...

xyst 2y ago

> predictive economic models Not possible with this neoclassical economy we live in. It’s an ongoing grift that has been ongoing for _decades_ which has allowed for irrational thinking to prevail, for wealth and success to be focused into a select group, and for multibillion dollar corporations to be bailed out when their own decisions drive the company to the ground. Privatize the profits, socialize the losses.

xpe 2y ago
I get your point of view. But I'm intentionally trying to focus on prediction. There are ways to track ownership of entities and relationships. There are ways to quantify how contracts are awarded, the results, and the impacts. These are complex models, but they can give insight into the dynamics.

xpe 2y ago

A follow up idea: Don’t forget we modelers are still in a phase that might be characterized as “shifting mindsets”. The capability to model is getting steadily better, but I don’t think the practice of modeling is anywhere close to maturity nor broad diffusion. I was hopeful ~5-10 years ago with the rise of data science, but I don’t know how much this growth corresponded with decision makers using the models both broadly and in intellectually honest ways. There are areas where the results seem clear; e.g. advertising and product-specific decision points, such as recommendation engines. Beyond these short-term ML applications, there are other domains where modeling is essential: weather forecasting, politics, markets, economics. But this is just the tip of the iceberg. Here is a metric I’m interested in: what percentage of decisions are informed by quantitative models? Aside: Before you complain that e.g. “some decisions aren’t impactful enough to warrant a model-backed approach”, let me say that I sort of agree, but… There are indeed lightweight quantitative models (or at least processes) that can track / predict when using a heavier model is worth it! I don’t have systematic data on this; it is just a hunch, inspired by what I have heard and seen. Why? Effective modeling provides differential advantages. -intra-organizational diffusion: technically savvy people will rise into management. - inter-organizational dispersion: techniques used by winners will spread across organizations and industries.