5 ms·

>we do need and should have good fine-grain computational models of complex systems such [...] the global economy. Many years ago when 'social graphs' were sti

>we do need and should have good fine-grain computational models of complex systems such [...] the global economy. Many years ago when 'social graphs' were still a hot area to do research in I started building a simulation of the equivalent of a small medieval village. What became quickly apparent is that you didn't just need interactions between any two individuals like classical social graphs talked about, but between any number of arbitrary groups of individuals. Otherwise something as simple as an extended family couldn't be modeled. That meant that instead of being able to use a matrix as the fundamental data structure you'd need a tensor of rank N, where N is the number of people in the economy. Just to see how intractable this is if the village had 20 people in it with the traditional matrix approach you'd need 400 weights to model interactions. With the tensor approach you need ~1e+26. In short: it's impossible to have fined grained simulations of complex societies. The best we can do is drastic over simplifications that give us _some_ predictive power.

abdullahkhalids 2y ago
A set of size N=20, has total number of subsets equal to 2^N = 1.05 million. You must have had other degrees of freedom to bump that to 1E26.
- noosphr 2y ago
  That's only for two way relationships, you need N way relationships, which is N^N.
  - wordpad 2y ago
    Wouldn't you be able to quickly prune away invalid/unlikely interactions? Maybe, have some cutoff based on proximity of members or something?
    - noosphr 2y ago
      Yes, the resulting tensor is incredibly sparse, but still too large to ever be practical for use as anything but the theoretical upper limit on the complexity of a model. The issue is that while you can remove pretty much all possible interactions for a specific case you have no idea where an interaction could pop up unexpectedly with a huge impact ahead of time. For the medieval theme the leader of the village may be a cousin of the king which is a very distant but very strong interaction.
      - exe34 2y ago
        how many people do you talk to in a day?
        noosphr 2y ago
        This is a perfect example of the limitations of old school social graphs. The number of people you talk to is just the first order effect. What about all the people that people in your company talked to? That has a non zero economic impact on you. Similarly for any other company that your company talks to, and so on and so on.
        exe34 2y ago
        that still scales linearly with number of people, not quadratically.
        noosphr 2y ago
        It scales super exponentially since each person is a member of an arbitrary number of groups and their actions have some non zero impact on each other member of the groups they are a member of, and each individual in each group with a member of which they have interacted with.
        exe34 2y ago
        if that were the case, numerical weather prediction wouldn't work. you can add up the impact for each neighbour.
        noosphr 2y ago
        Humans are not mindless molecules whose only interactions are with the people right next to them.
        exe34 2y ago
pornel 2y ago
Use of a dense matrix is an artificial constraint you've imposed yourself, but that only disproves feasibility of your proposed solution, not the entire problem in general. A similar problem, n-body simulation*, has n² gravitational interactions. You will similarly hit a wall if you try to do it with a dense n² matrix. However, there's a hierarchical solution that takes advantage of the sparsity and exponential decay, and can solve it in (n log n) with an imperceptibly low loss of precision. Social interactions are sparse, and group interactions can be optimized with clustering. Fine-grained simulation of the entire society is such a massive chaotic problem with so many variables, that some loss of precision from clustering is completely insignificant compared to the inevitable simplifications you'll have to make in the design of the model itself. * I mean the naive one with a fixed timestemp, not trying to solve chaos.
- noosphr 2y ago
  >Use of a dense matrix is an artificial constraint you've imposed yourself, but that only disproves feasibility of your proposed solution, not the entire problem in general. The use of a dense matrix is the traditional way of solving the problem. The issue is that it solves the wrong problem. You need a dense tensor which requires more storage than the world currently has for an economy of 20 people. Social interactions are sparse until they aren't. If you think otherwise try to estimate what every Europeans interaction with Gavrilo Princip was on 27 June 1914 vs 28 June 1914. As for gravitation: I'm very happy for the planets and asteroids out there. Unfortunately the economy isn't a solar system.
- trod1234 2y ago
  Isn't the naive one hopeless since those systems require perfect measurement, and economic interactions are lagging, the calculations are ordinal, and mathematical systems fail because you can't relabel? (i.e. fails a function test so you end up trying to compare unrelated mathematical objects within differing systems). Even statistics is fully thrown out with the islands of regularity. The stochastic nature, lack of measure-ability, and multiple hidden underlying states (value is subjective) require any model to solve chaos somehow. https://science.ku.dk/english/press/news/2024/islands-of-regularity-discovered-in-the-famously-chaotic-three-body-problem/ https://science.ku.dk/english/press/news/2024/islands-of-reg...

abdullahkhalids 2y ago

A set of size N=20, has total number of subsets equal to 2^N = 1.05 million. You must have had other degrees of freedom to bump that to 1E26.

noosphr 2y ago
That's only for two way relationships, you need N way relationships, which is N^N.
- wordpad 2y ago
  Wouldn't you be able to quickly prune away invalid/unlikely interactions? Maybe, have some cutoff based on proximity of members or something?
  - noosphr 2y ago
    Yes, the resulting tensor is incredibly sparse, but still too large to ever be practical for use as anything but the theoretical upper limit on the complexity of a model. The issue is that while you can remove pretty much all possible interactions for a specific case you have no idea where an interaction could pop up unexpectedly with a huge impact ahead of time. For the medieval theme the leader of the village may be a cousin of the king which is a very distant but very strong interaction.
    - exe34 2y ago
      how many people do you talk to in a day?
      - noosphr 2y ago
        This is a perfect example of the limitations of old school social graphs. The number of people you talk to is just the first order effect. What about all the people that people in your company talked to? That has a non zero economic impact on you. Similarly for any other company that your company talks to, and so on and so on.
        exe34 2y ago
        that still scales linearly with number of people, not quadratically.
        noosphr 2y ago
        It scales super exponentially since each person is a member of an arbitrary number of groups and their actions have some non zero impact on each other member of the groups they are a member of, and each individual in each group with a member of which they have interacted with.
        exe34 2y ago
        if that were the case, numerical weather prediction wouldn't work. you can add up the impact for each neighbour.
        noosphr 2y ago
        Humans are not mindless molecules whose only interactions are with the people right next to them.
        exe34 2y ago

pornel 2y ago

Use of a dense matrix is an artificial constraint you've imposed yourself, but that only disproves feasibility of your proposed solution, not the entire problem in general. A similar problem, n-body simulation*, has n² gravitational interactions. You will similarly hit a wall if you try to do it with a dense n² matrix. However, there's a hierarchical solution that takes advantage of the sparsity and exponential decay, and can solve it in (n log n) with an imperceptibly low loss of precision. Social interactions are sparse, and group interactions can be optimized with clustering. Fine-grained simulation of the entire society is such a massive chaotic problem with so many variables, that some loss of precision from clustering is completely insignificant compared to the inevitable simplifications you'll have to make in the design of the model itself. * I mean the naive one with a fixed timestemp, not trying to solve chaos.

noosphr 2y ago
>Use of a dense matrix is an artificial constraint you've imposed yourself, but that only disproves feasibility of your proposed solution, not the entire problem in general. The use of a dense matrix is the traditional way of solving the problem. The issue is that it solves the wrong problem. You need a dense tensor which requires more storage than the world currently has for an economy of 20 people. Social interactions are sparse until they aren't. If you think otherwise try to estimate what every Europeans interaction with Gavrilo Princip was on 27 June 1914 vs 28 June 1914. As for gravitation: I'm very happy for the planets and asteroids out there. Unfortunately the economy isn't a solar system.
trod1234 2y ago
Isn't the naive one hopeless since those systems require perfect measurement, and economic interactions are lagging, the calculations are ordinal, and mathematical systems fail because you can't relabel? (i.e. fails a function test so you end up trying to compare unrelated mathematical objects within differing systems). Even statistics is fully thrown out with the islands of regularity. The stochastic nature, lack of measure-ability, and multiple hidden underlying states (value is subjective) require any model to solve chaos somehow. https://science.ku.dk/english/press/news/2024/islands-of-regularity-discovered-in-the-famously-chaotic-three-body-problem/ https://science.ku.dk/english/press/news/2024/islands-of-reg...