4 ms·

Agreed. Allow me to be skeptical, but I find it hard to believe that less than 4000 lines of NetLogo are going to accurately model a system as complex as bee ec

by SamBam 5y ago

Agreed. Allow me to be skeptical, but I find it hard to believe that less than 4000 lines of NetLogo are going to accurately model a system as complex as bee ecosystems. I say this as someone who has spent many years writing ecosystem models for schools (some of them with NetLogo). Toy models often require more than that to be useful. You'll have a set of assumptions. You'll have a number of formulae that you can individually say there is experimental evidence they roughly models reality, if everything else is held constant. But you almost certainly do not have the experimental evidence to say that when these six formulae are interacting together, they will behave in known, deterministic ways. The paper does not appear to show any applications where researchers were able to predict something novel using the simulation, test it in the real world, and see how well it matched the results. Their "results" (if I'm interpreting correctly) appear to be mostly summarized in figure 3, which seems to be created by trying different settings until the model very, very roughly matched some historical dataset. Someone correct me if I'm wrong.

troelsSteegin 5y ago
Is there any literature you'd recommend on the disjunction between credible ecosystem models and toy models? I don't disagree, I just want to know more about the middle space between toy on one hand and authoritatively callibrated on the other. If there is one single person to read on system dynamics modeling, it's John Sterman. Also, the Systems Dynamics Review (SDR), https://systemdynamics.org/system-dynamics-review/ https://systemdynamics.org/system-dynamics-review/ , is what I think of as the journal of record. I am also aware of some work by the Concord Consortium for simulation in education. What seems to get written up is either high effort/high cred (SDR) or edu/toy.
- SamBam 5y ago
  I have probably written most of the ecosystem simulations for the Concord Consortium, as well as many other non-ecosystem dynamic models, so that's where I'm speaking from. I would describe all my work as "toy" models, in the sense that they are designed to demonstrate some aspect of an ecological system in a pedagogical way. The purpose of any one model is generally (1) to show 2 or 3 emergent phenomena, (2) to simply showcase the complexity of interconnected systems, where (say) eliminating a predator species can have surprising effects on the ecosystem at large, and (3) to introduce students to the idea of modeling as a practice -- i.e. making it explicit that they are working with a model, that models are not reality but make certain assumptions and decisions, and that the students themselves may be involved in making those decisions, and needing to defend them (from some work with Uri Wilensky, I also know that this is explicitly why NetLogo models always allow the end-user to edit the code, and this cannot be turned off). I would describe a credible ecosystem model as one from which justifiable predictions can be drawn. These would need to be demonstrated experimentally. Personally I don't know any credible ecosystem models, because that's not my field, but I'm sure that some exist, to varying degrees of credibility. My skepticism of the Bee model was based more on my knowledge of how high that cliff would be to scale, and how this model seemed not significantly more complex than plenty of models I have written.
  - troelsSteegin 5y ago
    Thank you. Thinking speculatively about a middle ground, I wonder what is minimally necessary in a model to accurately predict population dynamics - this intervention will tend to grow your colony, this other intervention will tend to decrease it. I think of higher fidelity models as supporting prediction of degree of population change - by how much up or down, and by when. But, I think a thing about "systems" is that you really can't separate dynamics from degree.
    - SamBam 5y ago
      The thing is, I would expect every individual "intervention" to be modelled by its own model or equation (whether or not it's literally an equation is irrelevant, at the end all system dynamics are governed by equations), each of which has presumably been backed by some scientific research. For instance, if bees are getting afflicted by some fungus, then there is some research that says that applying fungicide X will allow the colony to prevent y% decline in the ideal case, or whatever. At that point, even though the model will then incorporate this "sub-model," you don't need the system dynamics model to say what will happen when you apply the fungicide, you only need that well-researched sub-model. I feel (and I'm getting a little out of my depth because of my limited research on real-world uses of these models) that the system dynamics model is only useful for attempting to determine what happens when you have a conjunction of various interventions, or interventions with various environmental parameters that were outside the original research of the sub-model. But that's precisely were the limitations of the system dynamics model will come into play, because the non-linear or chaotic effects may mean that the model's prediction that the colony will grow or fail may have little to do with reality, because of the complexity of accurately accounting for so many parameters, and guessing how they interact without specific research on how they interact.

by SamBam 5y ago

troelsSteegin 5y ago

Is there any literature you'd recommend on the disjunction between credible ecosystem models and toy models? I don't disagree, I just want to know more about the middle space between toy on one hand and authoritatively callibrated on the other. If there is one single person to read on system dynamics modeling, it's John Sterman. Also, the Systems Dynamics Review (SDR), https://systemdynamics.org/system-dynamics-review/ https://systemdynamics.org/system-dynamics-review/ , is what I think of as the journal of record. I am also aware of some work by the Concord Consortium for simulation in education. What seems to get written up is either high effort/high cred (SDR) or edu/toy.

SamBam 5y ago
I have probably written most of the ecosystem simulations for the Concord Consortium, as well as many other non-ecosystem dynamic models, so that's where I'm speaking from. I would describe all my work as "toy" models, in the sense that they are designed to demonstrate some aspect of an ecological system in a pedagogical way. The purpose of any one model is generally (1) to show 2 or 3 emergent phenomena, (2) to simply showcase the complexity of interconnected systems, where (say) eliminating a predator species can have surprising effects on the ecosystem at large, and (3) to introduce students to the idea of modeling as a practice -- i.e. making it explicit that they are working with a model, that models are not reality but make certain assumptions and decisions, and that the students themselves may be involved in making those decisions, and needing to defend them (from some work with Uri Wilensky, I also know that this is explicitly why NetLogo models always allow the end-user to edit the code, and this cannot be turned off). I would describe a credible ecosystem model as one from which justifiable predictions can be drawn. These would need to be demonstrated experimentally. Personally I don't know any credible ecosystem models, because that's not my field, but I'm sure that some exist, to varying degrees of credibility. My skepticism of the Bee model was based more on my knowledge of how high that cliff would be to scale, and how this model seemed not significantly more complex than plenty of models I have written.
- troelsSteegin 5y ago
  Thank you. Thinking speculatively about a middle ground, I wonder what is minimally necessary in a model to accurately predict population dynamics - this intervention will tend to grow your colony, this other intervention will tend to decrease it. I think of higher fidelity models as supporting prediction of degree of population change - by how much up or down, and by when. But, I think a thing about "systems" is that you really can't separate dynamics from degree.
  - SamBam 5y ago
    The thing is, I would expect every individual "intervention" to be modelled by its own model or equation (whether or not it's literally an equation is irrelevant, at the end all system dynamics are governed by equations), each of which has presumably been backed by some scientific research. For instance, if bees are getting afflicted by some fungus, then there is some research that says that applying fungicide X will allow the colony to prevent y% decline in the ideal case, or whatever. At that point, even though the model will then incorporate this "sub-model," you don't need the system dynamics model to say what will happen when you apply the fungicide, you only need that well-researched sub-model. I feel (and I'm getting a little out of my depth because of my limited research on real-world uses of these models) that the system dynamics model is only useful for attempting to determine what happens when you have a conjunction of various interventions, or interventions with various environmental parameters that were outside the original research of the sub-model. But that's precisely were the limitations of the system dynamics model will come into play, because the non-linear or chaotic effects may mean that the model's prediction that the colony will grow or fail may have little to do with reality, because of the complexity of accurately accounting for so many parameters, and guessing how they interact without specific research on how they interact.