15 ms·

Hash – Complex Systems Simulation

108 points by cocoflunchy 7y ago

garmaine 7y ago
Simulation... of what? "General purpose simulation" doesn't really make sense to me as a concept, unless maybe there is some other meaning of the word simulation that is meant here?
- saghm 7y ago
  I saw a talk from one of the founders just earlier this week at a meetup; it's essentially an engine that lets you model agent-based simulations and then multiple frontends that let you design them. He did several demos of their web-based UI, which could model simple things like Conway's Game of Life as well as much more complex things, like a 3D-representation of a thunderstorm in a rain forest. It was really cool!
  - 7thaccount 7y ago
    Like Netlogo and GAMA?
    - antpls 7y ago
      From reading the link, it looks more like https://gym.openai.com/ https://gym.openai.com/ There is a Machine Learning component that is not in NetLogo
  - garmaine 7y ago
    That doesn’t actually answer the question, although the examples inch forward towards an unknown answer. Agent-based simulations of... what, exactly?
    - koliber 7y ago
      Assuming you can code an agent representing the actor or component of what you are simulating, it sounds like “what” gets defined by you, the user.
      - DavidWilkinson 7y ago
        This ^^ Our starter models are pretty diverse and span real-world problems like optimizing controlled burns w/r/t wildfire containment, and improving Stack Overflow community dynamics (sorry... not a problem!) In addition we have several dozen classic ABM toy models such as Boids, Schelling segregation, and Conway's GoL. These individually showcase different components of the system.
- tyingq 7y ago
  I'm guessing it's similar to this, at least in purpose: https://github.com/NotSoOld/OpenGPSS/blob/master/README.md https://github.com/NotSoOld/OpenGPSS/blob/master/README.md "Examples of such systems: shops (one can move among shelves with food, then stay in queue, then process at the cashbox), computer systems (transacts are electric signals or information messages), public transport (people move from stop to stop)"
- ericand 7y ago
  Sounds like it is mostly business operations and organizational planning concerns. From the solutions page... Here are a few problems the community is working on: Cloud infrastructure orchestration Sales team remuneration Mergers and acquisitions Price sensitivity and revenue Warehouse operations Fleet efficiency Supply chain risk Macroeconomic policy Call center operations Store design/layout Critical national infrastructure Evacuation planning Political risk Marketing campaign efficiency Public health
- wespiser_2018 7y ago
  All the atoms in the universe!
- mkl 7y ago
  Agreed. This is one of the most bizarrely impenetrable landing pages I've ever seen. The "solutions" page is a wide-ranging list of vague topics, not solutions. Even the "demo simulations" link at the bottom provides no information without signing up.
wespiser_2018 7y ago
This looks like a bunch of "Sales-Foo". What is the underlying algorithm, and how does it work to help business solve specific business problems? Machine learning techniques are rarely revolutionary, so what is this based off of, and what are the pros/cons of that approach? I don't know, but this "coming soon" code base always makes me think it's vaporware.
- EsssM7QVMehFPAs 7y ago
  > As much as 10-50x faster than existing best-in-class tools. We’ve flipped simulation software on its head through modern functional programming. I'd guess they accelerate simulations on the GPU and/or over multiple cores. Rarely revolutionary since this is certainly best practice in any serious application domain. Might make it much more accessible to implement though. We'll see, I also remain rather sceptical.
  - wespiser_2018 7y ago
    It looks like they are using Rust, which is at least a positive.[1] Further, there are a few "wins" for agent-based modeling, like traffic congestion and supply chain optimization (wikipedia). However, in agent based modeling, you are looking for emergent behavior (i.e. simulating sheep farmers and observing a "tragedy of the commons") and require you to correctly assume the proper agent actions for given states. Further, explainability (why the model says X given conditions W,Y,Z) is pretty poor for ABM, but the solutions could still be useful in a black box way. There's no doubt in my mind that you can run 10-50x faster ABMs than whatever tool exists using by leveraging advances like cloud computing GPUs, I'm just not sure what that actually get you in terms of problem solving capability. It's not like 50x speed like bring these models onto my phone, they will still be the domain of data scientists and subject matter experts with the patience to wait. [1] https://www.meetup.com/Rust-NYC/events/266627924/?aff=garysguide https://www.meetup.com/Rust-NYC/events/266627924/?aff=garysg...
    - willvarfar 7y ago
      I imagine you can do 50x faster ABM just by tuning, writing in a compiled language, owning the low level library routines that do the work etc. I mean, most ABM is probably sub-optimally implemented. There must be lots of “leading” software that squanders the resources it is given to run on. So, just implement with some mechanical sympathy and you get order of magnitudes improvement.
kick 7y ago
This is the new company by Joel Spolsky, the founder of Stack Overflow (and notorious blogger), which is why it's relevant to HN.
- tobr 7y ago
  He’s chairman, I’m not sure if it’s “by” him exactly. He seems to be describing Hash as “they” rather than “we”.[1] 1: https://www.joelonsoftware.com/2019/12/05/so-hows-that-retirement-thing-going-anyway/ https://www.joelonsoftware.com/2019/12/05/so-hows-that-retir...
tirumaraiselvan 7y ago
This is one of the game-changer use-cases of wasm. Not sure how such a product would work without that technology.
- mkl 7y ago
  Probably just the same but a bit slower. People have been compiling to Javascript (and asm.js) for many years.
  - DavidWilkinson 7y ago
    I think it's fair to say we would have chosen an entirely different architectural approach had it not been for Wasm. Whilst simulations would have run slower, our dev progress would also have taken a hit. We're big fans and optimistic about its future :)
sdan 7y ago
So does it replicate what Mujoco and Pybullet do? What exactly do you mean by simulation?
d--b 7y ago
Simulation of what, ffs??
- TomMarius 7y ago
  I guess it's a generalized environment to simulate any (or a subset of) complex system https://en.m.wikipedia.org/wiki/Complex_system https://en.m.wikipedia.org/wiki/Complex_system
  - DavidWilkinson 7y ago
    Hey Åsmund, all. Tom's got it ^^ In theory you can model all of the above, but simulating complex environments in an agent-based fashion is the initial thing we've focused on creating a really streamlined experience for.
- semi-extrinsic 7y ago
  That was my question as well. Can this thing solve toy simulation problems like Stokes flow around a sphere, stress in a cantilever beam, or even a van der Pol oscillator?
- wespiser_2018 7y ago
  traffic, packages moving through sorting centers, shoppers in a store, people choosing where to live, in general, the what, in agent-based modeling is just an agent, which will interact with the environment through a set of defined rules.
- davidgl 7y ago
  Agent Based Modelling, an approach for modelling complex systems - The most popular open source tech is Java based NetLogo. Here is the web version, hit setup and then run to see an example simulation of ants collecting food - http://netlogoweb.org/launch#http://netlogoweb.org/assets/modelslib/Sample%20Models/Biology/Ants.nlogo http://netlogoweb.org/launch#http://netlogoweb.org/assets/mo...
adfhghg123345 7y ago
This is a cool idea. It seems like currently only the major players have troves of real data to feed their ML algorithms. Google has clocked how many real hours of driving for their cars? But if an upstart wants to bring a better driving AI/algorithm to the market they won't be able to because they don't have a Google-scale fleet with unlimited cash to burn on driving hours. In those cases simulation becomes a really good substitute. Driving is one example but I can imagine other cases where simulating the real word as data for an ML model is just as good as clocking the real world hours.
- DavidWilkinson 7y ago
  This is a great example. Relatedly Josh Tobin and team (OpenAI) have done some awesome recent work on how domain randomization can be applied to simulated worlds to improve the generalizability of algorithms as well: https://arxiv.org/abs/1710.06425 https://arxiv.org/abs/1710.06425
willvarfar 7y ago
I wonder the extent to which it builds on or learns from Joel’s Monte Carlo project estimates? (https://www.joelonsoftware.com/2007/10/26/evidence-based-scheduling/ https://www.joelonsoftware.com/2007/10/26/evidence-based-sch... 2007) Or perhaps it’s a completely different thing, made by different people, who had no knowledge of all that? Perhaps Joel is just the accidental sponsor like he was with Trello? A bunch of interns working for him invent something cool and present it to him and then let him take over the world? (Not that I think that a bad thing!)
- DavidWilkinson 7y ago
  Hey William! Monte Carlo is one experiment type we support. :) You're half right -- Jude started as an intern at Fog Creek eleven years ago and rose over a decade to become CTO. So technically he once was an intern... But the startup originated outside of Fog Creek, and Joel's involvement is very much intentional.
decentralised 7y ago
I've been using cadCAD lately, might be interesting for others here: https://community.cadcad.org/ https://community.cadcad.org/

108 points by cocoflunchy 7y ago

Simulation... of what? "General purpose simulation" doesn't really make sense to me as a concept, unless maybe there is some other meaning of the word simulation that is meant here?

saghm 7y ago
I saw a talk from one of the founders just earlier this week at a meetup; it's essentially an engine that lets you model agent-based simulations and then multiple frontends that let you design them. He did several demos of their web-based UI, which could model simple things like Conway's Game of Life as well as much more complex things, like a 3D-representation of a thunderstorm in a rain forest. It was really cool!
- 7thaccount 7y ago
  Like Netlogo and GAMA?
  - antpls 7y ago
    From reading the link, it looks more like https://gym.openai.com/ https://gym.openai.com/ There is a Machine Learning component that is not in NetLogo
- garmaine 7y ago
  That doesn’t actually answer the question, although the examples inch forward towards an unknown answer. Agent-based simulations of... what, exactly?
  - koliber 7y ago
    Assuming you can code an agent representing the actor or component of what you are simulating, it sounds like “what” gets defined by you, the user.
    - DavidWilkinson 7y ago
      This ^^ Our starter models are pretty diverse and span real-world problems like optimizing controlled burns w/r/t wildfire containment, and improving Stack Overflow community dynamics (sorry... not a problem!) In addition we have several dozen classic ABM toy models such as Boids, Schelling segregation, and Conway's GoL. These individually showcase different components of the system.
tyingq 7y ago
I'm guessing it's similar to this, at least in purpose: https://github.com/NotSoOld/OpenGPSS/blob/master/README.md https://github.com/NotSoOld/OpenGPSS/blob/master/README.md "Examples of such systems: shops (one can move among shelves with food, then stay in queue, then process at the cashbox), computer systems (transacts are electric signals or information messages), public transport (people move from stop to stop)"
ericand 7y ago
Sounds like it is mostly business operations and organizational planning concerns. From the solutions page... Here are a few problems the community is working on: Cloud infrastructure orchestration Sales team remuneration Mergers and acquisitions Price sensitivity and revenue Warehouse operations Fleet efficiency Supply chain risk Macroeconomic policy Call center operations Store design/layout Critical national infrastructure Evacuation planning Political risk Marketing campaign efficiency Public health
wespiser_2018 7y ago
All the atoms in the universe!
mkl 7y ago
Agreed. This is one of the most bizarrely impenetrable landing pages I've ever seen. The "solutions" page is a wide-ranging list of vague topics, not solutions. Even the "demo simulations" link at the bottom provides no information without signing up.

wespiser_2018 7y ago

This looks like a bunch of "Sales-Foo". What is the underlying algorithm, and how does it work to help business solve specific business problems? Machine learning techniques are rarely revolutionary, so what is this based off of, and what are the pros/cons of that approach? I don't know, but this "coming soon" code base always makes me think it's vaporware.

EsssM7QVMehFPAs 7y ago
> As much as 10-50x faster than existing best-in-class tools. We’ve flipped simulation software on its head through modern functional programming. I'd guess they accelerate simulations on the GPU and/or over multiple cores. Rarely revolutionary since this is certainly best practice in any serious application domain. Might make it much more accessible to implement though. We'll see, I also remain rather sceptical.
- wespiser_2018 7y ago
  It looks like they are using Rust, which is at least a positive.[1] Further, there are a few "wins" for agent-based modeling, like traffic congestion and supply chain optimization (wikipedia). However, in agent based modeling, you are looking for emergent behavior (i.e. simulating sheep farmers and observing a "tragedy of the commons") and require you to correctly assume the proper agent actions for given states. Further, explainability (why the model says X given conditions W,Y,Z) is pretty poor for ABM, but the solutions could still be useful in a black box way. There's no doubt in my mind that you can run 10-50x faster ABMs than whatever tool exists using by leveraging advances like cloud computing GPUs, I'm just not sure what that actually get you in terms of problem solving capability. It's not like 50x speed like bring these models onto my phone, they will still be the domain of data scientists and subject matter experts with the patience to wait. [1] https://www.meetup.com/Rust-NYC/events/266627924/?aff=garysguide https://www.meetup.com/Rust-NYC/events/266627924/?aff=garysg...
  - willvarfar 7y ago
    I imagine you can do 50x faster ABM just by tuning, writing in a compiled language, owning the low level library routines that do the work etc. I mean, most ABM is probably sub-optimally implemented. There must be lots of “leading” software that squanders the resources it is given to run on. So, just implement with some mechanical sympathy and you get order of magnitudes improvement.

kick 7y ago

This is the new company by Joel Spolsky, the founder of Stack Overflow (and notorious blogger), which is why it's relevant to HN.

tobr 7y ago
He’s chairman, I’m not sure if it’s “by” him exactly. He seems to be describing Hash as “they” rather than “we”.[1] 1: https://www.joelonsoftware.com/2019/12/05/so-hows-that-retirement-thing-going-anyway/ https://www.joelonsoftware.com/2019/12/05/so-hows-that-retir...

tirumaraiselvan 7y ago

This is one of the game-changer use-cases of wasm. Not sure how such a product would work without that technology.

mkl 7y ago
Probably just the same but a bit slower. People have been compiling to Javascript (and asm.js) for many years.
- DavidWilkinson 7y ago
  I think it's fair to say we would have chosen an entirely different architectural approach had it not been for Wasm. Whilst simulations would have run slower, our dev progress would also have taken a hit. We're big fans and optimistic about its future :)

sdan 7y ago

So does it replicate what Mujoco and Pybullet do? What exactly do you mean by simulation?

d--b 7y ago

Simulation of what, ffs??

TomMarius 7y ago
I guess it's a generalized environment to simulate any (or a subset of) complex system https://en.m.wikipedia.org/wiki/Complex_system https://en.m.wikipedia.org/wiki/Complex_system
- DavidWilkinson 7y ago
  Hey Åsmund, all. Tom's got it ^^ In theory you can model all of the above, but simulating complex environments in an agent-based fashion is the initial thing we've focused on creating a really streamlined experience for.
semi-extrinsic 7y ago
That was my question as well. Can this thing solve toy simulation problems like Stokes flow around a sphere, stress in a cantilever beam, or even a van der Pol oscillator?
wespiser_2018 7y ago
traffic, packages moving through sorting centers, shoppers in a store, people choosing where to live, in general, the what, in agent-based modeling is just an agent, which will interact with the environment through a set of defined rules.
davidgl 7y ago
Agent Based Modelling, an approach for modelling complex systems - The most popular open source tech is Java based NetLogo. Here is the web version, hit setup and then run to see an example simulation of ants collecting food - http://netlogoweb.org/launch#http://netlogoweb.org/assets/modelslib/Sample%20Models/Biology/Ants.nlogo http://netlogoweb.org/launch#http://netlogoweb.org/assets/mo...

adfhghg123345 7y ago

This is a cool idea. It seems like currently only the major players have troves of real data to feed their ML algorithms. Google has clocked how many real hours of driving for their cars? But if an upstart wants to bring a better driving AI/algorithm to the market they won't be able to because they don't have a Google-scale fleet with unlimited cash to burn on driving hours. In those cases simulation becomes a really good substitute. Driving is one example but I can imagine other cases where simulating the real word as data for an ML model is just as good as clocking the real world hours.

DavidWilkinson 7y ago
This is a great example. Relatedly Josh Tobin and team (OpenAI) have done some awesome recent work on how domain randomization can be applied to simulated worlds to improve the generalizability of algorithms as well: https://arxiv.org/abs/1710.06425 https://arxiv.org/abs/1710.06425

willvarfar 7y ago

I wonder the extent to which it builds on or learns from Joel’s Monte Carlo project estimates? (https://www.joelonsoftware.com/2007/10/26/evidence-based-scheduling/ https://www.joelonsoftware.com/2007/10/26/evidence-based-sch... 2007) Or perhaps it’s a completely different thing, made by different people, who had no knowledge of all that? Perhaps Joel is just the accidental sponsor like he was with Trello? A bunch of interns working for him invent something cool and present it to him and then let him take over the world? (Not that I think that a bad thing!)

DavidWilkinson 7y ago
Hey William! Monte Carlo is one experiment type we support. :) You're half right -- Jude started as an intern at Fog Creek eleven years ago and rose over a decade to become CTO. So technically he once was an intern... But the startup originated outside of Fog Creek, and Joel's involvement is very much intentional.

decentralised 7y ago

I've been using cadCAD lately, might be interesting for others here: https://community.cadcad.org/ https://community.cadcad.org/