Thanks for sharing all of these examples. It is great to see that you are using Anylogic which I think is an excellent workbench for creating discrete-event mo

Thanks for sharing all of these examples. It is great to see that you are using Anylogic which I think is an excellent workbench for creating discrete-event models of production systems and entire supply chains even. Anylogic also comes with an integrated tool for parameter tuning using global optimization. Now, the model that I was looking into was a stochastic economic lot-sizing problem with random orders. At the end of a production run or upon arrival of a new order, the policy decided which product to produce next on a single machine. Each change however incurs switching cost and any orders that met an empty inventory were lost (the backlog case was easier). The difficulty with this problem was noisy rewards after each state transition due to order randomness, as well as for the training algorithm to communicate the future value of an action in a particular state far enough backwards in time. As said, it worked for easy problems but not for problems with 5 products or more (which I think is a ridiculously small and simple problem). Now, I do not know what you guys are using, whether your policy is a DNN and you are tuning parameters using REINFORCE / evolution strategies or whether you are using value function approximation with fitted Q-iteration (FQI) or similar (like DeepMind's Atari players). While I think that policy search may actually work (despite rather slow with Anylogic), I'd expect that FQI will run into trouble on more difficult problems. That said, deep RL is a great piece of tech, but I am a bit skeptical of the claim that RL will transform the future of manufacturing, as I have made the experience that it can easily struggle even with simple problems as soon as the action space becomes high-dimensional and reward information must be communicated across hundreds or even thousands of state transitions. Btw: are there any scientific papers on this?