4 ms·
AFAIU this is entirely simulated - if it turns out the simulation doesn't perfectly match reality, what are the challenges in re-adjusting? Does the RL network
by klabb3 5y ago
AFAIU this is entirely simulated - if it turns out the simulation doesn't perfectly match reality, what are the challenges in re-adjusting? Does the RL network need to be re-trained with massive amounts of data on a new improved simulation or can it be fine tuned on the real thing?
- kevinqi 5y agoThat's a great question; I'm curious about this as well.
- willis936 5y agoThis is why fundamental plasma physics is so valuable to fund. ML can come up with good models that fit the data in the domains that there are empirical datasets to train against, but it can't reliably extrapolate outside of those domains. So this is a good tool for the computationalists but we still need the experimentalists. This is science.
- ShamelessC 5y agoTo clarify, this result did not have the benefit of a large labeled dataset. They used a simulator and got working results without _any_ finetuning on data measured from the real world. > So this is a good tool for the computationalists but we still need the experimentalists. This is science. I would say that this is a good example of collaboration between the groups.
- deleted 5y ago[deleted]
- starwind 5y agoProbably massive amounts of data but it depends on how far off the simulation is from reality and the expected cost of collecting that data vs running the simulation a few billion times. Not far off and you want perfect results? Go with actual data. Very far off where it's not even usable? You'd need to go back to the training environment and redo the reward function. Generally speaking (and I'm not sure if this is the case from reading the link but I may have missed it) reinforcement learning for optimal control is done to fine tune while traditional control methods are used for coarse adjustments. Since this is deepmind they probably want to use RL for everything
- 317070 5y agoHi, author here. We validated the neural network controller on the real plant in a couple of experiments. All experiments in the paper are based on data from running the controller on an actual tokamak. There is a video of one of these experiments in the linked blogpost (figure 2), where we show imagery taken from a camera looking into the vessel. Next to it is a reconstruction of the shot, visualised on a cross-section of the torus. The simulation not perfectly matching reality is indeed a major challenge, and the nature paper is mainly about showing how you can overcome that challenge. As others have said, plasma is a pain to model, and you cannot collect a ton of data on these machines. But it seems technology is coming at a point where we know how to work within these limits and still get it to work.
- klabb3 5y agoVery cool, and thanks for replying to laymen comments such as mine. Do you think this line of research will eventually feed back into improved designs of electromagnets, chambers and so on? My limited understanding of the field is that the optimal reactor design is a moving target that's changed a lot over the years due to engineering improvements in manufacturing, material sciences and so on.
- 317070 5y agoYeah, I think the main contribution of this paper in the long run, might be that the time is right for optimizing the controller together with the hardware? I'm imagining optimizing the plasma configuration, reactor shape, coil/sensor placement and controller all in one giant computational optimization. All the algorithmic components and models humanity needs to attempt this seem to be there.
- willis936 5y agoWell as long as there have been electronic computers they have been applied they have been applied to the problems of magnetic confinement. For tokamaks static configurations and dynamic controls have been simulated. For stellarators geometry optimizations have been done. The time has always been right for this application. Stellarators are particularly fun. First you optimize a last closed magnetic flux surface (a 3D sheet) with relatively simple plasma models, then you optimize coils (you need to weight things like magnetic field error, gaps big enough for heating and diagnostics, and coil complexity a la minimum radius), then you perform more accurate simulations to start the overall optimization loop again.
- sanxiyn 5y agoThis was trained in simulation and then zero shot transferred to actual tokamak, namely https://en.wikipedia.org/wiki/Tokamak_a_configuration_variable https://en.wikipedia.org/wiki/Tokamak_a_configuration_variab.... All reported results are on actual tokamak.