18 ms·
Controlling the nuclear fusion plasma in a tokamak with reinforcement learning
- skyde 5y agooh dear god don't put Machine learning (a big unpredictable blackbox) in control of a safety critical process.
- duvenaud 5y agoThis is a bit off-topic, but can anyone here explain the expected economic advantages of fusion power over fission? My understanding is that fusion plants would also use heat-driven turbines, and be pretty capital intensive. Would they fill a different niche, or are they expected to be fundamentally more cost-effective?
- vermilingua 5y agoOpEx > CapEx Yes, fusion is considerably more capital intensive than fission power, but the fuel is just water. You don’t need to worry about digging the fuel out of the earth, maintaining a costly disposal scheme for hazardous waste, etc.
- TheSpiceIsLife 5y agoWhere does tritium come from?
- willis936 5y agoLi6 fission in between the first wall and confinement coils. If you want some details: check out section 3.2 of the ITER research plan. https://www.iter.org/doc/www/content/com/Lists/ITER%20Technical%20Reports/Attachments/9/ITER_Research_Plan_within_the_Staged_Approach_levIII_provversion.pdf https://www.iter.org/doc/www/content/com/Lists/ITER%20Techni...
- sanxiyn 5y agoITER plans to test breeding tritium from lithium. It should work in theory, but there is no practical experience yet. Current tritium used in fusion research is entirely produced from fission reactor moderated by heavy water, and there is only one production facility in the world at Canada.
- duvenaud 5y agoThanks, but is the fuel cost a significant factor in fission plants? And the waste disposal? There's already lots of radioactive material in the ground, after all, for example in uranium mines.
- colinmhayes 5y agoBuilding one power plant is incredibly expensive. Building it again costs less. Building it 1000 times costs much less per plant. We're not building thousands of fission plants because of politics. Fusion might be an easier sell.
- sanxiyn 5y agoThis isn't really true. Yes first of a kind is expensive and there is learning curve, but learning is serial and iteration cycle is long. You can't build 1000 plants in parallel and learn 1000x. Solar power cost is falling fast because iteration cycle is short. South Korea built 28 reactors over 36 years and cost fell 1.5% per year, totalling 40% reduction. Even with this South Korean nuclear power is only about as half cheap as solar power locally (as of 2020), and South Korea is a very poor country for solar power considering its latitude and weather.
- sanxiyn 5y agoFuel cost is not significant, waste disposal even less so. (In fact, usual accounting of nuclear fuel cost includes waste disposal cost. People argue cost is not accounted in full, but as accounted, waste disposal typically costs 1/6 of fuel cost.) In terms of cost, 80% of nuclear power is CAPEX. Among OPEX, about 1/3 is fuel, rest is operation and maintenance. So from total cost, about 6% is fuel, and about 1% is waste disposal.
- ncmncm 5y agoStill, fission operational expense is very, very large when compared to solar and wind, which also have radically lower capital cost. Operational expense of neutron-emission fusion would be radically more than for fission: not for the fuel, but for everything else.
- The_rationalist 5y agoNuclear byproducts storage (let's not call it waste since MOX fuel is the future for humanity) do not cost much, it's not a point that is salient enough to male fusion relevant in any way.
- torginus 5y agoNot an expert, but off the top of my head: - The end product of Deuterium + Tritium is regular, stable Helium, making waste disposal both safe, and cheap - The input of the process, heavy water, while not safe, is way less dangerous than thorium or uranium - The whole process is no way involved with nuclear weapons, making security concerns much less relevant - Since the process produces magnetically charged plasma, steam turbines are not necessary, a solution of directly harvesting energy with electromagnets was proposed.
- systemvoltage 5y agoMissing, arguably the most important factor: It is not a runaway reaction. You don't need a giant pool of water with functioning pumps to mitigate a disaster. There are some newer designs that are self-regulating but surprised to see this is not at the top of the list.
- u320 5y agoIt's not possible to have a runaway reaction in a light water reactor either. The major incidents (TMI and Fukushima) happened as the reactor cooled down. It's just that the power output of the decay heat is enough to cause problems.
- epgui 5y ago(Note that you can drink a lot of pure heavy water without worrying about any health impact. I still wouldn't recommend it, but bad things would be unlikely to happen by accident.)
- retrac 5y agoThere are no proliferation concerns with fusion. You can't make a nuclear bomb with a fusion reactor. They'll be largely free of the nuclear materials and technologies handling and export regulations. Quite a few international deals for fission plants over the years have been scuttled over concerns about misapplication of the reactors or fuel.
- melony 5y agoThere actually is proliferation concerns, maybe even worse.
- remarkEon 5y agoIn what way? Are you implying e.g. materials science advances will lead to non-nuclear weapons proliferation?
- KiwiJohnno 5y agoCan you elaborate? Fusion reactors don't produce any fissionable material.
- ncmncm 5y agoOne is not obliged to turn ordinary, mined uranium into weapons-grade stuff, in a neutron-emitting fusion system, but you could if you cared to. I.e., a nation that had control of such a fusion plant would have little difficulty attaching an enrichment process without interfering with power output. Fortunately, no economically practical power generation system can be built using hot-neutron fusion, so it is an idle concern, but almost all the money being spent on fusion pursues that impractical goal.
- willis936 5y agoIt's an idle concern because fast neutron sources are cheaply available without fusion reactors.
- 317070 5y agoAll of these points are up for debate of course, but: * compared to fission, the nuclear waste management for fusion looks to be done within a human generation, rather than outlast human civilisation. So how you discount the future has a large impact on the trade-off between fusion and fission. * the fuel for fission is uranium, a hard to get and limited resource. The fuel for fusion is hydrogen. Initially only the rarer hydrogen isotopes, but we might eventually get fusion to work for the more common isotopes as well.
- duvenaud 5y agoOkay, but is the fuel a major cost for fission plants? And it's not clear to me how nuclear waste can both be extremely long-lived and also very dangerous that entire time.
- 317070 5y agoFuel is not a major cost right now, but it inevitably will be at some point. Current estimates are that we have 80-250 years of uranium left [0], so we could use a different energy source for the long run. The difference in radioactive danger, is because not all nuclear reactions are treated equally. Tritium decays once, emitting a 0.019MeV beta particle [1]. Uranium creates an avalanche of particles [2] for a total of 51.7 MeV. So it has the leeway to be both more radioactive and to be it for longer. But both the type and energy of the radiation have intricate effects on how they interact with biology and so how dangerous they really are. That said, exactly how radioactive fusion waste will be is a bit a philosophical problem, as nobody knows what the minimal requirements are for a functional plant. [0] https://bettermeetsreality.com/how-much-uranium-is-left-in-the-world-on-land-in-oceans-when-will-we-run-out/ https://bettermeetsreality.com/how-much-uranium-is-left-in-t... [1] en.m.wikipedia.org/wiki/Tritium [2] https://en.m.wikipedia.org/wiki/Decay_chain https://en.m.wikipedia.org/wiki/Decay_chain
- flaviut 5y agoFYI the bettermeetsreality site you link to is SEO spam. But anyway, those numbers sound about right, here's what wikipedia (and their source, https://doi.org/10.1787/uranium-2018-en https://doi.org/10.1787/uranium-2018-en) says: > As of 2017, identified uranium reserves recoverable at US$130/kg were 6.14 million tons (compared to 5.72 million tons in 2015). At the rate of consumption in 2017, these reserves are sufficient for slightly over 130 years of supply. The identified reserves as of 2017 recoverable at US$260/kg are 7.99 million tons (compared to 7.64 million tons in 2015). I wouldn't be too worried right now though, because Table 1.1 in uranium-2018 shows that the known recoverable uranium sources @ $40/kgU grew 50% between 2015 and 2017, and known recoverable uranium sources @ $80/kgU grew 4.6% over the same time period. No one's going to go around prospecting unless they think they can make a profit doing it.
- axg11 5y agoAs the other commenter mentioned, fuel is limitless and the waste product doesn’t need special disposal. This isn’t just a technical advantage over fission, but also a political one. In its current form though, the CapEx for fusion projects is huge. Fusion won’t play a big part as an energy source if reactors take 10+ years to come online and cost tens of billions of dollars. Fusion proponents will argue that costs will come down, but if the political and entrepreneurial pressure to reduce cost isn’t there, fusion will end up the same as fission.
- willis936 5y agoTo tack on to the last sentence: "as long as there is fossil fuel to burn".
- duvenaud 5y agoThanks. So it sounds like you're saying that, unless there is some radically different plant design in the future, that there won't be an economic advantage to fusion over fission, at least in the near term.
- colinmhayes 5y agoWell fission energy would be much cheaper if most of our energy was produced that way. Economies of scale provide huge opportunities that are currently limited by political will. I don't see a reason fusion can't quickly become a large scale source of power, because unlike fission, there would be much less political backlash.
- sanxiyn 5y agoI am sad to report that in South Korea, which produces fission power very well and does excellent fusion research, the leading anti-nuclear activist (also a member of congress) is vocally against fusion and already proposed to cut 100% of fusion research budget in 2018. (People thought that's too extreme so she haven't repeated it so far, but I don't think her position changed in any way.)
- conradev 5y agoHelion is trying to capture the energy directly using the magnetic confinement field instead of a heat-driven turbine I am not sure if any of the other designs avoid the turbine
- KiwiJohnno 5y agoA completely different approach to fusion that may or may not work but can do direct generation of electricity without a turbine is pollywell fusion
- ncmncm 5y agoBut, even less is being spent on that than on other aneutronic designs, which themselves are on shoestring budgets. Theoretical reasons make it seem like an unpromising avenue.
- willis936 5y agoConfinement approaches aren't inherently neutronic or aneutronic. What matters is the temperature, desnity, and confinement time (lawson criterion). Get it sufficiently high then DT becomes a breeze. Get it a factor of 500 higher and pB11 is on the table. No one serious talks about aneutronic fusion because we need to walk 1 mph before we sprint at 500 mph. We'll seriously discuss aneutronic fusion in 200 years when it's relevant.
- willis936 5y agoWhich will never work because of conduction losses in the coils. You can't just put solid materials inside your confinement volume and expect to make power.
- deleted 5y ago[deleted]
- anonymousDan 5y agoI think the main advantage of fusion is they are not susceptible to Chernobyl style meltdowns line fission reactors.
- klabb3 5y agoAFAIU this is entirely simulated - if it turns out the simulation doesn't perfectly match reality, what are the challenges in re-adjusting? Does the RL network need to be re-trained with massive amounts of data on a new improved simulation or can it be fine tuned on the real thing?
- kevinqi 5y agoThat's a great question; I'm curious about this as well.
- willis936 5y agoThis is why fundamental plasma physics is so valuable to fund. ML can come up with good models that fit the data in the domains that there are empirical datasets to train against, but it can't reliably extrapolate outside of those domains. So this is a good tool for the computationalists but we still need the experimentalists. This is science.
- ShamelessC 5y agoTo clarify, this result did not have the benefit of a large labeled dataset. They used a simulator and got working results without _any_ finetuning on data measured from the real world. > So this is a good tool for the computationalists but we still need the experimentalists. This is science. I would say that this is a good example of collaboration between the groups.
- deleted 5y ago[deleted]
- starwind 5y agoProbably massive amounts of data but it depends on how far off the simulation is from reality and the expected cost of collecting that data vs running the simulation a few billion times. Not far off and you want perfect results? Go with actual data. Very far off where it's not even usable? You'd need to go back to the training environment and redo the reward function. Generally speaking (and I'm not sure if this is the case from reading the link but I may have missed it) reinforcement learning for optimal control is done to fine tune while traditional control methods are used for coarse adjustments. Since this is deepmind they probably want to use RL for everything
- deleted 5y ago[deleted]
- dekhn 5y agoI worked on a previous project that was sort of an ancestor to this. We used grid computing (before "cloud" was even a thing) to rapidly scale up compute- right after an experiment, you want to recalculate the optimal parameters based on what you just learned from the experiment and then start over as fast as possible. This was well before anybody could say "reinforcement learning" or "neural network" around physicists without being pilloried. The best part was the name of our collaborator- "General Atomics" https://en.wikipedia.org/wiki/General_Atomics https://en.wikipedia.org/wiki/General_Atomics Here's a writeup, https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.588.150&rep=rep1&type=pdf https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.58...
- shmageggy 5y agoSorry, I don't see how this has any similarity to the posted project, apart from being also related to fusion?
- dekhn 5y agoI'm just an old man, pointing out that it's all been done before, and it will all be done again. The big difference is it looks like they're in the realtime controller for the plasma, rather than determining static settings for a subsequent run.
- regitempus 5y agoThis is super cool, thanks for sharing. Also, you’re right, that name is dope.
- systemvoltage 5y agoSilicon valley would name it, ahem... Atomicly. Uber for atoms.
- iSnow 5y agoSince the .com domain would be gone, it would most likely be Atomcly
- pontifier 5y ago
- deleted 5y ago[deleted]
- ClassAndBurn 5y agoAnd suddenly the plot of one of the better Spiderman movies was happening in real life.
- equivocates 5y agoPretty sure this was the plot of Spider-Man 2 (Raimi Trilogy).
- teraflop 5y agoVia SF&F Stack Exchange, a short story from 1990 with a similar premise: https://scifi.stackexchange.com/questions/260530/ai-controlling-fusion-reactor https://scifi.stackexchange.com/questions/260530/ai-controll...
- aqme28 5y agoTechnically also the plot of Spider-Man 2. Controlling a burning plasma with an AI was Doc Oc’s field of study.
- AceJohnny2 5y ago(offtopic) I'm reminded of Alastair Reynolds' sci-fi short story "Weather" <spoilers follow> The twist of the story is that the alien/incomprehensible starship drives are controlled by an embedded, augmented, disembodied human brain. I've long felt the gimmick was a bit silly because what system would require an actual human brain over a normal control algorithm. Well...
- mLuby 5y agoSaw the headline, immediately thought of the same short story. Excerpts where they encounter said brain: "It's a monstrosity." "Not to us," she said sharply. "We see a thing of wonder and beauty." "No," I said firmly. "Let's be clear about this. What you're showing me here is a human brain, a living mind, turned into some kind of slave." "No slavery is involved," Weather said. "The mind chose this vocation willingly." "It *chose* this?" "It's considered a great honour. Even in [our] society, even given all that we have learned about the maximization of our mental resources, on a few are ever born who have the skills necessary to tame and manage the reactions in the heart of a [star]-drive." … "The degree of concentration is quite intense. He can barely spare any resources for what we might call normal thought. He's in a state of permanent unconscious flow, like someone engaged in an enormously challenging game. But now the game has begun to get the better of him. It isn't fun anymore. And yet he knows the cost of failure." … "It'll be more like one very long dream. Someone else's dream, certainly, but I don't doubt that there'll be a certain rapturous quality to it. I remember how good it felt to find an elegant solution, when the parameters looked so unpromising. Like making the most beautiful music imaginable. I don't think anyone can really know how that feels unless they've also held some of that fire in their minds. It's ecstasy, Inigo, when it goes right." "And when it goes wrong?" Definitely worth reading.
- nicholast 5y agoThe root enabling factor was not only the reinforcement learning, it was the simulation. Put differently, any environment that you can simulate within sufficient accuracy, you can derive robust and efficient control by reinforcement learning.
- EGreg 5y agoHow to simulate human humor responses? Then we can evolve AIs that are funnier and more resourceful than anyone on the planet. How to simulate human connection, attraction and arousal responses? Then we could have bots that are smoother talking and far more preferable to human lovers. How to simulate human responses to nutrition? Then we could have individualized recommendations for diets, achieving goals like holistic health or muscle definition and addressing issues like diabetes.
- ksubedi 5y agoThese are all complex systems where figuring out how to simulate them is a much difficult task than even reinventing reinforcement learning from scratch.
- avs733 5y agoI'm probably being excessively pedantic but I would suggest sufficient fidelity is a more appropriate phrase than sufficient accuracy. The exact precision doesn't seem to be the underlying problem being solve, what matters is being able to craft a stronger understanding of relationships of underlying phenomena that can then be leveraged to product new control theories.
- adrianmonk 5y agoI think they are just talking about how accurately the model reflects the real world. If someone draws a map and it has some streets with the wrong names, you say, "This map is not accurate." It doesn't mean you are talking about something numerical.
- 5y ago
- MathMonkeyMan 5y agoIsn't this the plot of Spiderman 2? There the network used was Doc Ock's brain.
- cma 5y agoAnother interesting approach with some stuff on near fusion the end: Steve Brunton - Sparse Nonlinear Models for Fluid Dynamics with Machine Learning and Optimization https://www.youtube.com/watch?v=z_CZ_VyMDXE https://www.youtube.com/watch?v=z_CZ_VyMDXE
- manholio 5y agoToo little, too late. The traditional fission industry is being killed by high CAPEX and nothing about the current tokamak based designs suggests they will be any cheaper to build for a given power rating: - They need large, highly advanced cryocooled superconducting magnets in very close proximity to a hundred million degree plasma. This only makes economic sense in massive, and expensive plants. - They are a very strong source of fast neutrons useful to transmute cheap depleted uranium into plutonium, so carry massive proliferation risks, need close regulatory scrutiny and will require mounts of paperwork to operate, thus exceptionally inflexible to improvements and rapid iteration. Just like the current fission crop. - Aneutronic fusion is a currently a purely theoretical concept, in the last 70 years nobody has been able to contain even the much cooler D-T plasma for economically viable durations and temperatures. - The structure of the reactor becomes radiologically active and cleanup operations must be considered. Highly penetrating neutron radiation means some radiation will escape regardless of containment, requiring a radiological exclusion zone. No Mr. Fusion in your car, sorry. - They operate and must breed sensitive nuclear materials - Tritium, a well known component of boosted thermonuclear weapons. The limited efficiency of tritium production from lithium-6 might require obtaining some from fission reactors to top up the fuel cycle and keep fusion reactors operating. So when you draw the line, a life time of magnetic containment research has produced a speculative design that even if it were to work, which it doesn't, would be, in the best case scenario, comparable to existing fission designs that are being phased out for cost and risk issues. A PhD money pit with zero chance of ever building anything useful. I know it, you know it, I'm ready for the proverbial downvote protecting one's paycheck from their own knowledge.
- Izikiel43 5y agoCheck Sparc before all that stuff, iter is outdated but there are alternatives being worked on.
- manholio 5y agoWhen SPARC ignites, 5 years from now, and when it achieves long burns at Q=10, after a good few more years, then I will redact the "nobody has been able to contain D-T plasma for economically viable durations and temperatures" part. The rest of the points will probably remain true decades into the future, though.
- hamilyon2 5y agoNo-no-no! Don't put machine learning and nuclear power together. Bad idea from all sides.
- willis936 5y agoAre you scared of a computer making a physically impossible chain reaction?
- vixen99 5y agoChris Bishop and others were working on this with neural nets for the Tokamak at Harwell back in the 1980s which illustrates just how difficult this problem is.