aiqbench analysis
This research applies a goal-conditioned reinforcement learning controller to automate the complex scenario development process in tokamak fusion reactors, replacing iterative trial-and-error tuning with a single pre-trained neural network. By successfully managing plasma current ramps and divertor configurations across hundreds of test scenarios, the study demonstrates how advanced ML techniques can optimize control loops for high-dimensional physical systems.
Reinforcement Learning Controllers for Fusion Scenario Development
arXiv:2610.26138v1 Announce Type: cross Abstract: Scenario development, the design of the waveforms that take a tokamak discharge from breakdown to flat-top, is still an iterative, labour-intensive process. Every candidate scenario has to be simulated with a controller in the loop, and the classical shape and current controllers need low-level actuator references. So the process is iterative and includes a number of trial-and-error steps. We propose to replace the per-scenario controller tuning by a single goal-conditioned reinforcement-learning controller trained once, in advance, against a free-boundary equilibrium and transport simulator. A scenario is then specified only by the target waveforms of the physical quantities of interest (plasma current, line-averaged electron density, LCFS shape, x-point positions), and the controller produces the coil-current and fueling commands that realize them. We demonstrate the approach on an EAST plasma-current ramp-up: the controller takes over a small limited plasma, ramps the current and the density, grows the cross-section, forms two x-points and holds a double-null diverted configuration against sampled shape targets for the rest of a 1.2 s episode. The neural network controller is trained using TQC algorithm. On 1024 test scenarios sampled from a low-discrepancy scrambled Sobol sequence in 11-dimensional target space, the controller forms the divertor in 86% of the cases, holds it for more than 0.5 s in 67% and reaches the end of the episode in a diverted state in 58%, with flat-top tracking errors of a few kA in plasma current, $\sim 0.15\times10^{19}$ m$^{-3}$ in density, $\sim 2$ cm in position and $\sim 8$ cm in x-point location; the failures concentrate at the high-current density ($J$) region which is believed to be physically or from engineering perspective not feasible within ohmic mode.
Abstract:Scenario development, the design of the waveforms that take a tokamak discharge from breakdown to flat-top, is still an iterative, labour-intensive process. Every candidate scenario has to be simulated with a controller in the loop, and the classical shape and current controllers need low-level actuator references. So the process is iterative and includes a number of trial-and-error steps. We propose to replace the per-scenario controller tuning by a single goal-conditioned reinforcement-learning controller trained once, in advance, against a free-boundary equilibrium and transport simulator. A scenario is then specified only by the target waveforms of the physical quantities of interest (plasma current, line-averaged electron density, LCFS shape, x-point positions), and the controller produces the coil-current and fueling commands that realize them. We demonstrate the approach on an EAST plasma-current ramp-up: the controller takes over a small limited plasma, ramps the current and the density, grows the cross-section, forms two x-points and holds a double-null diverted configuration against sampled shape targets for the rest of a 1.2 s episode. The neural network controller is trained using TQC algorithm. On 1024 test scenarios sampled from a low-discrepancy scrambled Sobol sequence in 11-dimensional target space, the controller forms the divertor in 86% of the cases, holds it for more than 0.5 s in 67% and reaches the end of the episode in a diverted state in 58%, with flat-top tracking errors of a few kA in plasma current, $\sim 0.15\times10^{19}$ m$^{-3}$ in density, $\sim 2$ cm in position and $\sim 8$ cm in x-point location; the failures concentrate at the high-current density ($J$) region which is believed to be physically or from engineering perspective not feasible within ohmic mode.
Plasma Physics (physics.plasm-ph); Machine Learning (cs.LG)
arXiv:2610.26138 [physics.plasm-ph]
(or arXiv:2610.26138v1 [physics.plasm-ph] for this version)
https://doi.org/10.48550/arXiv.2610.26138 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 370 words
Related coverage
Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping
arXiv · Jul 28, 2026
Certifiable Safe Model-Based Reinforcement Learning with Control-Affine Dynamics Approximation
arXiv · Jul 21, 2026
Reinforcement Learning for Robust Calibration of Multi-Qudit Quantum Gates
arXiv · Aug 27, 2026
Offline Reinforcement Learning for Plasma Control in Nuclear Fusion: Codebase and Benchmark
arXiv · Jun 9, 2026