analiza aiqbench
This research introduces the Propensity Straight-Through (PST) estimator, a novel gradient-based method that enables efficient training of discrete stochastic systems modeled by continuous-time Markov chains. By differentiating exact conditional means rather than relying on relaxation techniques like Gumbel-Softmax, PST achieves faster convergence and high accuracy across benchmarks ranging from genetic oscillators to large-scale reaction networks. The approach offers a robust path for integrating hard sampling into deep learning workflows without introducing temperature-dependent biases or approximation errors.
Propensity Straight-Through Gradients for Discrete Stochastic Systems
arXiv:2608.25631v1 Announce Type: cross Abstract: Continuous-time Markov chains (CTMCs) provide the backbone for modeling discrete stochastic dynamics across applied, physical, and biological sciences. Their integration with modern gradient-based machine learning, however, is limited by the hard categorical event selection intrinsic to Gillespie-type simulation algorithms. We exploit the affine state update to obtain the exact one-step conditional-mean sensitivity by differentiating normalized reaction propensities. We pair this backward rule with exact forward trajectories to define the propensity straight-through (PST) estimator. At the trajectory level, we show that one-step sensitivities composed across events can depart from the exact multistep sensitivity. We derive the resulting per-step discrepancy in closed form and prove that it vanishes identically for affine downstream dependence. PST matches the accuracy of Gumbel-Softmax straight-through across all benchmarks: reversible dimerization (0.06% error), a genetic oscillator (1.7% error), a 50-task repressilator suite (0.17% median error), and patch-clamp ion-channel recordings ($R^2$ = 0.988). Under matched settings, PST converges 3.0-fold faster on the oscillator and 2.1-fold faster on the ion channel. At deep-learning scale, PST trains a 203,796-parameter stochastic reaction network with hard sampling, reaching 98.22% MNIST digit classification accuracy. By differentiating an exact conditional mean rather than a relaxed sample, PST offers a temperature- and Gumbel-free path to scalable gradient-based learning through exact stochastic trajectories.
Abstract:Continuous-time Markov chains (CTMCs) provide the backbone for modeling discrete stochastic dynamics across applied, physical, and biological sciences. Their integration with modern gradient-based machine learning, however, is limited by the hard categorical event selection intrinsic to Gillespie-type simulation algorithms. We exploit the affine state update to obtain the exact one-step conditional-mean sensitivity by differentiating normalized reaction propensities. We pair this backward rule with exact forward trajectories to define the propensity straight-through (PST) estimator. At the trajectory level, we show that one-step sensitivities composed across events can depart from the exact multistep sensitivity. We derive the resulting per-step discrepancy in closed form and prove that it vanishes identically for affine downstream dependence. PST matches the accuracy of Gumbel-Softmax straight-through across all benchmarks: reversible dimerization (0.06% error), a genetic oscillator (1.7% error), a 50-task repressilator suite (0.17% median error), and patch-clamp ion-channel recordings ($R^2$ = 0.988). Under matched settings, PST converges 3.0-fold faster on the oscillator and 2.1-fold faster on the ion channel. At deep-learning scale, PST trains a 203,796-parameter stochastic reaction network with hard sampling, reaching 98.22% MNIST digit classification accuracy. By differentiating an exact conditional mean rather than a relaxed sample, PST offers a temperature- and Gumbel-free path to scalable gradient-based learning through exact stochastic trajectories.
Quantitative Methods (q-bio.QM); Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (cs.LG); Computational Physics (physics.comp-ph); Molecular Networks (q-bio.MN)
arXiv:2608.25631 [q-bio.QM]
(or arXiv:2608.25631v1 [q-bio.QM] for this version)
Text integral sub licența deschisă a publicației · 232 cuvinte
Subiecte conexe
Gradient estimators for parameter inference in discrete stochastic kinetic models
arXiv · 5 iun. 2026
Descent-Guided Policy Gradient for Scalable Cooperative Multi-Agent Learning
arXiv · 30 sept. 2026
Boundary-aware Reinforcement Learning for Hypercube State Spaces via Deterministic Policy Gradient
arXiv · 8 oct. 2026
Quantum-hardware spectral co-design framework for multifrequency Rydberg electrometry
arXiv · 11 sept. 2026