FLOP Around and Find Out: LLM Training Workload Size Estimation With Power Monitoring

Abstract As the capabilities of frontier AI systems rapidly advance, there is growing interest in a multilateral agreement between nations to mitigate the risks posed by future, powerful systems. Any such agreement will require a quantifiable definition of what counts as a "frontier" model. Existing legislation, like the EU AI Act and California's SB 53, uses the number of floating-point operations (FLOPs) in the training run as the threshold for which models count as frontier, and proposals for international agreements extend this framing. Today, these thresholds rest entirely on AI developers' self-reports. An international agreement between rival nations cannot assume compliance, so a treaty will be contingent on a technical means of estimating the number of FLOPs in a training run that can be performed by a verification team independent from the AI developers. In this work, I stress-test FLOP estimation on a dual-GPU Nvidia V100 node to develop a minimum viable product (MVP) for how FLOP estimation could work in a real data center. Following prior experiments on an Nvidia Jetson Orin Nano, a verifier ("blue team") with system-level access, but no visibility into workload code, estimates training FLOPs from external signals: GPU power draw, GPU memory bandwidth utilization, and GPU interconnect traffic. Calibrated against sample LLM pre-training workloads spanning 88 hyperparameter configurations, the estimator achieves 10.4% median absolute error on held-out workloads against a ground truth computed by PyTorch's FlopCounterMode library. Then, assuming the role of a non-compliant party, I design and run adversarial workloads that increase the estimator's median error to ~25% under the strongest strategy, with the worst configuration under-reporting by 41%. The result is an MVP for adversarially-tested FLOP verification. Introduction Motivation For want of a quantifiable way to decide what counts as a frontier AI model, compute thresholds have emerged as the standard for AI policy: California's SB 53 uses mjx-math { display: inline-block; text-align: left; line-height: 0; text-indent: 0; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; border-collapse: collapse; word-wrap: normal; word-spacing: normal; white-space: nowrap; direction: ltr; padding: 1px 0; } mjx-container[jax="CHTML"][display="true"] { display: block; text-align: center; margin: 1em 0; } mjx-container[jax="CHTML"][display="true"][width="full"] { display: flex; } mjx-container[jax="CHTML"][display="true"] mjx-math { padding: 0; } mjx-container[jax="CHTML"][justify="left"] { text-align: left; } mjx-container[jax="CHTML"][justify="right"] { text-align: right; } mjx-msup { display: inline-block; text-align: left; } mjx-mn { display: inline-block; text-align: left; } mjx-c { display: inline-block; } mjx-utext { display: inline-block; padding: .75em 0 .2em 0; } mjx-TeXAtom { display: inline-block; text-align: left; } mjx-mtext { display: inline-block; text-align: left; } mjx-c.mjx-c31::before { padding: 0.666em 0.5em 0 0; content: "1"; } mjx-c.mjx-c30::before { padding: 0.666em 0.5em 0.022em 0; content: "0"; } mjx-c.mjx-c32::before { padding: 0.666em 0.5em 0 0; content: "2"; } mjx-c.mjx-c36::before { padding: 0.666em 0.5em 0.022em 0; content: "6"; } mjx-c.mjx-c35::before { padding: 0.666em 0.5em 0.022em 0; content: "5"; } mjx-container[jax="CHTML"] { line-height: 0; } mjx-container [space="1"] { margin-left: .111em; } mjx-container [space="2"] { margin-left: .167em; } mjx-container [space="3"] { margin-left: .222em; } mjx-container [space="4"] { margin-left: .278em; } mjx-container [space="5"] { margin-left: .333em; } mjx-container [rspace="1"] { margin-right: .111em; } mjx-container [rspace="2"] { margin-right: .167em; } mjx-container [rspace="3"] { margin-right: .222em; } mjx-container [rspace="4"] { margin-right: .278em; } mjx-container [rspace="5"] { margin-right: .333em; } mjx-container [size="s"] { font-size: 70.7%; } mjx-container [size="ss"] { font-size: 50%; } mjx-container [size="Tn"] { font-size: 60%; } mjx-container [size="sm"] { font-size: 85%; } mjx-container [size="lg"] { font-size: 120%; } mjx-container [size="Lg"] { font-size: 144%; } mjx-container [size="LG"] { font-size: 173%; } mjx-container [size="hg"] { font-size: 207%; } mjx-container [size="HG"] { font-size: 249%; } mjx-container [width="full"] { width: 100%; } mjx-box { display: inline-block; } mjx-block { display: block; } mjx-itable { display: inline-table; } mjx-row { display: table-row; } mjx-row > * { display: table-cell; } mjx-mtext { display: inline-block; } mjx-mstyle { display: inline-block; } mjx-merror { display: inline-block; color: red; background-color: yellow; } mjx-mphantom { visibility: hidden;