SurgicalMamba: Dual-Path SSD with State Regramming for Online Surgical Phase Recognition

Abstract:Online surgical phase recognition must commit to a prediction at every frame of a procedure that runs for hours, from past frames alone and at a per-frame cost that does not grow with elapsed length. Structured state-space duality (SSD) meets that constraint, but only by having the scan see a per-head scalar transition, which fixes both where the state puts a frame and how fast it decays. The same views recur through an operation, so repeated content is written over itself and can afterwards be told apart only by age. How fast to decay is left to the step, and when the past stops being useful has to be inferred from a loss that never marks the moment. Procedures run long and change little visually from frame to frame, leaving the step with little to select on. Phases also vary widely in length, so no fixed rate serves as a fallback. We address the two with two mechanisms. State regramming rotates the carried state at each chunk boundary, by an amount the chunk's content decides, so where a frame is written also depends on what has passed since: two occurrences of the same view are held apart when different phases intervene, which no decay rate can achieve once both have aged. Intensity-modulated stepping increases the decay at the annotated phase transitions, so the state empties quickly where a phase ends and slowly in between and the decay itself can be set for the longest phase. Both leave SSD's N-semiseparable structure and O(d) per-frame cost intact. Across seven public benchmarks SurgicalMamba reaches state-of-the-art online accuracy and phase-level Jaccard (94.6%/82.7% on Cholec80, 89.5%/68.9% on AutoLaparo) at 312.88 fps on a single GPU. Adding the rotation alone to a plain Mamba2 improves multi-query associative recall (MQAR) wherever the recurrent state is the binding constraint, indicating that the mechanism is not specific to surgical video.

Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

I.2.10; I.4.8; J.3

arXiv:2605.14889 [cs.CV]

(or arXiv:2605.14889v4 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2605.14889 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.