S3-Tracker: Self-Supervised Surgical Tissue Tracking With Contrastive Random Walks

Abstract:Robust point tracking in endoscopic videos is essential for computer-assisted intervention and autonomous robotic surgery, enabling continuous registration between intraoperative video and preoperative imaging despite soft tissue deformation. However, supervised tracking methods depend on large annotated datasets, while surgical conditions make reliable trajectory annotation challenging. We propose a self-supervised Track-Any-Point approach that learns from unlabeled surgical videos by establishing global pixel correspondences and inferring point trajectories through contrastive random walks. Trained without annotations, our method achieves performance comparable to existing semi-supervised approaches while implicitly handling tissue deformation. These findings demonstrate the feasibility of self-supervised point tracking in surgical environments and its potential to reduce reliance on annotated data.

Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Robotics (cs.RO)

arXiv:2609.14313 [cs.CV]

(or arXiv:2609.14313v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2609.14313 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

J. Zhang et al. Proc. 18th Hamlyn Symp. Med. Robot. (2026) 25-26;

https://doi.org/10.31256/HSMR26.13 Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.