aiqbench analysis
This research demonstrates that the format of supervision data critically impacts how large language models update medical knowledge, with EMQ formats offering superior stability and contrast signals in oncology staging tasks. By introducing SEER-Bench, the study provides a rigorous benchmark for evaluating model retention against evolving clinical guidelines without increasing training budgets. These findings highlight that optimizing knowledge structure is as vital as algorithmic changes for deploying reliable AI in dynamic healthcare environments.
Dense Clinical Contrasts Enhance Medical Knowledge Updating in Large Language Models
arXiv:2608.30405v1 Announce Type: new Abstract: Medical knowledge changes continually, making large language models vulnerable to relying on outdated yet clinically plausible information. We study whether the format of supervision affects medical knowledge updating under a matched training-budget setting. We introduce SEER-Bench, a temporally anchored oncology-staging benchmark curated from the latest versioned SEER Research Data release, and render identical medical update events from NCCN oncology guidelines into four supervision formats: EMQ, MSQ, FITB, and SAQ. Across SEER-Bench and HealthBench Professional, EMQ gives the most stable external transfer and retention among same-budget SFT variants. With EMQ supervision, the updated 4B model produces competitive results on temporally anchored oncology staging, reaching 64.8% answer accuracy and 59.6% rationale accuracy on SEER-Bench. Diagnostic analyses suggest that EMQ exposes denser clinical contrast signals while preserving discriminative representations with smaller movement from the base model. These results show that medical knowledge updating depends not only on the update algorithm, but also on how knowledge is structured as supervision.
Abstract:Medical knowledge changes continually, making large language models vulnerable to relying on outdated yet clinically plausible information. We study whether the format of supervision affects medical knowledge updating under a matched training-budget setting. We introduce SEER-Bench, a temporally anchored oncology-staging benchmark curated from the latest versioned SEER Research Data release, and render identical medical update events from NCCN oncology guidelines into four supervision formats: EMQ, MSQ, FITB, and SAQ. Across SEER-Bench and HealthBench Professional, EMQ gives the most stable external transfer and retention among same-budget SFT variants. With EMQ supervision, the updated 4B model produces competitive results on temporally anchored oncology staging, reaching 64.8% answer accuracy and 59.6% rationale accuracy on SEER-Bench. Diagnostic analyses suggest that EMQ exposes denser clinical contrast signals while preserving discriminative representations with smaller movement from the base model. These results show that medical knowledge updating depends not only on the update algorithm, but also on how knowledge is structured as supervision.
Artificial Intelligence (cs.AI)
arXiv:2608.30405 [cs.AI]
(or arXiv:2608.30405v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2608.30405 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 250 words
Related coverage
Introducing Instella-MoE: A State-of-the-Art Fully Open Mixture-of-Experts Language Model
AMD · Jul 24, 2026
Knowledge Graph Integration With Poro2 For Enriching Medical Text Processing
AMD · Sep 15, 2026
MSM-Mem: A Universal Medical Structured Multimodal Memory Framework for Medical AI Agents
arXiv · Aug 25, 2026
Surgical Alignment in Knowledge Graph Training for Clinical Diagnosis with Large Language Models
arXiv · Aug 28, 2026