analiza aiqbench
This research introduces scKITE, a single-cell foundation model that leverages biological knowledge—such as cell annotations and gene regulatory data—to achieve superior performance with significantly less training data than current state-of-the-art models. By integrating these constraints into a Transformer encoder via lightweight auxiliary decoders, the approach demonstrates that quality-driven scaling can overcome the diminishing returns of raw data expansion in genomics. This paradigm shift suggests a more efficient path forward for biologically grounded AI models without relying on massive computational resources.
Towards a knowledge-enhanced single-cell foundation model
arXiv:2609.14970v1 Announce Type: new Abstract: Single-cell foundation models (scFMs) increasingly rely on large-scale transcriptomic pretraining, yet expanding pretraining data can yield diminishing gains while substantially increasing computational cost. Our data scaling analyses showed that incorporating biological knowledge, including cell-level text annotation and gene-level regulatory information, provided additional scaling dimension than simply increasing data size. Motivated by this observation, we present scKITE, a simple yet effective scFM that integrates cell-annotation and gene-regulatory supervision into a shared transcriptomic Transformer encoder through lightweight auxiliary decoders. These decoders are used only during pretraining and subsequently discarded, yielding a general-purpose encoder enriched with biological knowledge for downstream applications. With only 179,067 pretraining samples, i.e., less than 0.5\% of those used by previous strong scFMs, scKITE outperformed these models across diverse downstream tasks, highlighting knowledge-enhanced pretraining as a promising paradigm for biologically grounded scFMs.
Abstract:Single-cell foundation models (scFMs) increasingly rely on large-scale transcriptomic pretraining, yet expanding pretraining data can yield diminishing gains while substantially increasing computational cost. Our data scaling analyses showed that incorporating biological knowledge, including cell-level text annotation and gene-level regulatory information, provided additional scaling dimension than simply increasing data size. Motivated by this observation, we present scKITE, a simple yet effective scFM that integrates cell-annotation and gene-regulatory supervision into a shared transcriptomic Transformer encoder through lightweight auxiliary decoders. These decoders are used only during pretraining and subsequently discarded, yielding a general-purpose encoder enriched with biological knowledge for downstream applications. With only 179,067 pretraining samples, i.e., less than 0.5\% of those used by previous strong scFMs, scKITE outperformed these models across diverse downstream tasks, highlighting knowledge-enhanced pretraining as a promising paradigm for biologically grounded scFMs.
Artificial Intelligence (cs.AI); Genomics (q-bio.GN)
arXiv:2609.14970 [cs.AI]
(or arXiv:2609.14970v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.14970 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 229 cuvinte
Subiecte conexe
Integrating gene regulatory priors into Transformer attention with scTransformer for interpretable scRNA-seq analysis
arXiv · 9 iun. 2026
Assessing Information Ecosystems : How Governments Can Get Ahead of Hybrid Threats
carnegieendowment.org · 5 aug. 2026
NVIDIA and Japan Bring Full-Stack AI and Robotics to Every Industry
NVIDIA · 15 iul. 2026
Digital States 2026: AI Moves From Experiment to Infrastructure
govtech.com · 24 sept. 2026