A Theory of the Mechanics of Information: Generalization Through Measurement of Uncertainty (Learning is Measuring)

arXiv:2510.22809v2 Announce Type: replace Abstract: Traditional machine learning relies on explicit models and domain assumptions, often limiting flexibility and interpretability. We introduce a "model-free" framework of instance-based learning using surprisal (information theoretic uncertainty) to directly analyze and perform inferences from raw data, eliminating distribution modeling, reducing bias, and enabling efficient updates including direct edits and deletion of training data. By quantifying relevance through uncertainty, the approach enables generalizable inference across tasks including generative inference, causal discovery, anomaly detection, and time series forecasting. Our approach emphasizes traceability, interpretability, and data-driven decision making, offering a unified, human-understandable framework for machine learning, can be adapted to virtually all machine learning tasks, achieving competency across all and state-of-the-art performance on anomaly detection and some supervised tasks. The mathematical foundations create a "physics" of information, which enable these techniques to apply effectively to a wide variety of complex data types, including missing data. Our results suggest that this may be a viable foundation upon which to build scalable machine learning and artificial intelligence that can maintain human understandability of the underlying mechanics.

Abstract:Traditional machine learning relies on explicit models and domain assumptions, often limiting flexibility and interpretability. We introduce a "model-free" framework of instance-based learning using surprisal (information theoretic uncertainty) to directly analyze and perform inferences from raw data, eliminating distribution modeling, reducing bias, and enabling efficient updates including direct edits and deletion of training data. By quantifying relevance through uncertainty, the approach enables generalizable inference across tasks including generative inference, causal discovery, anomaly detection, and time series forecasting. Our approach emphasizes traceability, interpretability, and data-driven decision making, offering a unified, human-understandable framework for machine learning, can be adapted to virtually all machine learning tasks, achieving competency across all and state-of-the-art performance on anomaly detection and some supervised tasks. The mathematical foundations create a "physics" of information, which enable these techniques to apply effectively to a wide variety of complex data types, including missing data. Our results suggest that this may be a viable foundation upon which to build scalable machine learning and artificial intelligence that can maintain human understandability of the underlying mechanics.

Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Statistics Theory (math.ST); Machine Learning (stat.ML)

arXiv:2510.22809 [cs.LG]

(or arXiv:2510.22809v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2510.22809 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

A Theory of the Mechanics of Information: Generalization Through Measurement of Uncertainty (Learning is Measuring)

Related coverage