Finite Certificates for In-Context Determinacy and a Threshold Theory of Emergence in Language Models

Abstract:This paper develops a model-theoretic framework for verifying context-conditioned language-model behavior by replacing benchmark labels with finite semantic certificates. The first problem is finite determinacy: when do examples in a context force the answer to a query without changing model parameters? In finite-field linear task families, we prove an exact row-space criterion, compute the residual hypothesis count, derive full and query-local identification curves, and show that extracting a smallest forcing subcontext is NP-complete even for binary outputs. The second problem is threshold emergence: when does an apparent benchmark jump reflect a semantic transition rather than a discontinuity of the scoring map? We prove an anti-mirage theorem separating thresholded metrics from semantic confidence and give a rate-sensitive crossing bound for latent commitments becoming visible above threshold. The common semantic object is a confidence functional on definable events. We show that it is a Boolean probability measure, equivalently a Keisler measure on the relevant type space, whose measure-one formulas form a proper filter and whose Stone-space representation is invariant under definitional expansion. The resulting calculus provides finite context certificates, pair-separator hitting sets, query teaching dimension, prompt-preservation criteria, and scale-limit witnesses. Exact-arithmetic ancillary scripts reproduce the finite-field and threshold calculations and generate the data used by the figures.

Machine Learning (cs.LG); Logic in Computer Science (cs.LO)

arXiv:2606.07623 [cs.LG]

(or arXiv:2606.07623v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2606.07623 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

certificate_benchmark_protocol.txt

certificate_benchmark_responses.jsonl

certificate_benchmark_summary.csv

fixed_query_curve.csv

identification_curve.csv

rowspace_determinacy.csv

run_certificate_benchmark.py

verification_summary.txt

verify_theorems.py

(5 additional files not shown) You must enabled JavaScript to view entire file list.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.