aiqbench analysis
This paper introduces a formal evaluation semantics for ML dataset descriptors, resolving ambiguities in how data use conditions are enforced and recorded during inference. By defining a closed set of operators with a specific decision procedure, it enables efficient, auditable gating that adds negligible overhead while ensuring policy consistency across different implementations. This work advances the infrastructure layer required for trustworthy and compliant deployment of machine learning models.
A Policy Profile for Croissant: Refusal as a Property of the Dataset
arXiv:2609.19640v1 Announce Type: new Abstract: Croissant is the de facto machine-readable descriptor for ML datasets: JSON-LD over schema.org. Since version 1.1 it also carries data use conditions, recommending DUO and ODRL for them. What no version specifies is how any of them is evaluated: no decision procedure, no bound on evaluation cost, no outcome for a condition an implementation cannot evaluate, no record of what was checked, and nothing on composition with caller-side authority. We supply that half. An additive profile lets a dataset declare the operations it admits and the conditions under which it admits them, over a closed set of five operators whose decision procedure is given in full, so a gate decides from the descriptor alone and records what it checked. Two corpora evaluate it and their evidence is kept apart. Three descriptors that gated a real nf-core pipeline give the deployment result: decisions from a profile document match the gate's native descriptor record for record, stripping the layer leaves a valid Croissant document, and the added cost is 11.7 $\mu$s against a 119 $\mu$s decision. A corpus generated from the profile's grammar gives the breadth, covering every operator, refusal class and conformance clause. Across its valid cases, 552 complete decision records agree three ways -- native descriptor, profile terms, and the same policy as ODRL in usageInfo. The carrier is therefore not the contribution; the evaluation semantics is. Finally, caller-bound and data-bound policies range over non-overlapping state spaces, so neither permit set contains the other.
Abstract:Croissant is the de facto machine-readable descriptor for ML datasets: JSON-LD over this http URL. Since version 1.1 it also carries data use conditions, recommending DUO and ODRL for them. What no version specifies is how any of them is evaluated: no decision procedure, no bound on evaluation cost, no outcome for a condition an implementation cannot evaluate, no record of what was checked, and nothing on composition with caller-side authority. We supply that half. An additive profile lets a dataset declare the operations it admits and the conditions under which it admits them, over a closed set of five operators whose decision procedure is given in full, so a gate decides from the descriptor alone and records what it checked. Two corpora evaluate it and their evidence is kept apart. Three descriptors that gated a real nf-core pipeline give the deployment result: decisions from a profile document match the gate's native descriptor record for record, stripping the layer leaves a valid Croissant document, and the added cost is 11.7 $\mu$s against a 119 $\mu$s decision. A corpus generated from the profile's grammar gives the breadth, covering every operator, refusal class and conformance clause. Across its valid cases, 552 complete decision records agree three ways -- native descriptor, profile terms, and the same policy as ODRL in usageInfo. The carrier is therefore not the contribution; the evaluation semantics is. Finally, caller-bound and data-bound policies range over non-overlapping state spaces, so neither permit set contains the other.
23 pages. Reference implementation and conformance corpus archived at doi:https://doi.org/10.5281/zenodo.22018156 and doi:https://doi.org/10.5281/zenodo.22016112
Machine Learning (cs.LG); Cryptography and Security (cs.CR); Databases (cs.DB)
D.4.6; H.2.1; I.2.11
arXiv:2609.19640 [cs.LG]
(or arXiv:2609.19640v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.19640 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 285 words
Related coverage
A Red Line and Oversight Framework for Government AI Contracts
AI Alignment Forum · Jul 18, 2026
Improving CoT Monitorability of Evaluation Awareness via Verbalization Training
LessWrong · Oct 2, 2026
Three perspectives on corrigibility - shards, simulators, singular learning
LessWrong · Oct 16, 2026
State of the Field: AI for Epistemics and Coordination
LessWrong · Oct 2, 2026