From Efficiency to Sufficiency: The Cost-Aware Measurement Design Framework for Automated Essay Scoring

Abstract:Compute cost in automated essay scoring (AES) is usually treated as a technical efficiency issue rather than a measurement design concern. Yet deployment decisions require more than reporting latency, model size, or inference expense. In high-stakes settings, total burden includes validation, monitoring, adjudication, compliance, and governance; in instructional settings, pricing, licensing, and integration may dominate cost. This paper reframes AES as a cost-constrained measurement system and introduces the Cost-Aware Measurement Design (CAMD) Framework, a four-layer structure consisting of metering, allocation, unit economics, and decision, for translating computational and workflow resource records into decision-relevant cost constructs. Direct cost is specified as monetary outlay plus attributable labor and time, and two criteria, design attribution and auditability, determine what the cost boundary excludes. Burdens that satisfy neither criterion, such as institutional capability and governance posture, are represented as constraints on which designs are admissible rather than priced. The paper defines cost per defensible score (CPDS) and argues that AES systems should be compared by quality-cost sufficiency: minimizing cost among defensible scoring designs that satisfy measurement requirements. Scoring volume conditions all four layers, so cost per defensible score is treated as a function of scale rather than as a single transferable number. CAMD provides a portable basis for representing, comparing, and evaluating cost across AES studies and deployment contexts.

Computers and Society (cs.CY); Computation and Language (cs.CL)

arXiv:2610.19287 [cs.CY]

(or arXiv:2610.19287v1 [cs.CY] for this version)

https://doi.org/10.48550/arXiv.2610.19287 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.