TeachGrade: Training-Free Short Answer Scoring with Teacher Evidence

Abstract:Multi-subject, multi-step scoring requires consistent partial credit across grading conventions. We introduce TeachGrade, a training-free framework using teacher-graded examples. The Task Adapt module builds two frozen memories: Skill captures contextual teacher preferences beyond the rubric; CaseBank supplies local evidence. PreJudge provides retrieval cues; FinalJudge regrades the answer without preliminary scores. On 3,829 SAS-Bench answers across 12 tasks, official macro collaborative consistency score (CCS) rises from Native's 67.99 to 81.32 (+13.33 points), and pooled step exact match from 54.85% to 62.71%. Macro quadratic weighted kappa (QWK) reaches 84.71, exceeding Native, OpenJudge, ProofGrader, and Trojsten by 9.52, 3.50, 11.37, and 17.43 points, respectively, and leading on 8 tasks. Local Qwen3.8-27B gains 8.83 CCS and 4.08 QWK points, improving CCS on all 12 tasks. QWK gains over Native span 6.38--14.65 points on Portuguese automatic essay scoring (AES) and 2.16--2.24 on ASAP-SAS across label views, with lower mean absolute error. These results support reuse across languages, response formats, and backbones without weight updates.

Computation and Language (cs.CL)

arXiv:2610.14900 [cs.CL]

(or arXiv:2610.14900v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2610.14900 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

construction_example/README.md

construction_example/adaptation_request.json

construction_example/adaptation_system_prompt.txt

construction_example/adaptation_user_prompt.txt

construction_example/builder_v30.py

construction_example/case_entry.json

construction_example/compiled_casebank.json

construction_example/compiled_skill.json

construction_example/knowledge_input_full.json

construction_example/knowledge_input_prompt_view.json

construction_example/original_casebank.json

construction_example/original_skill.json

construction_example/skill_entry.json

construction_example/split_ids.json

construction_example/task_specification.json

construction_example/verification.json

supplementary_audit.pdf

task_comparison/README.md

task_comparison/math_casebank.json

task_comparison/math_skill.json

task_comparison/math_specification.json

(17 additional files not shown) You must enabled JavaScript to view entire file list.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.