Skip to content
aiqbench takeMachine Learning

aiqbench analysis

This arXiv preprint audits whether historical summaries improve ridge-regression forecasts in a rule-based university trajectory simulator, using controls that separate genuine history effects from calendar-length effects. The reported gains are small, and most comparisons have confidence intervals that cross zero once the length-only control is included. The paper also flags that many fatigue predictions fall outside the simulator's valid range, and it is explicit that it evaluates a simulator rather than real students. For ML readers, the main takeaway is methodological: forecasting gains need matched controls, strict temporal indexing, and out-of-family checks before they can be trusted.
Source coveringarXiv
What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

What Does History Add? Calendar Controls and Information-Timing Audits for University Trajectory Forecasting

arXiv3 min read

arXiv:2610.21419v1 Announce Type: new Abstract: We audit the predictive value of historical summaries in a rule-based university trajectory simulator. Seventeen ridge specifications and persistence separate current plans, past-state and past-plan summaries, and calendar-only history-length indicators. Across training, validation and two family-held-out evaluations, we reconstruct 16,750 routes and 804,000 monthly records. On the original 40-family set, six-month history lowers six-step normalized RMSE from 0.159234 to 0.151522, but its difference from a length-only control is -0.000895 (descriptive paired 95% interval [-0.001957,0.000248]). Fixed weights on 30 families unused in identified prior tasks reproduce this pattern: 0.156480 to 0.150074 versus current-state prediction, but -0.000558 [-0.001783,0.000728] versus the length control. These uncertain comparisons do not generalize to all-history summaries, which improve on their own calendar control. Exploratory decomposition finds clearer preparation-index and funds benefits, whereas fatigue dominates the six-month model's loss and has an uncertain length-controlled benefit. Moreover, 28.20% of its six-step fatigue predictions violate the simulator's [5,90] support. We document family dependence, prior exposure, strict temporal indexing, matched recursive references and independent numerical replay. The task conditions on recorded future plans; it does not evaluate real-student forecasts, causal action effects, autonomous planning or language-model memory. This is an evaluation audit, not a new memory architecture.

Title:What Does History Add? Calendar Controls and Information-Timing Audits for University Trajectory Forecasting

Abstract:We audit the predictive value of historical summaries in a rule-based university trajectory simulator. Seventeen ridge specifications and persistence separate current plans, past-state and past-plan summaries, and calendar-only history-length indicators. Across training, validation and two family-held-out evaluations, we reconstruct 16,750 routes and 804,000 monthly records. On the original 40-family set, six-month history lowers six-step normalized RMSE from 0.159234 to 0.151522, but its difference from a length-only control is -0.000895 (descriptive paired 95% interval [-0.001957,0.000248]). Fixed weights on 30 families unused in identified prior tasks reproduce this pattern: 0.156480 to 0.150074 versus current-state prediction, but -0.000558 [-0.001783,0.000728] versus the length control. These uncertain comparisons do not generalize to all-history summaries, which improve on their own calendar control. Exploratory decomposition finds clearer preparation-index and funds benefits, whereas fatigue dominates the six-month model's loss and has an uncertain length-controlled benefit. Moreover, 28.20% of its six-step fatigue predictions violate the simulator's [5,90] support. We document family dependence, prior exposure, strict temporal indexing, matched recursive references and independent numerical replay. The task conditions on recorded future plans; it does not evaluate real-student forecasts, causal action effects, autonomous planning or language-model memory. This is an evaluation audit, not a new memory architecture.

12 pages, 3 figures, 6 tables. Methodological simulator evaluation with information-timing and calendar-control audits

Machine Learning (cs.LG)

arXiv:2610.21419 [cs.LG]

(or arXiv:2610.21419v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2610.21419 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

family_metrics.json

figure_values.json

fresh_protocol.json

protocol_addendum.json

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Full text under the publisher's open licence · 321 words

Related coverage