aiqbench analysis
This research demonstrates that the choice between jointly estimating skill structures and learning trajectories versus a stepwise approach significantly alters conclusions about learner proficiency in dynamic environments. The study provides empirical evidence and R code showing that joint analysis is more reliable when item-skill mappings are uncertain or the item pool changes over time. For practitioners building adaptive learning systems, this highlights the critical importance of model selection strategy when interpreting fine-grained skill development data.
Interpreting Learning Under Competing Models: Joint and Stepwise Approaches for Dynamic Cognitive Diagnosis
arXiv:2606.06804v1 Announce Type: new Abstract: Digital learning environments record learners' responses to individual items, making it possible to study the development of specific skills rather than overall scores. Drawing conclusions about learning from these data requires a model that links responses to latent skills and tracks how mastery changes over time. When the skills measured by each item are unknown, the analyst must decide whether to estimate this structure, the Q-matrix, jointly with the learning process, or to establish it first and study learning afterwards. We show that this decision can change substantive conclusions about how learners develop. Using dynamic cognitive diagnostic models, we analyse data from two reading games measuring vocabulary and comprehension from Grade 2 to Grade 3, with item-text embeddings providing prior information for the unknown Q-matrix. A joint analysis and a bias-corrected stepwise analysis agree that most learners move toward mastering both skills, but disagree about how many remain only partially proficient at Grade 3, changing how reading progress would be reported. A simulation study identifies when the two analyses diverge and shows that joint analysis is more reliable when the item-skill structure is uncertain and the item pool changes between grades. We provide R code for both analyses.
Abstract:Digital learning environments record learners' responses to individual items, making it possible to study the development of specific skills rather than overall scores. Drawing conclusions about learning from these data requires a model that links responses to latent skills and tracks how mastery changes over time. When the skills measured by each item are unknown, the analyst must decide whether to estimate this structure, the Q-matrix, jointly with the learning process, or to establish it first and study learning afterwards. We show that this decision can change substantive conclusions about how learners develop. Using dynamic cognitive diagnostic models, we analyse data from two reading games measuring vocabulary and comprehension from Grade 2 to Grade 3, with item-text embeddings providing prior information for the unknown Q-matrix. A joint analysis and a bias-corrected stepwise analysis agree that most learners move toward mastering both skills, but disagree about how many remain only partially proficient at Grade 3, changing how reading progress would be reported. A simulation study identifies when the two analyses diverge and shows that joint analysis is more reliable when the item-skill structure is uncertain and the item pool changes between grades. We provide R code for both analyses.
Machine Learning (cs.LG); Applications (stat.AP)
arXiv:2606.06804 [cs.LG]
(or arXiv:2606.06804v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.06804 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 294 words
Related coverage
Learning Visually Interpretable Oscillator Networks for Soft Continuum Robots from Video
arXiv · Jun 12, 2026
ProtoTSNet: Interpretable Multivariate Time Series Classification With Prototypical Parts
arXiv · Jul 21, 2026
A Heterogeneous Mixture of Experts Framework for Interpretable Machine Learning
arXiv · Aug 26, 2026
Combining Hierarchical Cognitive Process with Process Supervision for Interpretable Scene Safety Understanding
arXiv · Sep 23, 2026