aiqbench analysis
This study presents a practical implementation of an LLM-based teaching assistant using RAG for a specialized engineering curriculum, offering concrete insights into prompt design and pipeline tuning. The research provides valuable empirical data on the impact of AI tutors in open-book exam settings, demonstrating that academic validity can be maintained while students find the tool beneficial.
An AI Teaching Assistant for Motion Picture Engineering
arXiv:2604.04670v2 Announce Type: replace-cross Abstract: The rapid rise of LLMs over the last few years has promoted growing experimentation with LLM-driven AI tutors. However, the details of implementation, as well as the benefit in a teaching environment, are still in the early days of exploration. This article addresses these issues in the context of implementation of an AI Teaching Assistant (AI-TA) using Retrieval Augmented Generation (RAG) for Trinity College Dublin's Master's Motion Picture Engineering (MPE) course. We provide details of our implementation (including the prompt to the LLM, and code), and highlight how we designed and tuned our RAG pipeline to meet course needs. We describe our survey instrument and report on the impact of the AI-TA through a number of quantitative metrics. The scale of our experiment (43 students, 296 sessions, 1,889 queries over 7 weeks) was sufficient to have confidence in our findings. Unlike previous studies, we experimented with allowing the use of the AI-TA in open-book examinations. Statistical analysis across three exams showed no performance differences regardless of AI-TA access (p > 0.05), demonstrating that thoughtfully designed assessments can maintain academic validity. Student feedback revealed that the AI-TA was beneficial (mean = 4.22/5), while students had mixed feelings about preferring it over human tutoring (mean = 2.78/5).
Abstract:The rapid rise of LLMs over the last few years has promoted growing experimentation with LLM-driven AI tutors. However, the details of implementation, as well as the benefit in a teaching environment, are still in the early days of exploration. This article addresses these issues in the context of implementation of an AI Teaching Assistant (AI-TA) using Retrieval Augmented Generation (RAG) for Trinity College Dublin's Master's Motion Picture Engineering (MPE) course. We provide details of our implementation (including the prompt to the LLM, and code), and highlight how we designed and tuned our RAG pipeline to meet course needs. We describe our survey instrument and report on the impact of the AI-TA through a number of quantitative metrics. The scale of our experiment (43 students, 296 sessions, 1,889 queries over 7 weeks) was sufficient to have confidence in our findings. Unlike previous studies, we experimented with allowing the use of the AI-TA in open-book examinations. Statistical analysis across three exams showed no performance differences regardless of AI-TA access (p > 0.05), demonstrating that thoughtfully designed assessments can maintain academic validity. Student feedback revealed that the AI-TA was beneficial (mean = 4.22/5), while students had mixed feelings about preferring it over human tutoring (mean = 2.78/5).
Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Signal Processing (eess.SP)
K.3.1; K.3.2; I.2.7
arXiv:2604.04670 [eess.IV]
(or arXiv:2604.04670v2 [eess.IV] for this version)
https://doi.org/10.48550/arXiv.2604.04670 Focus to learn more arXiv-issued DOI via DataCite
IEEE Signal Processing Magazine, vol. 43, no. 4, pp. 76-84, July 2026
https://doi.org/10.1109/MSP.2026.3665423 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 334 words
Related coverage

2026.7: Automations that speak your language
Home Assistant · Jul 1, 2026

2026.9: There's room on this bus
Home Assistant · Sep 2, 2026

Oscar Winner Brings Monsters to Life With His Simulation Software
IEEE Spectrum · Aug 28, 2026

IEEE Presidents’ Scholarship Honors Teen Innovators
IEEE Spectrum · Aug 17, 2026