Chat Debugging: An Exploratory Study of Human-AI Collaboration to Debug Analog Circuits

Abstract:This research paper describes an exploratory study on the effectiveness of Chat Debugging: troubleshooting malfunctioning analog circuits on breadboards and printed circuit boards (PCB) by undergraduates through conversations with public-domain large language models (LLMs). Through thematic analysis of students' voluntarily shared chat logs when debugging pre-determined buggy circuits under exam and time pressure, we discovered multimodal usage patterns by students and considerable domain knowledge and sensible debugging suggestions offered by off-the-shelf LLMs. Meanwhile, we also identified major gaps in LLM technologies and students' skills during human-AI collaborative debugging, such as LLMs' limitations in 2D/3D image-based reasoning, unjustified tone of confidence, and students' deficits in fundamental concepts and critical thinking.

This is the accepted version of a paper to be presented at the 2026 IEEE Frontiers in Education Conference (FIE). The final version will be available via IEEE Xplore

Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Image and Video Processing (eess.IV)

arXiv:2608.02955 [cs.HC]

(or arXiv:2608.02955v1 [cs.HC] for this version)

https://doi.org/10.48550/arXiv.2608.02955 arXiv-issued DOI via DataCite (pending registration)