analiza aiqbench
The paper proposes AIPC, an LLM agent framework that uses structured skills and toolchain, runtime, and profiling feedback to port PyTorch models onto Qualcomm's on-device NPU stack. Its reported results are narrow: two optimization cases, a Whisper-Small accuracy and latency pass rate of 6/7 versus 0/6 for the control, and a 2.791x YOLO-World speedup versus 1.599x. Readers should weigh these as single-vendor, small-sample evidence, but the cross-layer deployment problem it targets is a real bottleneck for edge AI.
AIPC: Evidence-Guided Agent Workflows for PyTorch Project Deployment
arXiv:2610.34890v1 Announce Type: cross Abstract: Deploying and optimizing a PyTorch inference project on a dedicated accelerator, especially an on-device NPU, requires more than using a framework device API as with CUDA. The project must be converted into hardware-runtime artifacts and adapted to the target toolchain and application interface. Differences in model formats, quantization flows, operator support, and toolchain interfaces create cross-layer problems spanning PyTorch/ONNX models, toolchains, runtimes, and device execution. LLM Agents can invoke tools to address such problems, but may drift because of insufficient domain knowledge, large configuration and repair spaces, or poor path selection. We propose AIPC (AI Porting & Conversion), an Agent method for deploying PyTorch inference projects. AIPC combines structured Skills with toolchain, runtime, and profiling evidence to drive cross-layer diagnosis and repair, organizing model adaptation, artifact building, on-target execution, accuracy repair, and performance optimization into one deployment flow. We evaluate AIPC on the on-device NPU path of Qualcomm QAIRT across vision, speech, LLM, and diffusion models. In two optimization cases, 6/7 Skill-arm Whisper-Small runs meet predefined accuracy and latency criteria versus 0/6 control runs, while YOLO-Worldv2 achieves a 2.791x end-to-end speedup versus 1.599x for the control. Under a pre-fixed deployment contract, Skill, tool permissions, and check conditions, some runs achieve zero in-run human intervention (0-HITL). We also examine Qwen3.8-27B as a locally deployable model for driving an Agent workflow.
Authors:Zhanwei Wu (1), Jianhao Su (1), Brian Zhang (1), ShengTing Huang (1), Weidong Feng (1), Gang Liu (1), Zhiguo Li (1), Wei Huang (1), Li Tan (1), Wanjun Li (1), Zejiong Teng (1), Changgeng Zhong (1), Guanping Wu (1), Han-Hsing Tu (2) ((1) Qualcomm Technologies, Inc., (2) Independent Researcher)
Abstract:Deploying and optimizing a PyTorch inference project on a dedicated accelerator, especially an on-device NPU, requires more than using a framework device API as with CUDA. The project must be converted into hardware-runtime artifacts and adapted to the target toolchain and application interface. Differences in model formats, quantization flows, operator support, and toolchain interfaces create cross-layer problems spanning PyTorch/ONNX models, toolchains, runtimes, and device execution. LLM Agents can invoke tools to address such problems, but may drift because of insufficient domain knowledge, large configuration and repair spaces, or poor path selection. We propose AIPC (AI Porting & Conversion), an Agent method for deploying PyTorch inference projects. AIPC combines structured Skills with toolchain, runtime, and profiling evidence to drive cross-layer diagnosis and repair, organizing model adaptation, artifact building, on-target execution, accuracy repair, and performance optimization into one deployment flow. We evaluate AIPC on the on-device NPU path of Qualcomm QAIRT across vision, speech, LLM, and diffusion models. In two optimization cases, 6/7 Skill-arm Whisper-Small runs meet predefined accuracy and latency criteria versus 0/6 control runs, while YOLO-Worldv2 achieves a 2.791x end-to-end speedup versus 1.599x for the control. Under a pre-fixed deployment contract, Skill, tool permissions, and check conditions, some runs achieve zero in-run human intervention (0-HITL). We also examine Qwen3.8-27B as a locally deployable model for driving an Agent workflow.
Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2610.34890 [cs.SE]
(or arXiv:2610.34890v1 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2610.34890 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 292 cuvinte