Optimizing Streaming Speech Recognition on Arm® Edge Devices with ExecuTorch

When you’re speaking to a device, you expect text to appear as you talk. That makes streaming automatic speech recognition (ASR) a different challenge from batch transcription, where the model has the complete recording before it starts. Strong batch accuracy does not necessarily carry over into streaming. One cross-model ASR study illustrates this gap. Qwen3-ASR-1.7B achieved the best batch accuracy among the models tested, but its word error rate (WER) rose by 77% when evaluated with 2.4-second audio chunks. In that study’s cross-model streaming comparison (Table 5), NVIDIA® Nemotron™ Speech Streaming 0.6B had the lowest average streaming WER when operating over 560 ms audio chunks. The comparison makes a critical point: accuracy must be measured at the chunk cadence used in deployment. This case study takes the newer, multilingual Nemotron 3.5 from PyTorch to ExecuTorch and profiles its on-device execution on a Google Pixel 11. We ask how precision and chunk size change streaming quality, how much processing time the audio requires with one inference thread, where the workload spends its time, and which native kernels run. The goal is a practical way to evaluate accuracy…