Transcribe, Then Reason: Two-Pass Decomposition for Multimodal Review

Abstract:The natural way to review a long recording or document with a multimodal model is to hand it the raw source and ask for a review in one call. We show that this quietly fails: the model satisfices, dropping roughly a third of the content and embellishing the rest. The failure is not perception--almost all of the dropped content reappears when the same model is simply asked to transcribe the source. The bottleneck is generation under load: a single pass cannot perceive, reason over, and write a long faithful review at the same time, because doing all three competes for one output. We rule out the obvious alternatives. It is not the modality: models read text and an image of the same text equally well. And it is not merely a matter of thinking harder: giving the single pass a far larger reasoning budget does not recover the lost content, because the model spends that budget planning a review rather than writing the source down. What works is to split the labor across two same-weights passes--first transcribe, then review the transcript--so each step gets a full output budget of its own. This transcribe-then-review decomposition improves both faithfulness and coverage across a 21-source suite. The benefit is not uniform: we observe that it helps most where the one-pass baseline is weakest and little where that baseline is already strong, a pattern that also tracks the source's length and modality. Decomposition comes with two failure modes--the review pass running out of room on very long sources, and confabulating from memory once the grounding source is removed.

Multimedia (cs.MM); Artificial Intelligence (cs.AI)

arXiv:2609.18958 [cs.MM]

(or arXiv:2609.18958v1 [cs.MM] for this version)

https://doi.org/10.48550/arXiv.2609.18958 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.