aiqbench analysis
This study demonstrates that while supervised models struggle to generalize from written news to transcribed spoken video, large language models effectively quantify social dimensions of peace in YouTube content with accuracy comparable to human coders. The findings highlight a critical modality gap in current NLP systems and validate the utility of LLMs for real-time sentiment analysis in unstructured video data. For developers, this underscores the necessity of training or fine-tuning models specifically on transcribed speech rather than relying on text-only datasets for social media applications.
AI Application Gives Users Real-Time Feedback on the Level of Peace in the Social Media Videos They Watch
arXiv:2601.05232v3 Announce Type: replace-cross Abstract: Most people now get their news from videos on social media, such as YouTube and Facebook, rather than through curated journalism. "We become what we behold." The content and tone of language plays an essential role in starting or ending conflicts. "Hate Speech" can enhance conflict, "Peace Speech" can enhance peace. We developed an application that measures, in real time, these aspects of speech from YouTube videos, which can give users helpful feedback on their own media diet. We used two approaches: 1) supervised machine learning. Language in the text of online news media text was tagged by surveys that measure the level of peace in those countries. One fully connected feedforward and 2 convolutional neural networks trained on that data were $\sim 97\%$ accurate in predicting levels of peace in the test set and $\sim 70\%$ accurate in another distinct news text data set, but did not generalize to YouTube videos, suggesting that written text is different than transcribed spoken language. 2) social science dimensions. There is no similar external data to tag the text in the YouTube video transcripts. We therefore used 2 word-level sentiment analysis (SA) and 6 context-level large language models (LLMs) to measure 5 social dimensions in peace identified by 59 social science studies: compassion-contempt, news-opinion, promotion-prevention, creativity-order, nuance-simplification. LLMs more closely matched the values by 3 human coders on 52 videos, $r^2\sim0.60$ than SA, at $r^2\sim0.03$. Results: LLMs successfully measured social dimensions important in peace in YouTube videos, compared to human coders. These results serve as the basis of an analysis engine that can give users and content creators feedback on their own media diet and creations.
Authors:P. Gilda (1), P. Dungarwal (1), A. Thongkham (1), E. T. Ajayi (2), S. Choudhary (1), T. M. Terol (1), C. Lam (1), J. P. Araujo (1), M. McFadyen-Mungalln (1), L. S. Liebovitch (1), P. T. Coleman (1), H. West (1), K. Sieck (3), S. Carter (3) ((1) Columbia University, (2) St John's University, (3) Toyota Research Institute)
Abstract:Most people now get their news from videos on social media, such as YouTube and Facebook, rather than through curated journalism. "We become what we behold." The content and tone of language plays an essential role in starting or ending conflicts. "Hate Speech" can enhance conflict, "Peace Speech" can enhance peace. We developed an application that measures, in real time, these aspects of speech from YouTube videos, which can give users helpful feedback on their own media diet. We used two approaches: 1) supervised machine learning. Language in the text of online news media text was tagged by surveys that measure the level of peace in those countries. One fully connected feedforward and 2 convolutional neural networks trained on that data were $\sim 97\%$ accurate in predicting levels of peace in the test set and $\sim 70\%$ accurate in another distinct news text data set, but did not generalize to YouTube videos, suggesting that written text is different than transcribed spoken language. 2) social science dimensions. There is no similar external data to tag the text in the YouTube video transcripts. We therefore used 2 word-level sentiment analysis (SA) and 6 context-level large language models (LLMs) to measure 5 social dimensions in peace identified by 59 social science studies: compassion-contempt, news-opinion, promotion-prevention, creativity-order, nuance-simplification. LLMs more closely matched the values by 3 human coders on 52 videos, $r^2\sim0.60$ than SA, at $r^2\sim0.03$. Results: LLMs successfully measured social dimensions important in peace in YouTube videos, compared to human coders. These results serve as the basis of an analysis engine that can give users and content creators feedback on their own media diet and creations.
Computation and Language (cs.CL); Computers and Society (cs.CY); Machine Learning (cs.LG)
arXiv:2601.05232 [cs.CL]
(or arXiv:2601.05232v3 [cs.CL] for this version)
Full text under the publisher's open licence · 350 words
Related coverage
[AINews] Meta Connect 2026: Muse glasses, voice, video, and Charm
Latent Space · Sep 24, 2026
Integrate NVIDIA Omniverse RTX Sensor Simulation Into Existing Apps
NVIDIA · Jul 20, 2026
Build real-time voice applications with vLLM-Omni on SageMaker AI – Part 1
AWS · Sep 28, 2026
Patter SDK Guide to Building a Restaurant Booking Phone Agent with Dynamic Variables, Guardrails, Latency Dashboards, and Eval Checks
MarkTechPost · Jul 16, 2026