analiza aiqbench
This research analyzes the predictive limits of AI models in ranking Chinese investor posts by maximum possible profit, revealing that a regime-specific semantic shortcut artificially inflated development accuracy compared to official test scores. The study decomposes financial sentiment into directional text, volatility, and horizon components, demonstrating that while pre-posting volatility correlates with future outcomes, transferred text provides minimal additional signal. Evaluation reliability is shown to depend heavily on the labeled profit gap rather than universal unpredictability, offering critical insights for AI applications in quantitative finance.
BAIBAICHUCHU at the NTCIR-19 FinArg-3 Task: When Is Maximum Possible Profit Predictable from Investor Text?
arXiv:2610.03962v1 Announce Type: new Abstract: The BAIBAICHUCHU team participated in the Social Media Subtask of NTCIR-19 FinArg-3, ranking Chinese investor posts by Maximum Possible Profit (MPP). A three-track ensemble of lexical features, a FinArg-2-pre-finetuned MacBERT ranker, and an LLM judge reaches 0.734 in post-grouped development evaluation, but our best official run scores 0.517. All twelve submitted runs lie between 0.4598 and 0.5402, and our 26 unanimous three-track pairs score 0.500. A post-hoc audit finds that the submitted judge applied a long-only rule to bearish posts although MPP is stance-aware. Correcting it changes 28 of 87 official predictions without changing accuracy, yet lowers development accuracy from 0.680 to 0.622: a regime-specific semantic shortcut improved validation fit. We decompose ranking into directional text, volatility and horizon, and pairwise margin. A running-extremum model predicts $\sigma\sqrt{T}$ scaling, observed ex post in 502 price-aligned posts from a separate July 2026 collection. Pre-posting volatility scaled by the post-specific observation horizon, $\sigma_{\mathrm{pre},20}\sqrt{N_i}$, is associated with a later truncated-horizon MPP proxy (Spearman $\rho=0.320$; ticker-cluster 95% CI [0.133,0.466]) and correctly orders 61.4% of unequal-outcome pairs. Transferred text is nearly uncorrelated with the July outcome ($\rho=0.055$) and adds little conditional on this score and stance. Development reliability rises from about 0.60 to 0.93 as the labeled MPP gap widens. An earlier ERAI result of 0.6207 rules out universal unpredictability as a simple explanation. Evaluation should jointly consider text, market regime, historical volatility, horizon, and pair composition.
Title:BAIBAICHUCHU at the NTCIR-19 FinArg-3 Task: When Is Maximum Possible Profit Predictable from Investor Text?
Abstract:The BAIBAICHUCHU team participated in the Social Media Subtask of NTCIR-19 FinArg-3, ranking Chinese investor posts by Maximum Possible Profit (MPP). A three-track ensemble of lexical features, a FinArg-2-pre-finetuned MacBERT ranker, and an LLM judge reaches 0.734 in post-grouped development evaluation, but our best official run scores 0.517. All twelve submitted runs lie between 0.4598 and 0.5402, and our 26 unanimous three-track pairs score 0.500. A post-hoc audit finds that the submitted judge applied a long-only rule to bearish posts although MPP is stance-aware. Correcting it changes 28 of 87 official predictions without changing accuracy, yet lowers development accuracy from 0.680 to 0.622: a regime-specific semantic shortcut improved validation fit. We decompose ranking into directional text, volatility and horizon, and pairwise margin. A running-extremum model predicts $\sigma\sqrt{T}$ scaling, observed ex post in 502 price-aligned posts from a separate July 2026 collection. Pre-posting volatility scaled by the post-specific observation horizon, $\sigma_{\mathrm{pre},20}\sqrt{N_i}$, is associated with a later truncated-horizon MPP proxy (Spearman $\rho=0.320$; ticker-cluster 95% CI [0.133,0.466]) and correctly orders 61.4% of unequal-outcome pairs. Transferred text is nearly uncorrelated with the July outcome ($\rho=0.055$) and adds little conditional on this score and stance. Development reliability rises from about 0.60 to 0.93 as the labeled MPP gap widens. An earlier ERAI result of 0.6207 rules out universal unpredictability as a simple explanation. Evaluation should jointly consider text, market regime, historical volatility, horizon, and pair composition.
15 pages, 1 figure, 6 tables. Both authors contributed equally. Selected for oral presentation at NTCIR-19 (FinArg-3 Social Media Subtask)
Computation and Language (cs.CL)
arXiv:2610.03962 [cs.CL]
(or arXiv:2610.03962v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.03962 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 361 cuvinte
Subiecte conexe

Deconfusing Corrigibility
AI Alignment Forum · 17 oct. 2026

NVIDIA GTC Taipei at COMPUTEX: Live Updates on What’s Next in AI
NVIDIA · 21 mai 2026

Examining AI Swarm Efficiency with the AI Futures Model
LessWrong · 19 oct. 2026

Benchtalks #3: We taught AI everything except how to learn
Snorkel AI Blog · 25 iun. 2026