analiza aiqbench
This research identifies a critical bottleneck in Large Language Model development where explicit cultural signals diminish sharply during post-training phases, despite the use of multilingual datasets. The authors propose that effective alignment requires shifting focus from inference-time interventions to restructuring training data pipelines with more balanced geographic and task-specific representations. By releasing a tagged dataset containing 5.6 million samples, they provide essential resources for improving downstream performance on cultural benchmarks.
The Culture Funnel: You Can't Align What isn't in the Data
arXiv:2606.13808v1 Announce Type: new Abstract: Current cultural alignment approaches focus on inference-time interventions, assuming models already contain sufficient cultural knowledge. We argue modern LLM pipelines suffer from a cultural data funnel. Using a multidimensional tagging framework across pretraining, fine-tuning, alignment, and reasoning datasets, we show explicit cultural signals decline sharply during post-training, while geographically concentrated, task-specialized data dominates. Multilinguality enhances geographic diversity of cultural knowledge but does not ensure balanced representation. Our tags improve downstream cultural benchmark performance, demonstrating that advances require shifting focus in training data pipelines. To facilitate future research, we release our culturally tagged dataset with 5.6M samples at https://huggingface.co/datasets/CohereLabs/CultureMarkers.
Abstract:Current cultural alignment approaches focus on inference-time interventions, assuming models already contain sufficient cultural knowledge. We argue modern LLM pipelines suffer from a cultural data funnel. Using a multidimensional tagging framework across pretraining, fine-tuning, alignment, and reasoning datasets, we show explicit cultural signals decline sharply during post-training, while geographically concentrated, task-specialized data dominates. Multilinguality enhances geographic diversity of cultural knowledge but does not ensure balanced representation. Our tags improve downstream cultural benchmark performance, demonstrating that advances require shifting focus in training data pipelines. To facilitate future research, we release our culturally tagged dataset with 5.6M samples at this https URL.
Computation and Language (cs.CL)
arXiv:2606.13808 [cs.CL]
(or arXiv:2606.13808v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.13808 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 195 cuvinte
Subiecte conexe
The Culture Funnel: You can’t align what isn’t in the data
Cohere · 19 aug. 2026
Inside the suddenly explosive world of AI safety
The Verge · 17 sept. 2026
The left and right agree on one thing: no data centers
The Verge · 6 aug. 2026
Cultural Alignment in Large Language Models Using Soft Prompt Tuning
arXiv · 21 sept. 2026