analiza aiqbench
This paper is primarily about LLM-based recommendation, so it passes the topic gate. Its focus is on how preprocessing choices in fine-tuning data (title representation, demographic balancing, and knowledge enrichment) affect both recommendation accuracy and demographic fairness in a generative recommender. The reported gains are modest (e.g., HR@10 rising from 10.67% to 12.47%), and the fairness effects are attribute-dependent, with gender balancing lowering utility without clear fairness benefits. For readers, the takeaway is that data preprocessing decisions need to be evaluated jointly on utility, output validity, and demographic sensitivity rather than on accuracy alone.
Preprocessing Choices in Fine-Tuning Data Matter: Utility and Demographic Sensitivity in LLM-Based Recommendation
arXiv:2611.03113v1 Announce Type: cross Abstract: Large language models (LLMs) have introduced a generative approach to recommendation, producing item lists from textual representations of user histories and item information. However, this generative setting raises challenges concerning output validity, prompt consistency, and sensitivity to demographic descriptors. In this work, we investigate how preprocessing choices in fine-tuning data affect recommendation utility and fairness-related behavior in a generative LLM-based recommender system. We first establish a controlled recommendation pipeline that combines supervised fine-tuning, catalog-constrained decoding, and prompt augmentation. We then study three preprocessing choices: item-title representation, demographic balancing, and auxiliary item-knowledge enrichment. Experiments on MovieLens-1M show that the catalog-constrained pipeline eliminates out-of-catalog items and ensures complete ten-item outputs, while prompt augmentation improves prompt consistency and utility. Original MovieLens titles increase HR@10 from 10.67% to 12.47% and Coverage@10 from 41.36% to 47.88%. Fairness-related effects are attribute-dependent: age balancing and knowledge enrichment reduce selected group-level utility gaps while largely preserving utility, whereas gender balancing reduces utility without improving the evaluated gender-related indicators. Overall, preprocessing choices in fine-tuning data should be evaluated jointly in terms of utility, output behavior, and demographic sensitivity.
Abstract:Large language models (LLMs) have introduced a generative approach to recommendation, producing item lists from textual representations of user histories and item information. However, this generative setting raises challenges concerning output validity, prompt consistency, and sensitivity to demographic descriptors. In this work, we investigate how preprocessing choices in fine-tuning data affect recommendation utility and fairness-related behavior in a generative LLM-based recommender system. We first establish a controlled recommendation pipeline that combines supervised fine-tuning, catalog-constrained decoding, and prompt augmentation. We then study three preprocessing choices: item-title representation, demographic balancing, and auxiliary item-knowledge enrichment. Experiments on MovieLens-1M show that the catalog-constrained pipeline eliminates out-of-catalog items and ensures complete ten-item outputs, while prompt augmentation improves prompt consistency and utility. Original MovieLens titles increase HR@10 from 10.67% to 12.47% and Coverage@10 from 41.36% to 47.88%. Fairness-related effects are attribute-dependent: age balancing and knowledge enrichment reduce selected group-level utility gaps while largely preserving utility, whereas gender balancing reduces utility without improving the evaluated gender-related indicators. Overall, preprocessing choices in fine-tuning data should be evaluated jointly in terms of utility, output behavior, and demographic sensitivity.
11 pages, 2 figures, 5 tables. Accepted at the 3rd Workshop on Evaluating and Applying Recommender Systems with Large Language Models (EARL) at CIKM 2026, November 8, 2026, Rome, Italy
Information Retrieval (cs.IR); Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2611.03113 [cs.IR]
(or arXiv:2611.03113v1 [cs.IR] for this version)
https://doi.org/10.48550/arXiv.2611.03113 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 234 cuvinte
Subiecte conexe
Hidden Data Challenges in Clinical Machine Learning
arXiv · 12 oct. 2026
Global tree forecasters collapse at the hierarchical aggregate: a five-panel failure characterization
arXiv · 24 sept. 2026
Deploying an HSTU Generative Recommender with NVIDIA Dynamo-Triton
NVIDIA · 30 sept. 2026
Prior Labs Releases TabPFN-3.5: A Tabular Foundation Model That Beats the Winning Otto Kaggle Solution With Default Settings
MarkTechPost · 16 sept. 2026