analiza aiqbench
This research establishes a practical framework for optimizing financial LLMs by demonstrating that specialized calibration data is only necessary when compression techniques like pruning cause significant task-level damage. The study reveals that quantization generally preserves performance without needing domain-specific tuning, whereas severe accuracy drops from pruning can be effectively recovered using targeted financial datasets. These findings offer a clear efficiency rule: measure compression impact first before investing in costly, domain-matched calibration corpora.
Damage Predicts Recovery: When Calibration Data Matters in Compressing Financial LLMs
arXiv:2609.26241v1 Announce Type: cross Abstract: Post-training quantization and pruning rely on a small calibration corpus. Whether specialized domains such as finance require domain-matched calibration data remains unsettled. We argue that the answer depends on the task-level damage caused by compression rather than on domain mismatch. If compression preserves the target capability, changing the calibration corpus has little effect. If compression causes large losses, task-formatted calibration can recover part of the loss. We test this hypothesis across two model families, six compression configurations, three token-matched calibration corpora, and ten financial classification and numerical question-answering tasks. The results support this hypothesis. Quantization largely preserves task performance, and calibration choice has little effect in this case. Pruning reduces numerical QA accuracy by over 40 points. In these damaged settings, another generic corpus does not help, while FinMix, a mixture of financial task examples, recovers a large part of the loss. The link between damage and recovery holds across model families and scales. These findings support a practical rule. Measure task-specific compression damage first, and construct specialized calibration data only when the damage is large.
Abstract:Post-training quantization and pruning rely on a small calibration corpus. Whether specialized domains such as finance require domain-matched calibration data remains unsettled. We argue that the answer depends on the task-level damage caused by compression rather than on domain mismatch. If compression preserves the target capability, changing the calibration corpus has little effect. If compression causes large losses, task-formatted calibration can recover part of the loss. We test this hypothesis across two model families, six compression configurations, three token-matched calibration corpora, and ten financial classification and numerical question-answering tasks. The results support this hypothesis. Quantization largely preserves task performance, and calibration choice has little effect in this case. Pruning reduces numerical QA accuracy by over 40 points. In these damaged settings, another generic corpus does not help, while FinMix, a mixture of financial task examples, recovers a large part of the loss. The link between damage and recovery holds across model families and scales. These findings support a practical rule. Measure task-specific compression damage first, and construct specialized calibration data only when the damage is large.
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2609.26241 [cs.CL]
(or arXiv:2609.26241v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.26241 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 274 cuvinte
Subiecte conexe
Preserving Mathematical Reasoning in Compressed Diffusion Language Models via Trajectory-Aware Low-Rank Approximation
arXiv · 6 oct. 2026
Good Memory Has ECC: Evaluating the Memory of Vision-Language Models Beyond Accuracy
arXiv · 2 sept. 2026
Simplex-Constrained Sparse Bagging: Transitioning from Uniform Priors to Sparse Posteriors in Ensemble Learning
arXiv · 15 iun. 2026
Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantization
arXiv · 28 aug. 2026