analiza aiqbench
This research identifies a critical flaw in customer churn prediction models where standard adjacent-window comparisons conflate natural seasonality with genuine account decline, leading to significant false alarms. By formalizing this mechanism and proposing a baseline alignment strategy that compares current activity against the same calendar months from the prior year, the study demonstrates a substantial improvement in model precision and resource efficiency. This work offers essential methodological guidance for practitioners deploying machine learning systems in time-series dependent domains like B2B marketplaces.
Seasonal false alarms in customer churn and decline early-warning systems: adjacent-window labels confound seasonality with decline, and a year-over-year correction
arXiv:2608.18174v1 Announce Type: cross Abstract: Customer decline early-warning systems feed account-manager action lists, and every flagged account consumes intervention capacity. In a deployed business-to-business marketplace system, one action-list slot in three went to flags that dissolve under a seasonally aligned label. The standard target in non-contractual churn prediction compares an entity's next k months of activity with its trailing k months. The two windows cover different calendar months, so for seasonal entities the threshold-ratio construction confounds seasonality with decline, and the event rate depends on the label's anchor calendar month. We formalize the mechanism and measure it on three public panels and the production panel (the public arms pre-specified). Of the adjacent-window decay events, 37--69% on the public panels and 28--50% in production have no counterpart under a seasonally aligned definition. Pooling anchors, the standard remedy, balances the training mixture but corrects no individual label. Measured alternatives repair the curve only partially or change the detection horizon. Aligning the baseline to the same k calendar months one year prior -- advice practitioners already state, here formalized, measured, and costed -- flattens the curve at the source. With the classifier held fixed, production holdout ROC-AUC rises from 0.767 to 0.864 for decline (different targets; the gap closes on synthetic ground truth and a production hindsight referee). The served action list shrinks by a third, 119 to 79 accounts. The measured price is extra history, a blind spot to decline-then-stabilization, and a stricter cut under sustained growth.
Abstract:Customer decline early-warning systems feed account-manager action lists, and every flagged account consumes intervention capacity. In a deployed business-to-business marketplace system, one action-list slot in three went to flags that dissolve under a seasonally aligned label. The standard target in non-contractual churn prediction compares an entity's next k months of activity with its trailing k months. The two windows cover different calendar months, so for seasonal entities the threshold-ratio construction confounds seasonality with decline, and the event rate depends on the label's anchor calendar month. We formalize the mechanism and measure it on three public panels and the production panel (the public arms pre-specified). Of the adjacent-window decay events, 37--69% on the public panels and 28--50% in production have no counterpart under a seasonally aligned definition. Pooling anchors, the standard remedy, balances the training mixture but corrects no individual label. Measured alternatives repair the curve only partially or change the detection horizon. Aligning the baseline to the same k calendar months one year prior -- advice practitioners already state, here formalized, measured, and costed -- flattens the curve at the source. With the classifier held fixed, production holdout ROC-AUC rises from 0.767 to 0.864 for decline (different targets; the gap closes on synthetic ground truth and a production hindsight referee). The served action list shrinks by a third, 119 to 79 accounts. The measured price is extra history, a blind spot to decline-then-stabilization, and a stricter cut under sustained growth.
34 pages, 4 figures. Replication package included as ancillary files
Applications (stat.AP); Machine Learning (cs.LG); Methodology (stat.ME)
arXiv:2608.18174 [stat.AP]
(or arXiv:2608.18174v1 [stat.AP] for this version)
https://doi.org/10.48550/arXiv.2608.18174 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
REPLICATION_PLAN.md
output/e35_seasonal_label_confound_synthetic_20260725T103727Z.json
output/e36_m5_seasonal_label_confound_20260725T104144Z.json
output/e37_tourism_seasonal_label_confound_20260725T103854Z.json
output/e38_online_retail_seasonal_label_confound_20260725T104018Z.json
output/e41_label_remedy_arms_deseason_k12_public_20260726T192858Z.json
output/e42_common_referee_latent_truth_20260726T082348Z.json
output/e43_disagreement_decomposition_public_20260726T192912Z.json
output/e44_tabfm_latent_truth_20260726T152241Z.json
output/e48_entity_year_bootstrap_dependence_public_20260727T114735Z.json
output/e49_multiseed_drift_world_robustness_public_20260727T120413Z.json
output/e50_public_panel_classifier_truth_20260728T114724Z.json
output/e51_truth_form_sensitivity_public_classifier_20260728T114757Z.json
output/e52_blindspot_eval_variant_20260728T151515Z.json
output/e54_aligned_added_events_decomposition_public_20260817T062319Z.json
scripts/_label_confound_protocol.py
scripts/_provenance.py
scripts/build_label_confound_figures.py
scripts/compare_label_remedies_deseason_k12.py
scripts/decompose_aligned_added_events.py
scripts/decompose_label_disagreements.py
scripts/evaluate_labels_against_latent_truth.py
scripts/evaluate_tabfm_zero_shot_on_latent_truth.py
scripts/probe_blindspot_eval_variant.py
scripts/probe_entity_year_bootstrap_dependence.py
scripts/probe_multiseed_drift_world_robustness.py
scripts/probe_public_panel_classifier_truth.py
scripts/probe_truth_form_sensitivity_public_classifier.py
scripts/replicate_seasonal_label_confound_on_m5_public.py
scripts/replicate_seasonal_label_confound_on_online_retail_b2b.py
scripts/replicate_seasonal_label_confound_on_tourism_public.py
scripts/reproduce_seasonal_label_confound_synthetic.py
(28 additional files not shown) You must enabled JavaScript to view entire file list.
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 394 cuvinte
