Support Thresholds, Not Algorithms, Limit Rare-Association Recovery in Co-Purchase Networks

arXiv:2609.20171v1 Announce Type: new Abstract: The support threshold of the Apriori algorithm involves a trade-off in conducting market basket analysis: the associations that occur frequently are noted with high threshold; however, the low ones lead to generating the large amount of rules. The paper compares five methods for co-purchase edge filtration on two grocery datasets: i.e., Instacart (3.2 million baskets) and Dunnhumby (208 thousand baskets), including Apriori, Apriori + lift post-filtering, top-$K$ ranking based on lift, and two methods based on networks, noise-corrected (NC) and disparity filter (DF). The top-$K$ method ensures the maximum average lift, while the NC achieves similar lift level by means of a single value of the significance parameter ($\alpha$). These two methods recover substantially more rare high-lift associations than Apriori (80-100% against 22-28%). NC and top-$K$ select meaningfully different edges (18-29% non-overlapping): NC retains statistically validated pairs, while top-$K$ retains rare pairs with high lift but low statistical significance. A rolling-origin holdout evaluation shows that top-$K$ edges recur at higher rates at every split, but NC edges are ~12 pp more likely to remain statistically significant in the held-out network.

Abstract:The support threshold of the Apriori algorithm involves a trade-off in conducting market basket analysis: the associations that occur frequently are noted with high threshold; however, the low ones lead to generating the large amount of rules. The paper compares five methods for co-purchase edge filtration on two grocery datasets: i.e., Instacart (3.2 million baskets) and Dunnhumby (208 thousand baskets), including Apriori, Apriori + lift post-filtering, top-$K$ ranking based on lift, and two methods based on networks, noise-corrected (NC) and disparity filter (DF). The top-$K$ method ensures the maximum average lift, while the NC achieves similar lift level by means of a single value of the significance parameter ($\alpha$). These two methods recover substantially more rare high-lift associations than Apriori (80-100% against 22-28%). NC and top-$K$ select meaningfully different edges (18-29% non-overlapping): NC retains statistically validated pairs, while top-$K$ retains rare pairs with high lift but low statistical significance. A rolling-origin holdout evaluation shows that top-$K$ edges recur at higher rates at every split, but NC edges are ~12 pp more likely to remain statistically significant in the held-out network.

Machine Learning (cs.LG); Computational Engineering, Finance, and Science (cs.CE)

arXiv:2609.20171 [cs.LG]

(or arXiv:2609.20171v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2609.20171 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

Support Thresholds, Not Algorithms, Limit Rare-Association Recovery in Co-Purchase Networks

Related coverage