PATTERN Can Be Solved from Memory: Generator-Aware Baselines for Synthetic Node Classification

arXiv:2610.15020v1 Announce Type: new Abstract: PATTERN, a synthetic node-classification benchmark still used to rank graph networks, plants a random subgraph in a random host graph and asks which nodes belong to it. Its generator draws 100 patterns once and plants the same ones in every split, so each test graph carries a pattern that also appears in 100 training graphs. A matcher that reads these patterns off the training labels and looks them up in each test graph reaches 99.2 class-weighted accuracy with no trained parameters, against 89.8 for the best published model. Inference under the generator without the patterns reaches 86.8. Two published models, CKGConv and GRIT, score lower on patterns they have not seen: on new graphs with fresh patterns both fall below this pattern-blind predictor, and withholding patterns from CKGConv's training lowers its accuracy on the test graphs that carry them. Training CKGConv with a new pattern in every graph raises its accuracy on fresh patterns by about 2 points, to the level of the pattern-blind predictor, at some cost on the reused patterns. The same kind of inference under the generator matches the best published result on CLUSTER and, without any training, scores above every published result we found on Minesweeper's released board. We recommend that synthetic benchmarks report scores on fresh templates beside reused ones, publish baselines built from their generators, and state how their metric is weighted and pooled.

Abstract:PATTERN, a synthetic node-classification benchmark still used to rank graph networks, plants a random subgraph in a random host graph and asks which nodes belong to it. Its generator draws 100 patterns once and plants the same ones in every split, so each test graph carries a pattern that also appears in 100 training graphs. A matcher that reads these patterns off the training labels and looks them up in each test graph reaches 99.2 class-weighted accuracy with no trained parameters, against 89.8 for the best published model. Inference under the generator without the patterns reaches 86.8. Two published models, CKGConv and GRIT, score lower on patterns they have not seen: on new graphs with fresh patterns both fall below this pattern-blind predictor, and withholding patterns from CKGConv's training lowers its accuracy on the test graphs that carry them. Training CKGConv with a new pattern in every graph raises its accuracy on fresh patterns by about 2 points, to the level of the pattern-blind predictor, at some cost on the reused patterns. The same kind of inference under the generator matches the best published result on CLUSTER and, without any training, scores above every published result we found on Minesweeper's released board. We recommend that synthetic benchmarks report scores on fresh templates beside reused ones, publish baselines built from their generators, and state how their metric is weighted and pooled.

Machine Learning (cs.LG)

arXiv:2610.15020 [cs.LG]

(or arXiv:2610.15020v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2610.15020 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

PATTERN Can Be Solved from Memory: Generator-Aware Baselines for Synthetic Node Classification

Related coverage