Rethinking the Relationship between the Power Law and Hierarchical Structures

Abstract:Statistical analysis of corpora provides an approach to quantitatively investigate natural languages. This approach has revealed that several power laws consistently emerge across different corpora and languages, suggesting universal mechanisms underlying languages. In particular, the power-law decay of correlations has been interpreted as evidence of underlying hierarchical structures in syntax, semantics, and discourse. This perspective has also been extended beyond corpora produced by human adults, including child speech, birdsong, and chimpanzee action sequences. However, the argument supporting this interpretation has not been empirically tested in natural languages. To address this gap, the present study examines the validity of the argument for syntactic structures. Specifically, we test whether the statistical properties of parse trees align with the assumptions in the argument. Using English and Japanese corpora, we analyze the mutual information, deviations from probabilistic context-free grammars (PCFGs), and other properties in natural language parse trees, as well as in the PCFG that approximates these parse trees. Our results indicate that the assumptions do not hold for syntactic structures and that it is difficult to apply the proposed argument not only to sentences by human adults but also to other domains, highlighting the need to reconsider the relationship between the power law and hierarchical structures.

Accepted for publication in Transactions of the Association for Computational Linguistics (TACL). This is a pre-MIT Press publication version. v4: Corrected a typo in an author name

Computation and Language (cs.CL)

arXiv:2505.04984 [cs.CL]

(or arXiv:2505.04984v4 [cs.CL] for this version)

https://doi.org/10.1162/TACL.a.785 DOI(s) linking to related resources