analiza aiqbench
This study investigates whether large language models possess human-like inductive biases by testing their ability to learn and distinguish between typologically attested languages and constructed "impossible" ones across 12 diverse linguistic families. The findings reveal that while GPT-2 small can largely separate real from impossible languages, it fails to perfectly differentiate all attested variants or specific word orders based on Greenberg's Universal 20 without generalization tasks. These results suggest a nuanced gap between model architecture and human cognitive constraints, offering critical insights into the limits of current machine learning approaches to linguistics.
Anything Goes? A Crosslinguistic Study of (Im)possible Language Learning in LMs
arXiv:2502.18795v4 Announce Type: replace Abstract: Do language models (LMs) offer insights into human language learning? A common argument against this idea is that because their architecture and training paradigm are so vastly different from humans, LMs can learn arbitrary inputs as easily as natural languages. We test this claim by training LMs to model impossible and typologically unattested languages. Unlike previous work, which has focused exclusively on English, we conduct experiments on 12 languages from 4 language families with two newly constructed parallel corpora. Our results show that while GPT-2 small can largely distinguish attested languages from their impossible counterparts, it does not achieve perfect separation between all the attested languages and all the impossible ones. We further test whether GPT-2 small distinguishes typologically attested from unattested languages with different NP orders by manipulating word order based on Greenberg's Universal 20. We find that the model's perplexity scores do not distinguish attested vs. unattested word orders, while its performance on the generalization test does. These findings suggest that LMs exhibit some human-like inductive biases, though these biases are weaker than those found in human learners.
Abstract:Do language models (LMs) offer insights into human language learning? A common argument against this idea is that because their architecture and training paradigm are so vastly different from humans, LMs can learn arbitrary inputs as easily as natural languages. We test this claim by training LMs to model impossible and typologically unattested languages. Unlike previous work, which has focused exclusively on English, we conduct experiments on 12 languages from 4 language families with two newly constructed parallel corpora. Our results show that while GPT-2 small can largely distinguish attested languages from their impossible counterparts, it does not achieve perfect separation between all the attested languages and all the impossible ones. We further test whether GPT-2 small distinguishes typologically attested from unattested languages with different NP orders by manipulating word order based on Greenberg's Universal 20. We find that the model's perplexity scores do not distinguish attested vs. unattested word orders, while its performance on the generalization test does. These findings suggest that LMs exhibit some human-like inductive biases, though these biases are weaker than those found in human learners.
Computation and Language (cs.CL)
arXiv:2502.18795 [cs.CL]
(or arXiv:2502.18795v4 [cs.CL] for this version)
Text integral sub licența deschisă a publicației · 192 cuvinte