aiqbenchInformații despre AI & calcul cuantic

...

Română · beta. Traducerea a fost făcută automat și este încă în curs de revizuire. Citește pagina în engleză

perspectiva aiqbenchCercetare AI

This research demonstrates that visually grounding a bilingual LSTM model enhances its ability to understand semantic similarity across English and Spanish while reducing perplexity. However, the study notes that this visual advantage does not extend significantly to abstract concepts, highlighting current limitations in multimodal language modeling. The findings underscore the need for more naturalistic multilingual datasets with perceptual grounding to better mimic human learning environments.

Sursă care acoperăarXiv

Ce este aceastaLectura unui model asupra articolului și a acoperirii conexe, scrisă pentru aiqbench. Nu sunt cuvintele publicației.

Like a bilingual baby: The advantage of visually grounding a bilingual language model

arXiv28 iulie 20261 min de cititArticolul și perspectiva sunt în engleză.

arXiv:2210.05487v3 Announce Type: replace Abstract: Unlike most neural language models, humans learn language in a rich, multi-sensory and, often, multi-lingual environment. Current language models typically fail to fully capture the complexities of multilingual language use. We train an LSTM language model on images and captions in English and Spanish from MS-COCO-ES. We find that the visual grounding improves the model's understanding of semantic similarity both within and across languages and improves perplexity. However, we find no significant advantage of visual grounding for abstract words. Our results provide additional evidence of the advantages of visually grounded language models and point to the need for more naturalistic language data from multilingual speakers and multilingual datasets with perceptual grounding.

Abstract:Unlike most neural language models, humans learn language in a rich, multi-sensory and, often, multi-lingual environment. Current language models typically fail to fully capture the complexities of multilingual language use. We train an LSTM language model on images and captions in English and Spanish from MS-COCO-ES. We find that the visual grounding improves the model's understanding of semantic similarity both within and across languages and improves perplexity. However, we find no significant advantage of visual grounding for abstract words. Our results provide additional evidence of the advantages of visually grounded language models and point to the need for more naturalistic language data from multilingual speakers and multilingual datasets with perceptual grounding.

Computation and Language (cs.CL)

arXiv:2210.05487 [cs.CL]

(or arXiv:2210.05487v3 [cs.CL] for this version)

Text integral sub licența deschisă a publicației · 123 cuvinte

Citește pe arXiv ↗Întreabă despre acest articol