Sari la conținut
aiqbenchInformații despre AI & calcul cuantic
ȘtiriAnalizeBenchmark-uriSkill-uriÎntreabă AIÎnvațăDespre
...Se încarcă sesiunea
ȘtiriAnalizeBenchmark-uriSkill-uriÎntreabă AIÎnvațăDespre

Română · beta. Traducerea a fost făcută automat și este încă în curs de revizuire. Citește pagina în engleză

Înapoi la listă
perspectiva aiqbenchCercetare AI

analiza aiqbench

This research evaluates Large Language Model steering techniques by measuring their alignment with human cognition using triadic similarity judgments, rather than relying solely on task-specific metrics. The study finds that prompt-based methods offer superior accuracy and better model-to-human alignment, while also revealing inherent biases in LLMs toward judging conceptual "kind" over physical "size." This work is significant for understanding the representational structures of steered models and improving their interpretability relative to human mental representations.
1
Sursă care acoperăarXiv

Evaluating Steering Techniques using Human Similarity Judgments

arXiv·9 septembrie 2026·2 min de citit·Articolul și perspectiva sunt în engleză.

De la arXiv

Text integral

arXiv:2505.19333v2 Announce Type: replace Abstract: Current evaluations of Large Language Model (LLM) steering techniques focus on task-specific performance, overlooking how well steered representations align with human cognition. Using a well-established triadic similarity judgment task, we assessed steered LLMs on their ability to flexibly judge similarity between concepts based on size or kind, two central dimensions organizing human mental representations. We found that prompt-based steering methods outperformed other methods both in terms of steering accuracy and model-to-human alignment.

We also found LLMs were biased towards `kind' similarity and struggled with `size' alignment. This evaluation approach, grounded in human cognition, adds further support to the efficacy of prompt-based steering and reveals privileged representational axes in LLMs prior to steering.

Abstract:Current evaluations of Large Language Model (LLM) steering techniques focus on task-specific performance, overlooking how well steered representations align with human cognition. Using a well-established triadic similarity judgment task, we assessed steered LLMs on their ability to flexibly judge similarity between concepts based on size or kind, two central dimensions organizing human mental representations. We found that prompt-based steering methods outperformed other methods both in terms of steering accuracy and model-to-human alignment.

We also found LLMs were biased towards `kind' similarity and struggled with `size' alignment. This evaluation approach, grounded in human cognition, adds further support to the efficacy of prompt-based steering and reveals privileged representational axes in LLMs prior to steering.

Artificial Intelligence (cs.AI)

arXiv:2505.19333 [cs.AI]

(or arXiv:2505.19333v2 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2505.19333 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Text integral sub licența deschisă a publicației · 204 cuvinte

Citește pe arXiv ↗Întreabă despre acest articol

Subiecte conexe

Cercetare AI

Evaluating the accuracy of KV cache reuse techniques

arXiv · 28 sept. 2026

Cercetare AI

Span-Level Machine Translation Meta-Evaluation

arXiv · 21 oct. 2026

Does preservation make sense before we know how to revive?

LessWrong · 15 iun. 2026

Siguranță AI

Improving Evaluation Realism with Inference-Time Compute and Deployment Scaffolds

arXiv · 3 sept. 2026

Article info

Source

arXiv

Published

9 septembrie 2026

Read time

2 min

Category

Cercetare AI
aiqbench
Informații despre AI & calcul cuantic© aiqbench. Toate drepturile rezervate.

Fluxuri:RSS·JSON Feed

LimbăEnglish·Românăbeta
ȘtiriAI & tehnologieCuanticAnalize
ExploreazăBenchmark-uriÎnvață
CompanieDespreÎncredereInformații legaleTermeni și condițiiPolitica de confidențialitateNotificări privind drepturile de autorȘtergerea datelorRaportează conținut

Perspectiva aiqbench, analizele, rezumatul zilnic și răspunsurile din Întreabă sunt scrise de modele AI din sursele afișate, fără revizuire editorială înainte de publicare. Pot conține erori — verifică sursa. Cum este folosit modelul →

  • Știri
  • Analize
  • Întreabă
  • Topuri
  • Cont