The Librarian Who Refused to Code: Model-Dependent Identity Enactment in LLM Code Generation

Abstract:Biographical personas are widely used in system prompts, but their effects on code generation are rarely evaluated under controlled, pre-registered conditions. We tested four prompt conditions (no persona, two engineer personas, and a research-librarian persona), 12 code-generation tasks, two frontier models, and five runs per cell (480 completions). Persona effects differed between the two tested models. Under the pre-registered mixed-effects analysis, the condition-by-model interaction was significant for provider-reported output tokens; a post-hoc visible-character measure showed the same qualitative pattern. Six GPT-5.5 completions were length-capped and are reported separately. On Claude Opus, the minimalist engineer persona reduced visible output by 30% (33% in provider tokens) without improving correctness, while the thorough engineer persona increased output without a correctness gain. In an exploratory post-hoc analysis, the librarian persona elicited in-character disclaimers in 55 of 60 Opus responses and 12 genuine no-code responses, lowering mean correctness from 0.92 to 0.67. GPT-5.5 produced neither behavior in its 59 non-truncated responses. These results are consistent with personas acting as Model-Dependent behavioral-policy biases rather than universal quality interventions. We release raw completions, derived scores, analysis artifacts, a pre-registration document, and an execution gate log; end-to-end test-based rescoring requires an unreleased task harness.

17 pages, 3 tables, no figures. Ancillary files include raw completions, derived scores, analysis scripts, persona texts, and preregistration

Computation and Language (cs.CL); Software Engineering (cs.SE)

arXiv:2607.17420 [cs.CL]

(or arXiv:2607.17420v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2607.17420 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

LOCKED_MODELS.txt

analysis/analyze_main.py

analysis/c2_by_cell.csv

analysis/identity_enactment.py

analysis/mixed_effects.json

analysis/mixed_effects_corrected.json

analysis/recompute_corrected.py

analysis/requirements.txt

analysis/results.csv

analysis/score_responses.py

analysis/scored_responses.csv

personas/P0_baseline.txt

personas/P_A_maya.txt

personas/P_B_ron.txt

personas/P_C_linnea.txt

preregistration/05_preregistration.md

runs/main/main_claude-opus-4-8_20260622T145939232171Z.jsonl

runs/main/main_gpt-5_5_20260622T145642342609Z.jsonl

(18 additional files not shown) You must enabled JavaScript to view entire file list.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.