Writerslogic at PAN 2026: Process over Content for Robust Detection under Domain Shift

Abstract:We describe the Writerslogic systems for three PAN at CLEF 2026 shared tasks (Reasoning Trajectory Detection, Voight-Kampff Generative AI Detection, and Multi-Author Writing Style Analysis), unified by a shared analytical framework: feature robustness under distribution shift is governed by support overlap between training and test distributions, not by training-set effect size. This yields a taxonomy (domain-anchored, domain-portable, domain-invariant) that explains why generator-specific features die under domain shift while vocabulary fingerprints (hapax ratio, Yule's K, Heaps' exponent), compression measures, and character n-grams survive. On Reasoning Trajectory Detection, where training was entirely mathematics and 84 percent of test was unseen domains, the framework guided system design to 1st place in source detection (0.85 macro F1 via Opus-Sonnet agreement) and 3rd place in safety classification (0.66 macro F1 via query-refusal decomposition). For Voight-Kampff, we built a calibrated ensemble of DeBERTa-v2 (ONNX), multi-seed LightGBM with 44 domain-portable stylometric features, and SVM on n-gram TF-IDF, combined via learned stacking with isotonic calibration; the best configuration achieved 0.891 on the PAN 2026 test set with balanced sub-metrics (0.853 to 0.902 across all evaluation dimensions). For Multi-Author Writing Style Analysis, we describe a system fusing spectral clustering over character n-gram similarity graphs, normalized compression distance for local boundary detection, and SmolLM-135M perplexity for neural change-point detection; a platform mix-up meant our run never reached the official evaluation, so we report the design and its a priori predictions. Across all three tasks, features measuring generation process properties are designed to outperform features measuring generated content properties under domain shift.

13 pages, 1 figure, 6 tables. Notebook for the PAN Lab at CLEF 2026. Code this https URL and this https URL

Computation and Language (cs.CL)

arXiv:2610.03565 [cs.CL]

(or arXiv:2610.03565v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2610.03565 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

CLEF 2026 Working Notes, CEUR Workshop Proceedings, pp. 5474-5486

trajectory-detection-clef2026/CITATION.cff

trajectory-detection-clef2026/CODE_OF_CONDUCT.md

trajectory-detection-clef2026/CONTRIBUTING.md

trajectory-detection-clef2026/PRIVACY.md

trajectory-detection-clef2026/README.md

trajectory-detection-clef2026/SECURITY.md

trajectory-detection-clef2026/docs/FEATURE_DEATH.md

trajectory-detection-clef2026/docs/ITERATION_LOG.md

trajectory-detection-clef2026/docs/MULTILINGUAL_REFUSAL.md

trajectory-detection-clef2026/llm_ensemble/__init__.py

trajectory-detection-clef2026/llm_ensemble/api_classifier.py

trajectory-detection-clef2026/llm_ensemble/safety_classification.py

trajectory-detection-clef2026/llm_ensemble/source_detection.py

trajectory-detection-clef2026/predict.py

trajectory-detection-clef2026/pyproject.toml

trajectory-detection-clef2026/rtd/__init__.py

trajectory-detection-clef2026/rtd/ablation_capability_asymmetry.py

trajectory-detection-clef2026/rtd/data_loader.py

trajectory-detection-clef2026/rtd/evaluate.py

trajectory-detection-clef2026/rtd/features.py

trajectory-detection-clef2026/rtd/refusal_detector.py

trajectory-detection-clef2026/submissions/subtask1/submission.csv

trajectory-detection-clef2026/submissions/subtask1/submission_v2.csv

trajectory-detection-clef2026/submissions/subtask2/sub_chargram.csv

trajectory-detection-clef2026/submissions/subtask2/sub_chargram_highconf_65.csv

trajectory-detection-clef2026/submissions/subtask2/sub_chargram_refusal.csv

trajectory-detection-clef2026/submissions/subtask2/sub_ensemble_any_2.csv

trajectory-detection-clef2026/submissions/subtask2/sub_ensemble_vote_2of3.csv

trajectory-detection-clef2026/submissions/subtask2/sub_harmful_only.csv

trajectory-detection-clef2026/submissions/subtask2/sub_instruction_harmful_len600.csv

trajectory-detection-clef2026/submissions/subtask2/sub_instruction_no_len.csv

trajectory-detection-clef2026/submissions/subtask2/sub_instruction_or_harmful.csv

trajectory-detection-clef2026/submissions/subtask2/sub_instruction_or_len500.csv

trajectory-detection-clef2026/submissions/subtask2/sub_no_refusal_is_unsafe.csv

trajectory-detection-clef2026/submissions/subtask2/sub_query_AND_instruction.csv

trajectory-detection-clef2026/submissions/subtask2/sub_query_danger_no_refusal.csv

trajectory-detection-clef2026/submissions/subtask2/sub_query_or_instr_len600.csv

trajectory-detection-clef2026/submissions/subtask2/sub_query_or_instruction.csv

trajectory-detection-clef2026/submissions/subtask2/sub_refusal_or_len300.csv

trajectory-detection-clef2026/submissions/subtask2/sub_refusal_or_len500.csv

trajectory-detection-clef2026/submissions/subtask2/sub_refusal_v2_instruction.csv

trajectory-detection-clef2026/submissions/subtask2/sub_sbert_t10.csv

trajectory-detection-clef2026/submissions/subtask2/sub_sbert_t50.csv

trajectory-detection-clef2026/submissions/subtask2/sub_sbert_t60.csv

trajectory-detection-clef2026/submissions/subtask2/sub_sentence_2plus.csv

trajectory-detection-clef2026/submissions/subtask2/sub_shortval_600.csv

trajectory-detection-clef2026/submissions/subtask2/sub_shortval_800.csv

trajectory-detection-clef2026/submissions/subtask2/sub_trace_only_highconf_65.csv

trajectory-detection-clef2026/submissions/subtask2/sub_trace_only_lr.csv

trajectory-detection-clef2026/submissions/subtask2/submission.csv

trajectory-detection-clef2026/submissions/subtask2/submission_heuristic.csv

trajectory-detection-clef2026/submissions/subtask2/submission_tuned.csv

trajectory-detection-clef2026/tests/conftest.py

trajectory-detection-clef2026/tests/test_features.py

trajectory-detection-clef2026/tests/test_imports.py

trajectory-detection-clef2026/tests/test_refusal.py

trajectory-detection-clef2026/train_s1.py

trajectory-detection-clef2026/train_s2.py

trajectory-detection-clef2026/uv.lock

voight-kampff-clef2026/README.md

voight-kampff-clef2026/augment.py

voight-kampff-clef2026/calibrate_ensemble.py

voight-kampff-clef2026/download_onnx.py

voight-kampff-clef2026/features.py

voight-kampff-clef2026/finetune_deberta.py

voight-kampff-clef2026/main.py

voight-kampff-clef2026/predict_deberta_val.py

voight-kampff-clef2026/pyproject.toml

voight-kampff-clef2026/train.py

voight-kampff-clef2026/train_qwen.py

voight-kampff-clef2026/train_transformer.py

(71 additional files not shown) You must enabled JavaScript to view entire file list.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.