analiza aiqbench
The paper releases a domain-specific named entity recognition dataset of 659 car listings with over 10,000 expert-annotated entities, and benchmarks rule-based extraction, fine-tuned encoders, and LLMs. Fine-tuned DeBERTa reaches 90% micro-F1, ahead of the strongest LLM at 77.8%, which suggests task-specific encoders still beat general-purpose models on structured extraction. The corpus is small and drawn from a single car-selling site, so the results may not transfer to other listing sources without further testing.
AutoSpecNER: A Fine-Grained Named Entity Recognition Dataset for Vehicle Specification Extraction
arXiv:2606.24387v2 Announce Type: replace Abstract: Vehicle advertisements contain rich specification information, but automotive NER resources remain limited. We introduce AutoSpecNER, an expert-annotated dataset for fine-grained entity recognition in vehicle listings. The dataset includes 659 advertisements from a popular car-selling website, with over 10,000 entities annotated across 15 categories, including MODEL, ENGINE_SPEC, and BATTERY_CAPACITY. Annotation quality was validated through inter-annotator agreement, achieving an average score of 91.5%. We benchmark rule-based extraction, fine-tuned transformer encoders, and large language models. DeBERTa achieves the best performance with a 90% micro-F1 score, outperforming the rule-based baseline (43%) and the strongest large language model (77.8%).
Abstract:Vehicle advertisements contain rich specification information, but automotive NER resources remain limited. We introduce AutoSpecNER, an expert-annotated dataset for fine-grained entity recognition in vehicle listings. The dataset includes 659 advertisements from a popular car-selling website, with over 10,000 entities annotated across 15 categories, including MODEL, ENGINE_SPEC, and BATTERY_CAPACITY. Annotation quality was validated through inter-annotator agreement, achieving an average score of 91.5%. We benchmark rule-based extraction, fine-tuned transformer encoders, and large language models. DeBERTa achieves the best performance with a 90% micro-F1 score, outperforming the rule-based baseline (43%) and the strongest large language model (77.8%).
Computation and Language (cs.CL)
arXiv:2606.24387 [cs.CL]
(or arXiv:2606.24387v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.24387 Focus to learn more arXiv-issued DOI via DataCite
Proceedings of the 9th International Conference on Natural Language and Speech Processing (ICNLSP-2026), pp. 67-81, 2026
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 203 cuvinte