From Scientific Crop Imagery to Agronomic Reasoning: A Multimodal LLM Benchmark for Plant Phenotyping

arXiv:2604.09907v2 Announce Type: replace-cross Abstract: High-throughput, effective and comprehensive phenotyping is a critical prerequisite for crop improvement. While phenotypic analysis was traditionally performed manually, recent advances in multimodal foundation models, especially in vision-language models, have offered new opportunities for automated plant phenotyping. However, plant science remains particularly challenging for general-purpose VLMs because it requires domain-specific knowledge, fine-grained visual interpretation, and complex agronomic reasoning. We introduce PlantXpert, an evidence-grounded multimodal reasoning benchmark for plant phenotyping across four major crops: soybean, cotton, rice, and corn. Our benchmark provides a structured and reproducible framework for agronomic adaptation of VLMs, and enables controlled comparison between base models and their domain-adapted counterparts. We constructed a dataset comprising 545 digital images and 4,351 benchmark samples spanning key agronomic domains including disease, pest control, weed management and yield. The benchmark can assess diverse capabilities including visual expertise, quantitative reasoning, and multi-step agronomic reasoning. We trained and evaluated a total of 11 state-of-the-art VLMs. Fine-tuning on our dataset improves overall accuracy for all models, with many achieving around 70% accuracy, and improvements on 9 of 11 remaining statistically significant after Holm correction. We also identify diminishing returns from model scaling, uneven performance across crop subsets, and persistent weaknesses in quantitative reasoning and specialized image analysis. These limitations show that general multimodal competence alone is insufficient for reliable agronomic reasoning and demonstrate the value of domain-specific adaptation. We hope PlantXpert can serve as a foundation for advancing evidence-grounded multimodal reasoning in plant science.

Abstract:High-throughput, effective and comprehensive phenotyping is a critical prerequisite for crop improvement. While phenotypic analysis was traditionally performed manually, recent advances in multimodal foundation models, especially in vision-language models, have offered new opportunities for automated plant phenotyping. However, plant science remains particularly challenging for general-purpose VLMs because it requires domain-specific knowledge, fine-grained visual interpretation, and complex agronomic reasoning. We introduce PlantXpert, an evidence-grounded multimodal reasoning benchmark for plant phenotyping across four major crops: soybean, cotton, rice, and corn. Our benchmark provides a structured and reproducible framework for agronomic adaptation of VLMs, and enables controlled comparison between base models and their domain-adapted counterparts. We constructed a dataset comprising 545 digital images and 4,351 benchmark samples spanning key agronomic domains including disease, pest control, weed management and yield. The benchmark can assess diverse capabilities including visual expertise, quantitative reasoning, and multi-step agronomic reasoning. We trained and evaluated a total of 11 state-of-the-art VLMs. Fine-tuning on our dataset improves overall accuracy for all models, with many achieving around 70% accuracy, and improvements on 9 of 11 remaining statistically significant after Holm correction. We also identify diminishing returns from model scaling, uneven performance across crop subsets, and persistent weaknesses in quantitative reasoning and specialized image analysis. These limitations show that general multimodal competence alone is insufficient for reliable agronomic reasoning and demonstrate the value of domain-specific adaptation. We hope PlantXpert can serve as a foundation for advancing evidence-grounded multimodal reasoning in plant science.

Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

arXiv:2604.09907 [cs.CV]

(or arXiv:2604.09907v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2604.09907 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

analiza aiqbench

From Scientific Crop Imagery to Agronomic Reasoning: A Multimodal LLM Benchmark for Plant Phenotyping

Subiecte conexe