aiqbench analysis
This research demonstrates an effective integration of Large Language Models into machine learning workflows by automating feature engineering for tabular data. By combining LLMs with evolutionary algorithms, the study shows how language models can generate mathematically sound attributes that enhance classification performance across diverse datasets. This approach highlights a practical pathway to reduce manual effort in pipeline construction while leveraging world knowledge and logical reasoning inherent to modern AI systems.
Feature Generation Using LLMs: An Evolutionary Algorithm Approach
arXiv:2607.16255v1 Announce Type: cross Abstract: A crucial step in machine learning pipelines is to present each entity with features or attributes that are representative of the characteristics of the processed entities. Feature engineering is an important step in finding a relation among attributes that otherwise may not be processed by the ML algorithms. Meanwhile, Large Language Models have shown promising abilities in coding, mathematical reasoning, and processing world knowledge. In this work, we utilize an LLM for the problem of feature generation from tabular data based on the previously given features. We have created a pipeline that takes a set of attributes and a prompt to generate new features. Then, our selection algorithm selects the best-performing sets of attributes. We apply our method to eight datasets from different domains and data types. Our results show that, in most cases, the language model can produce new features based on mathematical and logical operators that are useful for the given tasks and can improve classification results.
Abstract:A crucial step in machine learning pipelines is to present each entity with features or attributes that are representative of the characteristics of the processed entities. Feature engineering is an important step in finding a relation among attributes that otherwise may not be processed by the ML algorithms. Meanwhile, Large Language Models have shown promising abilities in coding, mathematical reasoning, and processing world knowledge. In this work, we utilize an LLM for the problem of feature generation from tabular data based on the previously given features. We have created a pipeline that takes a set of attributes and a prompt to generate new features. Then, our selection algorithm selects the best-performing sets of attributes. We apply our method to eight datasets from different domains and data types. Our results show that, in most cases, the language model can produce new features based on mathematical and logical operators that are useful for the given tasks and can improve classification results.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2607.16255 [cs.LG]
(or arXiv:2607.16255v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2607.16255 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
Commun. Comput. Inf. Sci. 2471, 48-64 (2025)
https://doi.org/10.1007/978-3-031-89103-8_4 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 277 words
Related coverage
Rewrite All the Code, All the Time
LessWrong · Aug 4, 2026
Humanity’s Last Invention — Richard Socher of Recursive
Latent Space · Sep 14, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026
Towards Diverse Scientific Hypothesis Search with Large Language Models
arXiv · Jun 10, 2026