Discovering Relationships in Data Lakes Using Large Language Models: An Industrial Case

Abstract:Data lakes rely on metadata to remain usable, yet this meta data is often limited or weakly informative for column relationship discovery, especially in ERP-derived datasets with coded or abbreviated schema labels. We propose ColRel, a two-stage method that builds column embeddings from metadata and data available at ingestion time. In difficult cases, such as coded schemata, business dictionaries help better interpret column names and support the generation of short natural-language descriptions used in the second stage. Experiments on public benchmarks and an industrial ERP dataset show that ColRel is particularly effective in semantically related, weak-signal settings.

Artificial Intelligence (cs.AI)

arXiv:2608.26750 [cs.AI]

(or arXiv:2608.26750v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2608.26750 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

28th International Conference on Big Data Analytics and Knowledge Discovery (DaWaK 2026), Aug 2026, Graz, Austria. pp.116-130

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.