analiza aiqbench
This paper proposes a formal framework for structuring context units within LLM-based systems engineering, specifically addressing how to assemble modular prompts and evaluate the compliance of generated architecture models. By introducing methods to assess intent adherence in "modelling-as-code" outputs, it offers a rigorous approach to integrating generative AI into critical design workflows without relying on vague heuristics.
Context operations to architecture modelling output from large language models and evaluation criteria for their use in systems engineering design
arXiv:2609.10132v1 Announce Type: cross Abstract: The development of generative artificial intelligence resources enables opportunities of speeding up systems and engineering design work. This contribution introduces a framework of formal operations for assembling context in LLM-based engineering design. This framework involves the assembly of modular context units, including policy prompts, reference units with persistence, and user questions with prompt vectoring. This approach enables the systematic structuring of interactions with generative models. A formal method for evaluating modelling-as-code LLM outputs is also presented, which enables the evaluation of compliance to intent from LLM answers and thereby asses the support from LLMs for systems architecture modelling.
Abstract:The development of generative artificial intelligence resources enables opportunities of speeding up systems and engineering design work. This contribution introduces a framework of formal operations for assembling context in LLM-based engineering design. This framework involves the assembly of modular context units, including policy prompts, reference units with persistence, and user questions with prompt vectoring. This approach enables the systematic structuring of interactions with generative models. A formal method for evaluating modelling-as-code LLM outputs is also presented, which enables the evaluation of compliance to intent from LLM answers and thereby asses the support from LLMs for systems architecture modelling.
Systems and Control (eess.SY); Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
arXiv:2609.10132 [eess.SY]
(or arXiv:2609.10132v1 [eess.SY] for this version)
https://doi.org/10.48550/arXiv.2609.10132 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 199 cuvinte
