analiza aiqbench
This arXiv preprint proposes an LLM-based multi-agent system that routes prompts and deploys multimodal models across mobile edge servers, reporting 42.9% and 80.6% latency reductions against two baselines on a city-wide testbed. The results come from the authors' own testbed and workloads, so they are not yet independently validated. For AI readers, the main question is whether natural-language agent reasoning over telemetry can outperform conventional online-learning schedulers in real edge deployments, and how well it generalizes beyond this setup.
Multi-Agentic AI for Fairness-Aware and Accelerated Multi-modal Large Model Inference in Real-world Mobile Edge Networks
arXiv:2602.07215v2 Announce Type: replace-cross Abstract: Generative AI (GenAI) has transformed applications in natural language processing and content creation, yet centralized inference remains hindered by high latency, limited customizability, and privacy concerns. Deploying large models (LMs) in mobile edge networks emerges as a promising solution. However, it also poses new challenges, including heterogeneous multi-modal LMs with diverse resource demands and inference speeds, varied prompt/output modalities that complicate orchestration, and resource-limited infrastructure ill-suited for concurrent LM execution. In response, we propose a Multi-Agentic AI framework for latency- and fairness-aware multi-modal LM inference in mobile edge networks. Our solution includes a long-term Global Planning Agent, a short-term Prompt Scheduling Agent, and multiple on-node LM Deployment Agents, all powered by foundation language models. These agents cooperatively optimize prompt routing and LM deployment through natural language reasoning over runtime telemetry and historical experience. To evaluate its performance, we further develop a city-wide testbed that supports network monitoring, containerized LM deployment, intra-server resource management, and inter-server communications. Experiments demonstrate that our solution reduces average end-to-end latency by 42.9% compared with a state-of-the-art online-learning baseline and by 80.6% compared with the full-activation baseline, while maintaining a mean service-type Jain's index of 0.925. Moreover, our solution adapts quickly without fine-tuning, offering a generalizable solution for optimizing GenAI services in edge environments.
Abstract:Generative AI (GenAI) has transformed applications in natural language processing and content creation, yet centralized inference remains hindered by high latency, limited customizability, and privacy concerns. Deploying large models (LMs) in mobile edge networks emerges as a promising solution. However, it also poses new challenges, including heterogeneous multi-modal LMs with diverse resource demands and inference speeds, varied prompt/output modalities that complicate orchestration, and resource-limited infrastructure ill-suited for concurrent LM execution. In response, we propose a Multi-Agentic AI framework for latency- and fairness-aware multi-modal LM inference in mobile edge networks. Our solution includes a long-term Global Planning Agent, a short-term Prompt Scheduling Agent, and multiple on-node LM Deployment Agents, all powered by foundation language models. These agents cooperatively optimize prompt routing and LM deployment through natural language reasoning over runtime telemetry and historical experience. To evaluate its performance, we further develop a city-wide testbed that supports network monitoring, containerized LM deployment, intra-server resource management, and inter-server communications. Experiments demonstrate that our solution reduces average end-to-end latency by 42.9% compared with a state-of-the-art online-learning baseline and by 80.6% compared with the full-activation baseline, while maintaining a mean service-type Jain's index of 0.925. Moreover, our solution adapts quickly without fine-tuning, offering a generalizable solution for optimizing GenAI services in edge environments.
Systems and Control (eess.SY); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)
arXiv:2602.07215 [eess.SY]
(or arXiv:2602.07215v2 [eess.SY] for this version)
https://doi.org/10.48550/arXiv.2602.07215 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 310 cuvinte