analiza aiqbench
DeInfer addresses a critical gap in decomposed LLMs by optimizing parallel inference performance, which existing methods often neglect when scaling model size. By integrating multiple optimizations compatible with state-of-the-art techniques, the system offers a practical solution for improving throughput in distributed computing environments. This work is particularly relevant for engineers and researchers focused on the infrastructure challenges of scaling large language model architectures.
DeInfer: Efficient Parallel Inferencing for Decomposed Large Language Models
arXiv:2604.17709v2 Announce Type: replace Abstract: Existing works on large language model (LLM) decomposition mainly focus on improving performance on downstream tasks, but they ignore the poor parallel inference performance when trying to scale up the model size. To mitigate this important performance issue, this paper introduces DeInfer, a high-performance inference system dedicated to parallel inference of decomposed LLMs. It consists of multiple optimizations to maximize performance and be compatible with state-of-the-art optimization techniques. Extensive experiments are carried out to evaluate DeInfer's performance, where the results demonstrate its superiority, suggesting it can greatly facilitate the parallel inference of decomposed LLMs.
Abstract:Existing works on large language model (LLM) decomposition mainly focus on improving performance on downstream tasks, but they ignore the poor parallel inference performance when trying to scale up the model size. To mitigate this important performance issue, this paper introduces DeInfer, a high-performance inference system dedicated to parallel inference of decomposed LLMs. It consists of multiple optimizations to maximize performance and be compatible with state-of-the-art optimization techniques. Extensive experiments are carried out to evaluate DeInfer's performance, where the results demonstrate its superiority, suggesting it can greatly facilitate the parallel inference of decomposed LLMs.
Computation and Language (cs.CL); Distributed, Parallel, and Cluster Computing (cs.DC)
arXiv:2604.17709 [cs.CL]
(or arXiv:2604.17709v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2604.17709 Focus to learn more arXiv-issued DOI via DataCite
https://doi.org/10.1145/3770743.3804360 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 203 cuvinte