analiza aiqbench
This research bridges queueing theory and machine learning by quantifying how predictive errors in AI models impact system-wide congestion costs. The authors propose an index-based scheduling policy that optimally incorporates predicted job classes to minimize delays, specifically illustrated through toxicity classification for content moderation tasks. These findings offer a rigorous framework for designing service systems where AI-driven triage must balance accuracy with operational efficiency under heavy traffic conditions.
Design and Scheduling of an AI-based Queueing System
arXiv:2406.06855v3 Announce Type: replace-cross Abstract: To leverage prediction models to make optimal scheduling decisions in service systems, we must understand how predictive errors impact congestion due to externalities on the delay of other jobs. Motivated by applications where prediction models interact with human servers (e.g., content moderation), we consider a large queueing system comprising of many single server queues where the class of a job is estimated using a prediction model. By characterizing the impact of mispredictions on congestion cost in heavy traffic, we design an index-based policy that incorporates the predicted class information in a near-optimal manner. Our theoretical results guide the design of predictive models by providing a simple model selection procedure with downstream queueing performance as a central concern, and offer novel insights on how to design queueing systems with AI-based triage. We illustrate our framework on a content moderation task based on real online comments, where we construct toxicity classifiers by finetuning large language models.
Abstract:To leverage prediction models to make optimal scheduling decisions in service systems, we must understand how predictive errors impact congestion due to externalities on the delay of other jobs. Motivated by applications where prediction models interact with human servers (e.g., content moderation), we consider a large queueing system comprising of many single server queues where the class of a job is estimated using a prediction model. By characterizing the impact of mispredictions on congestion cost in heavy traffic, we design an index-based policy that incorporates the predicted class information in a near-optimal manner. Our theoretical results guide the design of predictive models by providing a simple model selection procedure with downstream queueing performance as a central concern, and offer novel insights on how to design queueing systems with AI-based triage. We illustrate our framework on a content moderation task based on real online comments, where we construct toxicity classifiers by finetuning large language models.
Optimization and Control (math.OC); Machine Learning (cs.LG)
arXiv:2406.06855 [math.OC]
(or arXiv:2406.06855v3 [math.OC] for this version)
Text integral sub licența deschisă a publicației · 169 cuvinte
Subiecte conexe
Spur: Modern GPU Job Scheduling for HPC and AI Workloads
AMD · 22 iul. 2026
Hybrid Quantum and Classical Workload Management with Graph-based Scheduling
Quantum Zeitgeist · 13 iul. 2026
Multi-Agent Reinforcement Learning for Online Traffic Scheduling in Time-Sensitive Application
arXiv · 7 aug. 2026
Lend and borrow resources across teams with AWS Batch’s quota management for SageMaker Training jobs
AWS · 29 oct. 2026