Exploiting the Alternatives: Coordinated Learning via Hierarchical RL for Dynamic VNEAP

Abstract:Virtual Network Embedding (VNE) is a key enabler of network slicing, yet most formulations assume that each Virtual Network Request (VNR) has a fixed topology. Recently, VNE with Alternatives (VNEAP) was introduced to capture malleable VNRs, where each request can be instantiated using one of several functionally equivalent topologies that trade resources differently. This flexibility can improve embedding feasibility, but only if the orchestrator can jointly select suitable alternatives and embed them under dynamic arrivals. This paper proposes HRL-VNEAP, a hierarchical reinforcement learning approach for dynamic VNEAP. A high-level policy selects the most suitable alternative topology (or rejects the request), and a low-level policy embeds the chosen topology onto the substrate network. Experiments on realistic substrate topologies under varying arrival rates show that naive exploitation strategies provide only modest gains, whereas HRL-VNEAP outperforms state of the art approaches, improving acceptance ratio by up to 22%, and net profit by up to 20%. An offline MILP upper bound is also used on tractable instances to quantify the remaining optimality gap.

This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Networking and Internet Architecture (cs.NI); Machine Learning (cs.LG); Multiagent Systems (cs.MA)

arXiv:2512.05207 [cs.NI]

(or arXiv:2512.05207v3 [cs.NI] for this version)