Call admission control and routing in integrated services networks using neuro-dynamic programming

We consider the problem of call admission control (CAC) and routing in an i ntegrated services network that handles several classes of calls of differe nt value and with different resource requirements. The problem of maximizin g the average value of admitted calls per unit time (or of revenue maximiza tion) is naturally formulated as a dynamic programming problem, but is too complex to allow for an exact solution. We use methods of neuro-dynamic pro gramming (NDP) [reinforcement learning (RL)], together with a decomposition approach, to construct dynamic (state-dependent) call admission control an d routing policies. These policies are based on state-dependent link costs, and a simulation-based learning method is employed to tune the parameters that define these link costs. A broad set of experiments shows the robustne ss of our policy and compares its performance with a commonly used heuristi c.