论文标题

用于自动和可持续的边缘计算系统的多代理元提升学习

Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems

论文作者

Munir, Md. Shirajum, Tran, Nguyen H., Saad, Walid, Hong, Choong Seon

论文摘要

移动边缘计算(MEC)应用程序和功能的严格要求深入了解MEC主机在即将到来的无线网络中的高容量和密集部署。但是,运营如此高容量的MEC宿主可以显着增加能耗。因此,基站(BS)单元可以充当自动BS。在本文中,研究了具有边缘计算功能的自动无线网络的有效能源调度机制。首先,制定了两个阶段的线性随机编程问题,目的是在满足能源需求的同时最大程度地降低系统的总能耗成本。其次,通过开发新型的多代理元强化学习(MAMRL)框架来解决公式问题,提出了半分布的数据驱动解决方案。特别是,每个BS都扮演着当地代理的作用,该代理探讨了马尔可夫行为的能源消耗和发电,而每个BS都会将及时的特征转移到元代理。顺便说一句,元代理通过仅接受每个本地代理的观察结果,优化(即利用)能量调度决策。同时,每个BS代理商通过应用元代理的学习参数来估算其自己的能源调度策略。最后,提出的MAMRL框架是通过根据不可再生能源的使用,能源成本和准确性来分析确定性,不对称和随机环境来基准的。实验结果表明,与其他基线方法相比,提出的MAMRL模型可以降低11%的不可再生能源使用,而能源成本(预测准确性为95.8%)。

The stringent requirements of mobile edge computing (MEC) applications and functions fathom the high capacity and dense deployment of MEC hosts to the upcoming wireless networks. However, operating such high capacity MEC hosts can significantly increase energy consumption. Thus, a base station (BS) unit can act as a self-powered BS. In this paper, an effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. First, a two-stage linear stochastic programming problem is formulated with the goal of minimizing the total energy consumption cost of the system while fulfilling the energy demand. Second, a semi-distributed data-driven solution is proposed by developing a novel multi-agent meta-reinforcement learning (MAMRL) framework to solve the formulated problem. In particular, each BS plays the role of a local agent that explores a Markovian behavior for both energy consumption and generation while each BS transfers time-varying features to a meta-agent. Sequentially, the meta-agent optimizes (i.e., exploits) the energy dispatch decision by accepting only the observations from each local agent with its own state information. Meanwhile, each BS agent estimates its own energy dispatch policy by applying the learned parameters from meta-agent. Finally, the proposed MAMRL framework is benchmarked by analyzing deterministic, asymmetric, and stochastic environments in terms of non-renewable energy usages, energy cost, and accuracy. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost (with 95.8% prediction accuracy), compared to other baseline methods.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源