| 摘要: |
| 基于强化学习（Reinforcement Learning,RL），在保证用户服务质量（Quality of Service,QoS）的前提下，研究了人机物混合接入的异构网络中通信-计算资源联合分配算法。建立了一种新型人机物混合接入的异构网络拓扑结构。在最小服务质量需求、无人机（Unmanned Aerial Vehicle,UAV）传输功率等限制条件下，将信道分配、功率分配和计算资源联合分配问题建模为最小化系统时延和能耗的多目标优化问题。基于强化学习理论和多智能体马尔可夫决策过程，提出一种分布式Q学习通信-计算资源联合分配（Distributed Q-learning Communication and Computing joint Resources Allocation,DQ-CCRA）算法。该算法与现有算法相比，不仅能够降低人类型设备对物类型设备的干扰，还能有效减小系统时延和能耗，将系统总开销降低7.4%。 |
| 关键词: 异构网络 人机物混合接入 资源分配 分布式Q学习 多无人机通信 |
| DOI：10.20079/j.issn.1001-893x.230607002 |
|
| 基金项目:国家自然科学基金资助项目（61901043）；北京市教育委员会科学研究计划项目（KM202211232010）；北京信息科技大学勤信人才培养计划（QXTCPB202101） |
|
| A Reinforcement Learning Based Joint Communication and Computing Resource Allocation Algorithm in Heterogeneous Networks |
| LI Liyan,LI Xuehua,CHEN Shuo,SUN Lixin |
| (a.Key Laboratory of Modern Measurement & Control Technology,Ministry of Education;b.Baicells Joint Laboratory of Intelligent and IoT,Beijing Information Science and Technology University,Beijing 102206 China) |
| Abstract: |
| Based on reinforcement learning (RL),the joint communication and computing resource allocation algorithm is studied,while ensuring the quality of service (QoS) of users. A new topology of heterogeneous network under hybrid access of human-machine-object type devices is proposed. The joint allocation problem of channel,power,and computational resource is modelled to minimize system delay and energy consumption with QoS and unmanned aerial vehicle (UAV) transmission power constraints.A distributed Q-learning communication and computing joint resources allocation (DQ-CCRA) algorithm is proposed based on RL and a multi-agent Markovian decision process. Compared to existing algorithms this approach reduces interference from human-type devices to object-type devices,lowers system costs up to 7.4%. |
| Key words: heterogeneous network hybrid access of human-machine-object type devices resource allocation distributed Q learning multi-UAV communication |
