| 摘要: |
| 认知用户通过频谱感知和接入过程识别频谱状态并占用空闲频谱，可有效利用频谱资源。针对频谱感知中存在感知错误和频谱接入中存在用户碰撞的问题，首先建立多用户多信道模型，设计频谱感知和频谱接入过程；然后通过结合双深度Q网络和竞争Q网络，设计竞争双深度Q网络，解决过估计问题的同时优化网络结构；最后通过智能体与所设计模型中状态、观测、回报和策略的交互，完成使用竞争双深度Q网络解决频谱感知和接入问题的一体化研究。仿真结果表明，相比于已有深度强化学习方法，使用竞争双深度Q网络得到的数值结果更稳定且感知正确率和信道利用率都提高了4%。 |
| 关键词: 频谱感知 频谱接入 深度强化学习 竞争双深度Q网络 |
| DOI：10.20079/j.issn.1001-893x.220418005 |
|
| 基金项目: |
|
| Spectrum Sensing and Access Based on Dueling Double Deep Q-network |
| LIANG Yan,HU Yaolin,HUI Ying |
| (1.School of Communication and Information Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China;2.Chongqing Key Laboratory of Signal and Information Processing,Chongqing 400065,China) |
| Abstract: |
| Cognitive users identify spectrum status and utilize idle spectrum through two processes of spectrum sensing and spectrum access,so that spectrum resources can be effectively utilized.For the problems of sensing errors in the process of spectrum sensing and user collisions in the process of spectrum access,a multi-user multi-channel model is established firstly for the design of spectrum sensing and access processes.Then by combing the double deep Q-network and dueling Q-network,a dueling deep double Q-network is designed to solve the problem of both overestimation and the network structure optimization.Finally,through the interaction between the agent and the state,observation,reward and strategy in the designed model,the integrated research on using dueling double deep Q-network to solve the problem of spectrum sensing and access is completed.The simulation results show that,compared with the existing deep reinforcement methods,the numerical results obtained by using the dueling double deep Q-network are more stable,and the perceptual accuracy and channel utilization are both improved by 4%. |
| Key words: spectrum sensing spectrum access deep reinforcement learning dueling double deep Q-network |
