| 摘要: |
| 针对多信道动态频谱接入问题，建立了存在感知错误与接入碰撞的复杂信道场景，提出了一种结合双深度Q网络和竞争Q网络的竞争双深度Q网络学习框架。双深度Q网络将动作的选择和评估分别用不同值函数实现，解决了值函数的过估计问题，而竞争Q网络解决了神经网络结构优化问题。该方案保证每个次要用户根据感知和回报结果做出频谱接入决策。仿真结果表明，在同时存在感知错误和次要用户冲突的多信道情况下，竞争双深度Q网络相比于同类方法具有较好的损失预测模型，其回报更稳定且提高了4%。 |
| 关键词: 认知无线电 频谱感知 动态频谱接入 深度强化学习 竞争双深度Q网络 |
| DOI： |
|
| 基金项目:国家自然科学基金资助项目(61702066) |
|
| Dynamic spectrum access based on dueling double deep Q-network |
| LIANG Yan,HUI Ying |
| (1.School of Communication and Information Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China;2.Chongqing Key Laboratory of Signal and Information Processing,Chongqing 400065,China) |
| Abstract: |
| For the problem of multi-channel dynamic spectrum access,a complex channel scenario with perception errors and access collisions is established.A dueling double deep Q-network learning framework is proposed which combines double deep Q-network and dueling Q-network.The double deep Q-network solves the overestimation problem of the value function by implementing the selection and evaluation of actions with different value functions.Meanwhile the dueling Q-network solves the problem of neural network structure optimization.The proposed solution ensures that each secondary user makes a spectrum access decision based on the perception and reword results.The simulation results show that in the multi-channel situation with perception errors and secondary user conflicts,the dueling double deep Q-network has a better loss prediction model,whose reword is more stable and increased by 4% than that of similar methods. |
| Key words: cognitive radio spectrum sensing dynamic spectrum access deep reinforcement learning dueling double deep Q-network |
