| 摘要: |
| 传统的文本摘要方法，如基于循环神经网络和Encoder-Decoder框架构建的摘要生成模型等，在生成文本摘要时存在并行能力不足或长期依赖的性能缺陷，以及文本摘要生成的准确率和流畅度的问题。对此，提出了一种动态词嵌入摘要生成方法。该方法基于改进的Transformer模型，在文本预处理阶段引入先验知识，将ELMo(Embeddings from Language Models)动态词向量作为训练文本的词表征，结合此词对应当句的文本句向量拼接生成输入文本矩阵，将文本矩阵输入到Encoder生成固定长度的文本向量表达，然后通过Decoder将此向量表达解码生成目标文本摘要。实验采用Rouge值作为摘要的评测指标，与其他方法进行的对比实验结果表明，所提方法所生成的文本摘要的准确率和流畅度更高。 |
| 关键词: 文本摘要 Transformer模型 先验知识 动态词向量 句向量 |
| DOI： |
|
| 基金项目: |
|
| Text abstract generation based on improved Transformer model |
| WANG Kan,CAO Kaichen,XU Chang,PAN Yuanxiang,NIU Xinzheng |
| (Southwest China Institute of Electronic Technology，Chengdu 610036，China;School of Computer Science and Engineering，University of Electronic Science and Technology of China,Chengdu 610000,China;School of Information and Software Engineering，University of Electronic Science and Technology of China,Chengdu 610000,China) |
| Abstract: |
| The traditional text abstract methods,such as the abstract generation model based on the circular neural network and Encoder-Decoder framework,have the shortcomings of parallel ability,long-term dependent performance defects,as well as the accuracy and fluency problem in the generation of text abstracts.So,a dynamic word embedding abstract generation model(DWEM) is proposed.This method is based on the improved Transformer model.Firstly,prior knowledge is introduced in the text preprocessing stage,and the Embeddings from Language Models(ELMo) dynamic word vector is taken as the word representation of the training text.Secondly,the input text matrix is generated by combining this word with the text sentence vector of the corresponding sentence.The text matrix is then input into the Encoder to generate a fixed-length text vector representation.Finally,this vector is expressed and decoded by a Decoder to generate the target text abstract.In experiment,Rouge value is used as the evaluation index of the abstract,and the DWEM method is compared with other methods.The results show that the proposed method is more accurate and fluent. |
| Key words: text abstract Transformer model prior knowledge dynamic word vector sentence vector |
《电讯技术》编辑部 