Design of speech recognition system based on attention mechanism

Abstract

Attention mechanism is to let the machine pay attention to more key information and ignore the secondary information in complex tasks, that is, to assign different weights to different information to represent different degrees of attention so as to reduce the amount of calculation. The common end-to-end speech recognition model structure is to directly model speech and text, which not only simplifies the speech recognition model but also improves the recognition performance. In order to study the function of attention mechanism in the end-to-end speech recognition system, this paper will take wenet as the basic network framework, and study the conformer part of the encoder and the transformer structure of the decoder in the end-to-end basic framework. These two structures make full use of the attention mechanism.