Hungarian layer: A novel interpretable neural layer for paraphrase identification

Paraphrase identification serves as an important topic in natural language processing while sequence alignment and matching underlie the principle of this task. Traditional alignment methods take advantage of attention mechanism. Attention mechanism, i.e. weighting technique, could pick out the most similar/dissimilar parts, but is weak in modeling the aligned unmatched parts, which are the crucial evidence to identify paraphrases. In this paper, we empower neural architecture with Hungarian algorithm to extract the aligned unmatched parts. Specifically, first, our model applies BiLSTM/BERT to encode the input sentences into hidden representations. Then, Hungarian layer leverages the hidden representations to extract the aligned unmatched parts. Last, we apply cosine similarity to metric the aligned unmatched parts for a final discrimination. Extensive experiments show that our model outperforms other baselines, substantially and significantly.