1、摘要

BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) has set a new state-of-the-art performance on sentence-pair regression tasks like semantic textual similarity (STS). However, it requires that both sentences are fed into the network, which causes a massive computational overhead: Finding the most similar pair in a collection of 10,000 sentences requires about 50 million inference computations (~65 hours) with BERT. The construction of BERT makes it unsuitable for semantic similarity search as well as for unsupervised tasks like clustering.

BERT (Devlin et al., 2018) 和 RoBERTa (Liu et al., 2019) 在語義文字相似性 (STS) 等句子對迴歸任務上取得了新的最先進的效能。 然而，它需要將兩個句子都輸入到網路中，這會導致大量的計算開銷：在 10,000 個句子的集合中找到最相似的一對需要使用 BERT 進行大約 5000 萬次推理計算（約 65 小時）。 BERT 的構建使其不適用於語義相似性搜尋以及聚類等無監督任務。

In this publication, we present Sentence-BERT (SBERT), a modification of the pretrained BERT network that use siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65 hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT.

在本出版物中，我們介紹了 Sentence-BERT (SBERT)，這是對預訓練 BERT 網路的一種修改，該網路使用 siamese 和三元組網路結構來推導語義上有意義的句子嵌入，可以使用餘弦相似度進行比較。 這將尋找最相似對的工作量從使用 BERT / RoBERTa 的 65 小時減少到使用 SBERT 的大約 5 秒，同時保持了 BERT 的準確性。

We evaluate SBERT and SRoBERTa on common STS tasks and transfer learning tasks, where it outperforms other state-of-the-art sentence embeddings methods.

我們在常見的 STS 任務和遷移學習任務上評估 SBERT 和 SRoBERTa，它優於其他最先進的句子嵌入方法。