论文标题
无监督的文本样式转移带有带垫的语言模型
Unsupervised Text Style Transfer with Padded Masked Language Models
论文作者
论文摘要
我们提出了Masker，这是一种无监督的样式传输的文本编辑方法。为了解决没有平行源目标对的情况，我们为源和目标域训练蒙面语言模型（MLMS）。然后，我们发现文本跨越两个模型在可能性方面最不同意的情况。这使我们能够识别源代币以删除以转换源文本以匹配目标域的样式。已删除的令牌被目标传销代替，并使用带衬垫的MLM变体，我们避免必须预先确定插入令牌的数量。我们对句子融合和情感转移的实验表明，掩体在完全无监督的环境中竞争性能。此外，在低资源设置中，它在Masker生成的银训练数据中预先培训时，将监督方法的精度提高了10个百分点。
We propose Masker, an unsupervised text-editing method for style transfer. To tackle cases when no parallel source-target pairs are available, we train masked language models (MLMs) for both the source and the target domain. Then we find the text spans where the two models disagree the most in terms of likelihood. This allows us to identify the source tokens to delete to transform the source text to match the style of the target domain. The deleted tokens are replaced with the target MLM, and by using a padded MLM variant, we avoid having to predetermine the number of inserted tokens. Our experiments on sentence fusion and sentiment transfer demonstrate that Masker performs competitively in a fully unsupervised setting. Moreover, in low-resource settings, it improves supervised methods' accuracy by over 10 percentage points when pre-training them on silver training data generated by Masker.
