论文标题
回归变压器：分子语言建模的并发序列回归和生成
Regression Transformer: Concurrent sequence regression and generation for molecular language modeling
论文作者
论文摘要
尽管自然科学中生成模型的进展很大，但它们的可控性仍然具有挑战性。分子或蛋白质生成模型的一个根本缺失的方面是一种感应性偏差，可以反映感兴趣的连续特性。为此，我们提出了回归变压器（RT），这是一种新的方法，将回归作为条件序列建模问题提取。这引入了多任务语言模型的新范式，该模型无缝桥接序列回归和条件序列的产生。 我们彻底证明，尽管使用了名义尺度训练目标，但RT在小分子，蛋白质和化学反应的财产预测任务中匹配或超过了常规回归模型的性能。至关重要的是，使用连续性能启动相同的模型产生了高度竞争性的有条件生成模型，该模型优于下属构成的，属性驱动的分子生成基准中的专业方法。我们的二分法方法是通过一种新颖的交替训练方案来促进的，该方案使模型能够通过所需的特性（例如，优化反应产量）来装饰种子序列。 总而言之，RT是多任务模型的第一个报告，该报告同时出现在生物化学中的预测和生成任务上。这发现了在物业驱动的，对化学或蛋白质空间的局部探索中的特殊应用，并且可以在材料设计中铺平道路。 复制本文所有实验的代码可在以下网址获得：https：//github.com/ibm/regress-transformer
Despite significant progress of generative models in the natural sciences, their controllability remains challenging. One fundamentally missing aspect of molecular or protein generative models is an inductive bias that can reflect continuous properties of interest. To that end, we propose the Regression Transformer (RT), a novel method that abstracts regression as a conditional sequence modeling problem. This introduces a new paradigm of multitask language models which seamlessly bridge sequence regression and conditional sequence generation. We thoroughly demonstrate that, despite using a nominal-scale training objective, the RT matches or surpasses the performance of conventional regression models in property prediction tasks of small molecules, proteins and chemical reactions. Critically, priming the same model with continuous properties yields a highly competitive conditional generative model that outperforms specialized approaches in a substructure-constrained, property-driven molecule generation benchmark. Our dichotomous approach is facilitated by a novel, alternating training scheme that enables the model to decorate seed sequences by desired properties, e.g., to optimize reaction yield. In sum, the RT is the first report of a multitask model that concurrently excels at predictive and generative tasks in biochemistry. This finds particular application in property-driven, local exploration of the chemical or protein space and could pave the road toward foundation models in material design. The code to reproduce all experiments of the paper is available at: https://github.com/IBM/regression-transformer
