论文标题
部分可观测时空混沌系统的无模型预测
Nonparametric Decoding for Generative Retrieval
论文作者
论文摘要
生成检索模型仅取决于没有外部内存的模型参数中编码的信息，其信息容量有限且固定。为了克服限制，我们提出了非参数解码（NP解码），可以应用于现有的生成检索模型。 NP解码使用非参数上下文化的词汇嵌入（外部内存），而不是香草词汇嵌入作为解码器词汇嵌入。通过利用上下文化的词汇嵌入，生成检索模型能够同时利用参数和非参数空间。在文档检索任务中，对9个数据集（8个单跳和1个多跳）进行评估表明，将NP解码应用于生成检索模型可显着提高性能。我们还表明，NP解码是数据和参数有效的，并且在零拍设置中显示出高性能。
The generative retrieval model depends solely on the information encoded in its model parameters without external memory, its information capacity is limited and fixed. To overcome the limitation, we propose Nonparametric Decoding (Np Decoding) which can be applied to existing generative retrieval models. Np Decoding uses nonparametric contextualized vocab embeddings (external memory) rather than vanilla vocab embeddings as decoder vocab embeddings. By leveraging the contextualized vocab embeddings, the generative retrieval model is able to utilize both the parametric and nonparametric space. Evaluation over 9 datasets (8 single-hop and 1 multi-hop) in the document retrieval task shows that applying Np Decoding to generative retrieval models significantly improves the performance. We also show that Np Decoding is data- and parameter-efficient, and shows high performance in the zero-shot setting.
