论文标题
使用快速的重量记忆学习关联推理
Learning Associative Inference Using Fast Weight Memory
论文作者
论文摘要
人类可以快速将刺激联系起来，以解决新的环境中的问题。我们的新型神经网络模型学习了可以构成以执行这种关联推断的事实的状态表示。为此，我们使用称为快速重量存储器（FWM）的关联内存增强LSTM模型。通过在给定输入序列的每个步骤中的可区分操作，LSTM更新并维护存储在快速变化的FWM权重中的组成关联。我们的模型是通过梯度下降对端到端训练的，在构图语言推理问题，POMDP的荟萃方面学习以及小规模的单词级语言建模方面产生了出色的表现。
Humans can quickly associate stimuli to solve problems in novel contexts. Our novel neural network model learns state representations of facts that can be composed to perform such associative inference. To this end, we augment the LSTM model with an associative memory, dubbed Fast Weight Memory (FWM). Through differentiable operations at every step of a given input sequence, the LSTM updates and maintains compositional associations stored in the rapidly changing FWM weights. Our model is trained end-to-end by gradient descent and yields excellent performance on compositional language reasoning problems, meta-reinforcement-learning for POMDPs, and small-scale word-level language modelling.
