R.01 · 長期記憶 · 2020
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela · NeurIPS 2020
Summary
Combines parametric memory (a pre-trained sequence-to-sequence model) with non-parametric memory (a dense vector index of Wikipedia, accessed by a neural retriever) for language generation. The authors note that providing provenance and updating a model’s knowledge remain open problems for purely parametric models.
CommentaryThe paper that made “parametric” and “non-parametric” memory common vocabulary — and named provenance and updating as the problems memory outside the model can address.