Retrieval-based models have a number of strengths that make them particularly important from my point of view: - as said they can be updated with fresh informa

Retrieval-based models have a number of strengths that make them particularly important from my point of view: - as said they can be updated with fresh information on the fly, - they can give you sources for their results (meaning that you can fact-check the output!), - they achieve similar performances for significantly smaller models (as long as you have a large dataset to retrieve from) meaning that they could run locally on consumer hardware (coupled with a large dataset on disk or the ability to read information from internet). If I were to spend a significant amount of time building my own language model, it would be retrieval-based and try to preserve those properties (in particular the ability to run on consumer hardware, enabling developers to run models locally has proved to be a game changer when stable diffusion came out).