Recent paper showing how LMs "struggle with less popular factual knowledge, and that scaling fails to appreciably improve memorization of factual knowledge in t

Recent paper showing how LMs "struggle with less popular factual knowledge, and that scaling fails to appreciably improve memorization of factual knowledge in the tail". In other words, simply scaling LLMs will not result in replacing Google as a search engine. Approaches like theirs ie retrieval augmentation are needed. https://akariasai.github.io/files/llm_memorization.pdf https://akariasai.github.io/files/llm_memorization.pdf This is essentially what I'm building for the content marketing vertical.