Yes! I have long thought that GPT type of model are huge because they are forced to encode a lot of raw knowledge, giving them the ability to search for knowled

Yes! I have long thought that GPT type of model are huge because they are forced to encode a lot of raw knowledge, giving them the ability to search for knowledge in a database solves that problem which should help making them smaller while scaling to larger datasets. The cherry on top is that you could get not only information from the model but also the sources it used to make up its mind!