True, but in the context of computer science, isn't having encyclopedic knowledge a trivial problem? I mean, if you look at it that way, wikipedia itself "trans

True, but in the context of computer science, isn't having encyclopedic knowledge a trivial problem? I mean, if you look at it that way, wikipedia itself "transcends" the limits of the human brain, and it's just a database. And a way more efficient one at that, at 50GB vs. 700GB. In the context of AI, Bert already had knowledge embedded into it [1], and it's not considered a VLP model afaik. So it's more of a quantitative improvement, contrary to the claim of the author that there's a qualitative difference between VLP models and everything else. [1] https://syncedreview.com/2020/08/12/how-smart-is-bert-evaluating-the-language-models-commonsense-knowledge/ https://syncedreview.com/2020/08/12/how-smart-is-bert-evalua...